Payal Mohapatra

Payal Mohapatra

AI Researcher (Audio + Time Series)

About me

I am a third year PhD candidate in Computer Engineering and a part of IDEAS Lab at Northwestern University, advised by Dr. Qi Zhu.

I enjoy working on real world human-centric applications using methods inspired from computer science and health research. I investigate machine-learning methods for real-world applications with a focus on audio, healthcare and generally time series sensor data. These applications are ubiquitous and pose unique challenges in data analyses and learning. My focus is to develop robust learning algorithms for data (quality, quantity, and labels) and resource constrained applications. I am currently interested in exploring design trade-offs based on personalization and generalization of an algorithm for a system. Another aspect I wish to drive through my research is inclusivity in technology. I am interested in solving problems faced by the minority users (atypical speakers and voice assisted technology, darker skin-tone subjects and optical heart rate monitors). I want to emphasize on model explainability and resource efficiency in addition to model accuracy as a key performance index in my research.

Prior to joining Northwestern, I worked as an IC design engineer with Analog Devices Inc., where I developed metric driven and formal verification methods for application specific ICs aimed at industrial and consumer electronics. I have a Masters by Research in Electrical Engineering from Indian Institute of Technology, Madras during which I designed sensing and algorithms framework for cardiac wearables.

You can reach me at payalmohapatra2026 at u dot northwestern dot edu.
Link to my CV (last updated on February 2024)

Ongoing Projects


Efficient Multimodal Disfluency Detection

Curated a multi-modal (audio-visual-text) dataset for speech disfluency. Exploring low-footprint techniques to translate large-scale multi-modal speech-focused models in a resource-constrained paralinguistic task for unreliable data (noisy/missing modalities).


Human-Centred
AI

Working on low-resource personalisation on real-world sensor datasets which address the practical challenges of missing data in multi-variate inputs, continual learning for categorical target beyond support-set etc. Understanding commercial foundation models' capability to conduct zero-shot evaluation and consequently studying effective few-shot techniques for heterogenous input.

News

  • [March, 2024] - Submited a paper on multimodal speech disfluency detection to InterSpeech.
  • [February, 2024] - Submited a paper on my internship work with Meta RL on efficient event detection on smart glasses to MobileHCI.
  • [January, 2024] - Submited a paper on phase driven domain generalization for time series to ICML.
  • [October, 2023] - Corresponding with Meta Reality Labs, Audio Research group as a part-time student researcher.
  • [July, 2023] - Our paper on the Effect of Attention and Self-Supervised Speech Embeddings on Non-Semantic Speech Tasks has been accepted for ACM Multimedia 2023 Multimedia Grand Challenges Track.
  • [June, 2023] - I will be interning with Meta Reality Labs, Redmond, WA this summer as a Research Scientist. Reach out if you are here and want to collaborate (or just catch up over coffee).
  • [May, 2023] - We are participating in the ACM Multimedia 2023 Computational Paralinguistics Challenge (ComParE).
  • [February, 2023] - Our paper on EFFICIENT STUTTERING EVENT DETECTION USING SIAMESE NETWORKS is accepted in ICASSP’23.
  • [February, 2023] - Secured third place in e-Prevention: Person Identification and Relapse Detection from Continuous Recordings of Biosignals Challenge in ICASSP’23. Invited to present a paper on methodology - PERSON IDENTIFICATION WITH WEARABLE SENSING USING MISSING FEATURE ENCODING AND MULTI-STAGE MODALITY FUSION.
  • [December, 2022] - Demonstrated final working prototype of fatigue prediction in workers with near-real-time visualisation at the Boeing, Everett, Washington, factory floor.
  • [August, 2022] - Demonstrated initial working prototype of fatigue prediction in workers with near-real-time visualisation at the John Deere, Knoxville, Tennesse, factory floor.
  • [July, 2022] - Presented Speech Disfluency Detection with Contextual Representation and Data Distillation at Intelligent Acoustic Systems and Applications co-located with MobiSys’22.
  • [April, 2022] - Presented poster on Speech Disfluency Detection under data-constraints at CRA-WP grad cohort at New Orleans, Louisiana.

Services

  • Reviewer : ICASSP’24, Cambridge University Press early reader’s reviewing (Why Does Math Work … If It’s Not Real?)
    • External Reviewer - ASP-DAC’24, EMSOFT’23, ICCPS’23, NSys’22
  • Organize biweekly inter-laboratory Cyber-Physical Systems study group at Northwestern University.
  • I participate in the weekly ML reading group at Northwestern University. Here are a few topics I have discussed here - XGBoost, MultiModal Learning beyond audio-vision, Non-Stationary Transformers
  • Mentoring
    • 2023 :
      • Yueyuan Sui, Shamika Likhite (MS students, Computer Engineering, Northwestern University)
      • Brooks Hu, Kiva Joseph ((Undergraduate students, Computer Engineering, Northwestern University))
      • Jonathan Li Chen, Ben Forbes, Justin Lau (Undergraduate students, Mechanical Engineering(Mentored for a course project on sensor data analysis for injury detecion), Northwestern University)
    • 2022 : Devashri Naik, Jinjin Cai (MS, Computer Engineering, Northwestern University)

Manuscripts under preparation

  • Wearable Network for Multi-Level Physical Fatigue Prediction in Manufacturing Workers