Jiale Zhang

Crowd monitoring in sports stadiums is important to enhance public safety and improve audience experience. Existing approaches mainly rely on manual observation, cameras, and microphones, which can be disruptive and often raise privacy issues. Recently, floor vibration sensing has emerged as a less disruptive and more non-intrusive method for crowd monitoring in sports stadiums. However, because vibration-based crowd monitoring is newly developed, open-source datasets are lacking, making it challenging to develop data-driven models.In this paper, we introduce Vibration Leverages Audio (ViLA), a vibration-based crowd monitoring method that reduces the reliance on labeled data by pre-training with unlabeled cross-modality data. Specifically, ViLA is first pre-trained on general-purpose audio data in an unsupervised manner, and then fine-tuned with a limited amount of labeled vibration data in sensing domains. Through this approach, ViLA learns general spectral pattern representations from audio, then adapts this knowledge to vibrations. By leveraging general-purpose audio datasets, ViLA reduces the reliance on large quantities of domain-specific vibration data. This is particularly important in sensing environments characterized by high data variance and limited sensing durations, such as sports games. Our real-world experiments demonstrate that pre-training the vibration model using publicly available audio data (YouTube clips) achieved up to a 5.8X error reduction compared to the model without audio pre-training.

Urban street conditions such as cleanliness and infrastructure decay affect public health and community well-being, yet monitoring often relies on subjective human judgment. Existing machine learning approaches narrowly target single factors like trash detection or road damage and require large datasets or costly pipelines. To balance task generalization with limited data, we propose a lightweight, interpretable pipeline for holistic condition assessment from street-level imagery. Using 325 images from seven U.S. cities with ten annotators per image, we find conventional models plateau at  57% accuracy, but subjectivity-aware cleaning raises performance to  62%, surpassing the weakest human annotator. Our contribution is empirical and methodological: we show that explicitly modeling annotator subjectivity can yield more reliable gains in handling noisy, subjective datasets than increasing algorithmic complexity, positioning lightweight pipelines as practical first-pass classifiers for scalable urban monitoring.

Assessing urban infrastructure and environment quality at scale remains a challenge. This work presents a multimodal sensing framework that integrates computer vision-based infrastructure analysis with mobile air quality monitoring to explore urban conditions beyond the camera’s field of view. A vehicle-mounted system captures video data for semantic segmentation of roads and buildings while an air intake unit collects temperature, humidity, CO2, TVOC, and AQI levels. A preliminary test drive in Ann Arbor demonstrated expected correlations between CO2 and TVOC spikes in dense urban areas, providing a proof of concept for linking environmental sensing with visual urban analysis. Future work will refine sensor calibration, adaptive sampling strategies, and predictive modeling to improve accuracy and scalability.

Heart rate is one of the most critical metrics for human health. Most common methods for measuring human heart rate involve body contact, whether from wearable devices or manual measurement. However, such devices can cause discomfort to some patients. Past work for non-contact or remote heart rate measurement (e.g., camera or radio) is often limited by line-of-sight requirements that are not always possible in the real-world environment.This paper presents FloHR, an indirect heart rate monitoring system for human beings using heartbeat-induced floor vibrations. The key insight is that the human body generates a small wave of pressure and sound with each heartbeat. These are propagated as vibration through the structures the person is in contact with (e.g., a chair) and through the floor. FloHR then detects and interprets these small floor vibrations. We developed a highly sensitive vibration sensing system and heartbeat pattern modelling to identify these tiny vibrations among other body motions and ambient noise.We evaluated FloHR in a real home environment, demonstrating an average heart rate error similar to medical device standards on the floor near the subjects’ chair, and on the order of 10 beats per minute (bpm) on the floor 2 meters away from the subject.

Object classification is a vital technology that is widely used to track and identify misplaced and out-of-stock items in shopping centers. While there have been a number of studies utilizing various sensing modalities such as computer vision, RFID, and vibration sensors, these methods are limited in their use due to privacy concerns, scalability, and the inability to identify stationary objects. To overcome these limitations, we propose a novel active vibration-sensing approach for object classification by utilizing music as an excitation source. Different objects can induce different deformations of the surface and further change the surface structural response. Therefore, we leverage vibrations from music on a store shelf and measure the structural responses on the surface when different objects are placed. Our evaluation of a store shelf demonstrates that distinct object characteristics lead to unique vibration responses, enabling accurate classification of 98.6% accuracy in distinguishing five common store objects. This study provides a promising avenue for a reliable, privacy-preserving, and scalable object classification system in various settings beyond shopping centers.