Publications

2025

1. 

   (BEST PAPER AWARD) ViLA: Leveraging General-Purpose Audio for Training Vibration-Based Stadium Crowd Monitoring Models

   Yen Cheng Chang, Jesse Codling, Yiwen Dong, and 3 more authors

   In Proceedings of the 12th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Colorado School of Mines, Golden, CO, USA, 2025

   Abs DOI

   Crowd monitoring in sports stadiums is important to enhance public safety and improve audience experience. Existing approaches mainly rely on manual observation, cameras, and microphones, which can be disruptive and often raise privacy issues. Recently, floor vibration sensing has emerged as a less disruptive and more non-intrusive method for crowd monitoring in sports stadiums. However, because vibration-based crowd monitoring is newly developed, open-source datasets are lacking, making it challenging to develop data-driven models.In this paper, we introduce Vibration Leverages Audio (ViLA), a vibration-based crowd monitoring method that reduces the reliance on labeled data by pre-training with unlabeled cross-modality data. Specifically, ViLA is first pre-trained on general-purpose audio data in an unsupervised manner, and then fine-tuned with a limited amount of labeled vibration data in sensing domains. Through this approach, ViLA learns general spectral pattern representations from audio, then adapts this knowledge to vibrations. By leveraging general-purpose audio datasets, ViLA reduces the reliance on large quantities of domain-specific vibration data. This is particularly important in sensing environments characterized by high data variance and limited sensing durations, such as sports games. Our real-world experiments demonstrate that pre-training the vibration model using publicly available audio data (YouTube clips) achieved up to a 5.8X error reduction compared to the model without audio pre-training.

2. 

   Human vs. Machine: Comparing Urban Condition Classification Methods from Vehicular Vision

   Julia Gersey, Troy Zhong, Jiale Zhang, and 3 more authors

   In Proceedings of the 12th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Colorado School of Mines, Golden, CO, USA, 2025

   Abs DOI

   Urban street conditions such as cleanliness and infrastructure decay affect public health and community well-being, yet monitoring often relies on subjective human judgment. Existing machine learning approaches narrowly target single factors like trash detection or road damage and require large datasets or costly pipelines. To balance task generalization with limited data, we propose a lightweight, interpretable pipeline for holistic condition assessment from street-level imagery. Using 325 images from seven U.S. cities with ten annotators per image, we find conventional models plateau at  57% accuracy, but subjectivity-aware cleaning raises performance to  62%, surpassing the weakest human annotator. Our contribution is empirical and methodological: we show that explicitly modeling annotator subjectivity can yield more reliable gains in handling noisy, subjective datasets than increasing algorithmic complexity, positioning lightweight pipelines as practical first-pass classifiers for scalable urban monitoring.

3. 

   BEST POSTER AWARD Poster Abstract: Sniffing Out the City - Vehicular Multimodal Sensing for Environmental and Infrastructure Analysis

   Julia Gersey, Jatin Aggarwal, Jiale Zhang, and 2 more authors

   2025

   Abs

   Assessing urban infrastructure and environment quality at scale remains a challenge. This work presents a multimodal sensing framework that integrates computer vision-based infrastructure analysis with mobile air quality monitoring to explore urban conditions beyond the camera’s field of view. A vehicle-mounted system captures video data for semantic segmentation of roads and buildings while an air intake unit collects temperature, humidity, CO2, TVOC, and AQI levels. A preliminary test drive in Ann Arbor demonstrated expected correlations between CO2 and TVOC spikes in dense urban areas, providing a proof of concept for linking environmental sensing with visual urban analysis. Future work will refine sensor calibration, adaptive sampling strategies, and predictive modeling to improve accuracy and scalability.

2024

1. 

   FloHR: Ubiquitous Heart Rate Measurement using Indirect Floor Vibration Sensing

   Jesse R. Codling, Jeffrey D. Shulkin, Yen-Cheng Chang, and 5 more authors

   In Proceedings of the 11th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, Hangzhou, China, 2024

   Abs DOI

   Heart rate is one of the most critical metrics for human health. Most common methods for measuring human heart rate involve body contact, whether from wearable devices or manual measurement. However, such devices can cause discomfort to some patients. Past work for non-contact or remote heart rate measurement (e.g., camera or radio) is often limited by line-of-sight requirements that are not always possible in the real-world environment.This paper presents FloHR, an indirect heart rate monitoring system for human beings using heartbeat-induced floor vibrations. The key insight is that the human body generates a small wave of pressure and sound with each heartbeat. These are propagated as vibration through the structures the person is in contact with (e.g., a chair) and through the floor. FloHR then detects and interprets these small floor vibrations. We developed a highly sensitive vibration sensing system and heartbeat pattern modelling to identify these tiny vibrations among other body motions and ambient noise.We evaluated FloHR in a real home environment, demonstrating an average heart rate error similar to medical device standards on the floor near the subjects’ chair, and on the order of 10 beats per minute (bpm) on the floor 2 meters away from the subject.

2. Poster: Drive-by City Wide Trash Sensing for Neighborhood Sanitation Need

   Tomas Fernandez, Yen Cheng Chang, Jesse Codling, and 5 more authors

   In Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services, Minato-ku, Tokyo, Japan, 2024

   Abs DOI

   Computer vision has been used more ubiquitously in recent years to understand and measure the environment around us, particularly in our neighborhoods. However, many city-wide sensing applications using vision require large labeling efforts, making various applications difficult on a wide scale. We propose a framework for labeling and self-training of in-car video to detect trash on the roads. Our approach requires minimal manual labeling to identify items not meant to be in the street, sidewalk, or public places, from a front-viewing car camera. Our system provides each frame of a video with a score indicating the amount of trash. To prevent overfitting, due to minimal available data, we remove data with high certainty of trash from the training dataset. The results show that our prediction with manually labeled ground truth yield an R2 of 0.66.

2023

1. Demo Abstract: FreePulse Heart Rate Monitoring System using Ambient Structural Vibrations

   Jesse R Codling, Jeffrey D Shulkin, Yiwen Dong, and 4 more authors

   In Proceedings of the 22nd International Conference on Information Processing in Sensor Networks, San Antonio, TX, USA, 2023

   Abs DOI

   Heart rate is a critical metric for human cardiovascular health. Most common methods for measuring human heart rate involve wearable devices (e.g., electrocardiography, smart watches). However, such devices can cause discomfort to some patients, especially the elderly or young children. This paper presents FreePulse, a heart rate monitoring system for seated subjects using ambient vibrations. FreePulse builds on our past work using vibrations in the building structures around us to measure human activities and health. As people’s hearts beat, they push on the surfaces the body is touching, creating vibrations in those structures. We combine structure response characterization with human pulse modelling to identify these pulse-induced vibrations from ambient vibration. In testing, FreePulse has shown up to 96% pulse rate accuracy on average, competitive with consumer-grade wearable devices.

2. Poster Abstract: Vibration-Based Object Classification with Structural Response of Ambient Music

   Jiale Zhang, Shweta Pati, Jesse Codling, and 4 more authors

   In Proceedings of the 22nd International Conference on Information Processing in Sensor Networks, San Antonio, TX, USA, 2023

   Abs DOI

   Object classification is a vital technology that is widely used to track and identify misplaced and out-of-stock items in shopping centers. While there have been a number of studies utilizing various sensing modalities such as computer vision, RFID, and vibration sensors, these methods are limited in their use due to privacy concerns, scalability, and the inability to identify stationary objects. To overcome these limitations, we propose a novel active vibration-sensing approach for object classification by utilizing music as an excitation source. Different objects can induce different deformations of the surface and further change the surface structural response. Therefore, we leverage vibrations from music on a store shelf and measure the structural responses on the surface when different objects are placed. Our evaluation of a store shelf demonstrates that distinct object characteristics lead to unique vibration responses, enabling accurate classification of 98.6% accuracy in distinguishing five common store objects. This study provides a promising avenue for a reliable, privacy-preserving, and scalable object classification system in various settings beyond shopping centers.

2022

1. Deep-Learning-Enabled Microwave-Induced Thermoacoustic Tomography Based on Sparse Data for Breast Cancer Detection

   Jiale Zhang, Chenzhe Li, Weichao Jiang, and 3 more authors

   IEEE Transactions on Antennas and Propagation, 2022

   DOI

2021

1. Directly Controlling the Perceived Difficulty of a Shooting Game by the Addition of Fake Enemy Bullets

   Jiale Zhang

   In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 2021

   Abs DOI

   Adjusting the balance between the player’s game skill and the difficulty level is one of the most important factors to improve the player’s engagement. However, it is still quite rare to find works that aim to straightly control the subjective difficulty perceived by the player. Our research question is whether or not it is possible to control the perceived difficulty just by adding enemy objects that do not raise the actual difficulty level. To investigate this issue, we designed a simple shooting game with two ‘fake enemy bullets’: Unreachable Bullets and Non-collisionable Bullets, which do not damage the player character. The experiment suggests that the non-collisionable bullet can efficiently increase the perceived difficulty level but the unreachable bullet does not. Such a study in novel techniques that can control the perceived difficulty without changing the actual difficulty could contribute to both research and practices in game design.

2020

1. Broadband high-efficiency multiple vortex beams generated by aninterleaved geometric-phase multifunctional metasurface

   Dajun Zhang, Zhansong Lin, Ji Liu, and 4 more authors

   Opt. Mater. Express, Jul 2020

   Abs DOI

   Vortex beams have witnessed tremendous development in the pastdecade by exhibiting profound implications for both fundamental physicsand a multitude of novel engineering applications. In this work, broadbandhigh-efficiency multiple vortex beams with independent topological modesand inclination angles are generated leveraging an interleavedgeometric-phase multifunctional metasurface operating in a very broadbandfrequency range. A set of meta-atoms are elaborately engineered to offerbroadband high-efficiency complete phase control covering the entire2π range. Multiple geometric-phase sub-arrays implemented by thedesigned meta-atoms are synthesized into one metasurface via ashared-aperture interleaved manner, in which each sub-array can beindividually manipulated and serves as an independent channel forlaunching a vortex beam. According to the established design methodology,two vortex beams with topological modes of −1 and +2 anddistinct inclination angles are generated by one metasurface. Experimentalresults are provided to corroborate the proposed mechanism for multiplevortex beams generation, which exhibit broadband and high-efficiencyfeatures. The presented multifunctional metasurface paves the way for thegeneration of broadband high-efficiency multiple vortex beams in themicrowave, millimeter-wave and terahertz regions. This work is ofsignificance for high-capacity wireless communication applications,high-efficiency manipulation of electromagnetic waves, and novel design ofradar and imaging systems.

2. Optimizing energy efficiency of CNN-based object detection with dynamic voltage and frequency scaling

   Weixiong Jiang, Heng Yu, Jiale Zhang, and 3 more authors

   Journal of Semiconductors, Feb 2020

   Abs DOI

   On the one hand, accelerating convolution neural networks (CNNs) on FPGAs requires ever increasing high energy efficiency in the edge computing paradigm. On the other hand, unlike normal digital algorithms, CNNs maintain their high robustness even with limited timing errors. By taking advantage of this unique feature, we propose to use dynamic voltage and frequency scaling (DVFS) to further optimize the energy efficiency for CNNs. First, we have developed a DVFS framework on FPGAs. Second, we apply the DVFS to SkyNet, a state-of-the-art neural network targeting on object detection. Third, we analyze the impact of DVFS on CNNs in terms of performance, power, energy efficiency and accuracy. Compared to the state-of-the-art, experimental results show that we have achieved 38% improvement in energy efficiency without any loss in accuracy. Results also show that we can achieve 47% improvement in energy efficiency if we allow 0.11% relaxation in accuracy.

2019

1. Topological transformation OAM based on metasurface generated bessel beam

   Dajun Zhang, Jiale Zhang, Ji Liu, and 1 more author

   In 2019 International Symposium on Antennas and Propagation (ISAP), Feb 2019