Project Information
Award numbers: ECCS-2225337
Project period: January 1, 2023 – December 31, 2025 (Estimated)
Principal Investigators
Huacheng Zeng (PI)
Jeffrey Nanzer (Co-PI)
Graduate Students
Undergraduate Students
Brendan Bushbaker (5/2023 – 7/2023)
Project Overview
A scalable wireless system that can continuously, accurately, and simultaneously track multiple people's movements and detect their fine-grained body motions will enable the development of many novel real-life applications, such as diagnosing movement disorders, continuously monitoring children's behaviors, and alerting caregivers of elderly falls in real time. The objective of this project is to design and implement such a wireless tracking system by employing radio frequency (RF) tag based wireless sensing technology. A team of researchers from Michigan State University will address the fundamental challenges in both theory and experiments through innovations in signal processing, mathematical modeling, algorithm and protocol design, machine learning, optimization, radar design, and harmonic tag fabrication. The wireless tracking system will make it possible to monitor the activities of elderly residents in nursing homes while preserving their privacy, predict and prevent their possible falls, and guide them to perform doctor-suggested exercises. It will also enable the continuous monitoring of children's behaviors and activities in a non-invasive manner, making it possible to conduct data-driven studies of children's physical and mental development.
Project Goal
The overarching goal of this project is to design and implement a scalable wireless tag tracking system that features large coverage area, distributed sensing, inter-reader coordination, simultaneous multi-tag tracking, and high resolution in time. It comprises three complementary research thrusts. The first thrust is to develop fine-grained tracking solutions for a large-scale tag tracking system by coordinating and synchronizing location-distributed radio frequency (RF) readers for precise localization. It will develop new techniques to maximize the time resolution of tag tracking through optimizing signal processing algorithms, inter-reader communication and coordination protocols, tag scheduling, and resource allocation. The second thrust is to develop techniques to enable simultaneous multi-tag tracking similar to the idea of multi-user MIMO (multiple-input and multiple-output) in cellular networks. It will significantly improve the time resolution of tag tracking, especially in tag-dense scenarios. The third thrust focuses on motion tracking using narrowband harmonic tags and micro-Doppler radar. It will advance the design and fabrication of harmonic RF tags and radars for frequency-selective multi-tag operations, and explore the performance limits of harmonic tags in tracking applications. In addition, this project includes a strong component of system implementation and experimental evaluation, with the ultimate goal of demonstrating its real-life applications.
Publications
SiWiS: Fine-grained human detection using single Wi-Fi device [PDF] [PowerPoint] [Demo]
K. Song*, Q. Wang*, S. Zhang*, and H. Zeng,
ACM MobiCom, 2024. [Acceptance rate: 19.1%]
Measurement of the Radial and Angular Velocity of Tagged Objects Using Interferometric Harmonic Micro-Doppler Radar
C. Hilton and J. A. Nanzer
IEEE International Microwave Symposium, 2024
RadSee: See your handwriting through walls using FMCW radar [PDF]
S. Zhang*, Q. Wang*, M. Gan*, Z. Cao, and H. Zeng
Network and Distributed System Security (NDSS) Symposium, 2024. [Acceptance rate: 19.8%]
Is driver on phone call? Mobile device localization using cellular signal [PDF]
S. Zhang*, H. Zeng, and Y. T. Hou
IEEE Journal on Selected Areas in Communications, vol. 42, no. 9, pp. 2339-2353, September 2024.
mReader: Concurrent UHF RFID tag reading [PDF]
H. Pirayesh*, S. Zhang*, and H. Zeng,
ACM MobiHoc 2023. [Acceptance rate: 21.9%]
Research Accomplishments
Year 2 - Harmonic antenna design, test and fabrication: Harmonic antennas have been a recent topic of research in RFID and multi-frequency antenna design due to the unique signature produced by the introduction of commercial off the shelf (COTS) components. Future human-computer interfacing (HCI) technologies for applications including IoT, contraband detection, and distributed frequency synchronization require accurate measurement and classification of the motion of people and objects in cluttered living spaces. Microwave radar technology is increasingly being used in such SNR limited applications since it provides good measurement accuracy in the detection of the motion of tagged objects compared to other approaches. The main challenge with using microwave radar systems in consumer-facing scenarios is the presence of environmental objects which generate large clutter responses in the received signal. Further addressing this challenge, harmonic tags and harmonic radar systems have been a focus of recent research, in which tags retransmit incident signals at a harmonic frequency, thereby separating the response from that of the clutter. Previous work at Michigan State University has explored the use of harmonic tags for object tracking in cluttered environments by measuring the harmonic micro-Doppler response of moving tagged objects and explored the use of narrowband tags for multi-object gesture classification. These designed tags have passed several refining stages as discussed later in this work, where multiple designs were presented prior to the current iteration with dimensions of 2 cm x 2 cm at a fundamental resonance frequency of 2.5 GHz.
Current narrowband antennas can be characterized as those antennas that have a fractional bandwidth less than 1% designated by the ratio of the antenna S11 -10 dB bandwidth compared to the design resonance frequency. Our initial designs focused on wire-based methods as shown below, where cascaded T-network based impedance matching was used to reduce the input impedance bandwidth of the designed antenna. The simple design of this method allowed for an easily tunable design. This results in an antenna that is capable of resonating both at the fundamental and harmonic frequencies of the received signal. However. This design requires low-error-tolerance capacitance and inductance values within the matching network, making the design difficult to realize in practical applications.
The limitations imposed by the initial design led to the introduction of the annular and split ring resonator antennas (ARRA/SRRA). These designs were principally based on a simple bandpass ring filter whose resonance frequency f_c=1/√LC, where the inductive and capacitive values, L and C, are dependent upon the radius and feed line gap of the ring, as designated by a pi-impedance network. These values allowed a simple model to be created of the ring antenna, showing a nearly linear correlation between the ring antenna gap angle and resonance frequency as a result of shifts in the arc length of the antenna. This method allowed the design of simple ring antennas as shown in the below figure that met bandwidth requirements of narrow-band while maintaining gain between 1-2 dBi. This is achieved via tuning the split-gap angle of the SRRA and integration of an inter-digital capacitor (IDC), allowing for the low-tolerance design challenges introduced by the previous dipole antenna to be addressed.
This design was then further refined and implemented in an RFID-based harmonic micro-Doppler radar system. This system requires frequency selectivity to separate individual micro-Doppler responses and thus the novelty narrow-band of the narrow-band design becomes very useful for this approach. The radar system transmits a continuous wave signal at the fundamental frequency, which illuminates the ring antenna. The signal is then passed through a non-linear COTS device and re-transmitted to the radar system where post-processing allows the harmonic frequency and frequency shift to be captured. This design required the implementation of a second antenna, as the ring antenna only resonates at the fundamental frequency; to accommodate for this design constraint, a nearly circular patch antenna was introduced inside of the ring antenna with the non-linear device as the port between the two elements. This is shown in the below figure, allowing for the physical design of the harmonic tag to remain physically small with a ka<0.2 while maintaining dual frequency operation. Furthermore, measurements were taken with multiple harmonic tags, demonstrating sufficient frequency isolation from one another and the surrounding environment. This is shown in the harmonic micro-Doppler spectra, a time-frequency response displaying the frequency shifts associated with movement of the harmonic tags. This is shown in the below figure.
Year 2 - Fine-grained Human Detection Using Radio Signal: Sub-6GHz radio sensing offers several compelling advantages, such as resilience to poor lighting conditions, privacy preservation, and the ability to see through walls. However, in indoor environments, the sub-6GHz ISM spectrum is heavily occupied by WiFi devices, leaving little available spectrum for sensing purposes. In this paper, we introduce SiWiS, a new approach to integrate radio sensing capabilities into individual WiFi devices for fine-grained human activity detection. SiWiS comprises two main components: (i) a new hardware component that can be easily installed on an offthe-shelf WiFi device, and (ii) a dual-branch deep neural network (DNN) optimized for concurrent human mask segmentation and pose estimation. We have built a prototype of SiWiS and installed it on a commercial WiFi router for evaluation. Extensive experimental results demonstrate a significant performance improvement over WiFi channel state information (CSI) based sensing methods. More importantly, zero-shot experiments confirm that SiWiS can be directly transferred to unseen real-world environments.
Year 2 - Mobile Device Localization using Radio Signal: The use of mobile phones while driving is a major source of distraction for vehicle drivers and has resulted in a large number of car accidents. While surveillance cameras can be used to detect the violation of phone use, they do not work well in some scenarios (e.g., darkness and blockage) and may raise privacy concerns. In this paper, we present PhoLoc, a roadside device to detect the violation of phone use in personal vehicles using the cellular signals emitted by cellphones. PhoLoc is equipped with two sensors: a multi-antenna radio receiver and a lowcost lidar. It jointly processes the multimodal data from the two sensors to estimate the relative location of a phone in a vehicle. The enabler of PhoLoc is a new near-field localization scheme, which is capable of estimating the location of a moving phone at a specific time moment by overhearing its cellular signals. We have built a prototype of PhoLoc and evaluated its performance in realistic scenarios. Experimental results show that PhoLoc achieves 4.2% false positive rate and 13.8% false negative rate in the detection of phone call violation.
Year 1 - Enable concurrent UHF RFID tag reading using MU-MIMO technology: UHF RFID tags have been widely used for contactless inventory and tracking applications. One fundamental problem with RFID readers is their limited tag reading rate. Existing RFID readers (e.g., Impinj Speedway) can read about 35 tags per second in a read zone, which is far from enough for many applications. In this project, we developed the first-of-its-kind RFID reader (mReader), which borrows the idea of multi-user MIMO (MU-MIMO) from cellular networks to enable concurrent multi-tag reading in passive RFID systems. mReader is equipped with multiple antennas for implicit beamforming in downlink transmissions. It is enabled by three key techniques: uplink collision recovery, transition-based channel estimation, and zero-overhead channel calibration. In addition, mReader employs a Q-value adaptation algorithm for medium access control to maximize its tag reading rate. We have built a prototype of mReader on USRP X310 and demonstrated for the first time that a two-antenna reader can read two commercial off-the-shelf (COTS) tags simultaneously. Numerical results further show that mReader can improve the tag reading rate by 45% compared to existing RFID readers.
Year 1 - Compact harmonic tag design: Our initial narrowband harmonic tag design is based on a split-ring resonant antenna tuned to a narrow bandwidth that concentrically surrounds a circular patch antenna. The ring is tuned to the fundamental frequency f1 while the tag is designed to be resonant at the first harmonic 2f1. Between the split ring and the patch is a diode. When the ring receives the incident radiation at f1 and inputs the signal to the diode, a set of harmonic frequencies is generated. By tuning the patch to the second harmonic, only this frequency is retransmitted back to the interrogating system. From this retransmitted harmonic signal, detection and tracking of the tag is possible. We previously implemented this design at frequencies of 2.5/5 GHz. The design was scaled to a higher frequency (5/10 GHz) to reduce the size of the tag and explore its usage in a scalable format. We recently designed multiple tags at adjacent frequency bands, which will be experimentally measured in the coming months. We have also explored the use of a frequency subsampling radar receiver, which samples the received signal spectrum at a rate lower than the Nyquist rate. This causes the signals received from the tags to fold down to the first Nyquist zone, requiring significantly less bandwidth. Since the tags are narrowband, the received signals are essentially tones, with small modulation due to the motion of the tag. Because the transmit frequencies are known, they can be designed to avoid interference in the first Nyquist zone. The result is a lower-cost receiver that can detect tags at a wider range of carrier frequency bands.
Year 1 - Wireless network coordination: Wireless clock synchronization is important for accurate distributed processing. We are building a distributed receive processing system in software-defined radios (SDRs) that will leverage wireless frequency and time alignment. The frequency alignment approach is based on the transmission of a frequency reference that is double sideband modulated onto a carrier frequency and distributed between nodes. The receiving node captures the signals and demodulates the reference via a self-mixing circuit; the reference frequency is then used as the input to a phase-locked loop to synchronize the clocks. We are currently implementing a two-node SDR-based system and will evaluate the coordination accuracy and the ability to support cooperative transmission and reception.
Broader Impacts
Year 2 - PI Zeng hosts undergraduate students to study integrated communication and sensing (ICS) solutions
PI Zeng’s INSS laboratory is hosting two undergraduate students from the Department of Computer Science and Engineering for an innovative research project focused on Integrated Communication and Sensing (ICS). This outreach activity aims to bridge the gap between theoretical knowledge and practical applications, fostering a collaborative environment where students can engage in cutting-edge research. The primary objective of this ICS project is to enhance the capabilities of a single WiFi device by integrating advanced sensing functionalities. The students embarked on a comprehensive study to explore various solutions that enable concurrent transmission and reception of radio signals. This dual functionality is crucial for developing sophisticated applications that can leverage existing wireless infrastructure while adding significant value through new sensing capabilities. The students conducted extensive literature reviews and feasibility analyses. They evaluated current methodologies and technologies that allow for simultaneous signal processing, focusing on the challenges and opportunities within the realm of WiFi devices. Building on their findings, the students then transitioned to the development of deep learning models tailored for novel applications. Among the applications they explored are human body skeleton estimation and mask segmentation. These applications not only demonstrate the practical implications of their research but also address real-world problems, such as improving human-computer interaction and enhancing security measures in various settings. As part of this outreach activity, the students participated in regular presentations and discussions with their peers, faculty, and industry professionals. This interaction is designed to promote knowledge sharing and provide feedback that refine their research direction. Additionally, the students obtained the opportunity to showcase their work at departmental seminars, encouraging other students to engage in research and explore the possibilities within the ICS field.
Year 1 – PI Zeng has hosted undergraduate research on Bluetooth device tracking: This project hosted an undergraduate research project for Brendan Bushbaker (an undergraduate student in the Department of Computer Science at Michigan State University). He learned Bluetooth communication protocols and signal processing pipeline in this project. The developed a website to track any Bluetooth devices (e.g., iPhone, iPad, smart watch, and EarPods) in the proximity of our custom-designed sensors. In the website, end users can access the information of Bluetooth device location and tracking. The screenshot of the website from this undergraduate research project can be found below.
Year 1 – PI Zeng has given talk on smart device tracking to undegraduate students at UDC (HBCU): PI Zeng gave a talk on “Location privacy risk of 5G smart devices” to the undergraduate students in the Department of Computer Science and Information Technology at the University of the District of Columbia, which is one of the Historically Black Colleges and Universities (HBCUs). During the talk, PI Zeng introduced the communication protocols and signal structure of smart devices such as smart watches, EarPods, phones, and smart glasses. Then, he demonstrated that without any modification on the smart devices, those smart devices can be tracked by leveraging the radio signals emitted by those devices. Students also learned how to use machine learning algorithms to improve the accuracy of device location and tracking in a large area, as well as the potential security threats poised by illegitimate tracking systems.
Year 1 – Co-PI Nanzer has given talks presenting research supported by this effort at the following locations:
IEEE MTT/AP Syracuse Joint Chapter (virtual)
Dalhousie University (virtual)
James Cook University, Cairns Australia
University of Queensland, Brisbane Australia
University of Adelaide, Adelaide Australia
Royal Melbourne Institute of Technology, Melbourne Australia
University of Technology Sydney, Sydney Australia
Third MARIE IRTG Scientific Seminar (Keynote Speaker) (virtual)
The University of Colorado at Boulder, Boulder CO
The Ohio State University, Columbus OH
University of Texas at Dallas, Dallas TX
IEEE Texas Symposium on Wireless & Microwave Circuits and Systems, Waco TX
EuCAP workshop on phased array antennas, Florence Italy
University of Illinois at Urbana-Champaign, Urbana IL
IEEE MTT-S Kerala Section (virtual)