• Project group for Computer Engineering and Computer Science students
• Course number: L.079.07059
• Terms: Summer semester 24 and Winter semster 24/25
• Further information: Project group presentation slide deck (July 2023)

Recently, using WiFi access points (APs) as sensor nodes by passively analyzing WiFi signals, particularly the Channel State Information (CSI), has attracted a lot of interest. This emerging technology is currently progressing towards standardization under IEEE 802.11bf, which is expected to be finalized around 2025. Of particular interest is the detection and identification of humans, their activities, motion, gestures, and other relevant information.

Motivated by previous successes in vision and language processing domains, current trends indicate a shift from classical signal processing approaches towards deep learning based methods. However, these learning based methods rely on the availability of suitable training datasets and might cause an increased demand for memory and computational resources due to growing model size and complexity.

Depending on the specific application, various considerations come into play. Latency becomes crucial for real-time applications such as motion and gesture recognition, where timely responses are required. Throughput becomes a concern when scaling the system to a large number of nodes, antennas, or high sample rates. Resource and energy efficiency are particularly important at the edge, where power consumption and computational resources are limited.

Using FPGAs allows for efficient deployment of such deep learning models by building custom hardware accelerators to match the latency, throughput and resource constraints of the application by leveraging model compression techniques and the parallelization potential of the FPGA accelerator. To the best of our knowledge, no such FPGA-based DNN accelerator for wireless human sensing has been employed so far.

The overall goal of this project is to explore the application of lightweight, FPGA-accelerated DNNs in the domain of wireless human sensing. The results will demonstrate how the DNN/FPGA co-design approach can benefit existing human sensing use cases or enable new application scenarios with the expected advantages in performance and energy-efficiency. The outcome will not only guide future research, but also include a working prototype implementation that serves as a practical technology demonstrator, which could be used in teaching or further student projects.

The project is of interdisciplinary nature and covers the complete stack from DNN training to the deployment of the final system. After an initial research phase into current literature around WiFi-based human sensing, students will analyze and select suitable use cases, datasets, and DNN topologies. Further, the group will work with experimental tooling (FINN) from AMD/Xilinx Research to train efficient DNNs via quantization-aware training and to generate custom-tailored FPGA inference accelerators for these DNNs. As part of the introduction to these tools, students will participate in a two-day workshop hosted in October as part of the EKI research project, which deals with the improvement and automation of such DNN-to-FPGA mapping tools in a data-center context.

The evaluation of the developed models and accelerators will initially focus on open-source datasets and report metrics such as classification/regression accuracy, throughput, latency, power consumption, and hardware cost in terms of FPGA resources. Depending on the results and challenges encountered during this phase of the project, the group selects at least one use case for a real-world prototype implementation, which might involve the generation of own datasets for fine-tuning. The hardware implementation for these experiments can rely on embedded FPGA boards and commodity WiFi access points. If desired, students could also implement the measurement infrastructure via a custom software-defined radio (SDR) solution based on an AMD/Xilinx RFSoC platform or move the DNN acceleration from edge to cloud by utilizing the large FPGA cards found in the PC² Noctua 2 cluster.

Students should bring knowledge or a deep interest in at least one of the following topics:

• Machine learning, specifically deep learning
• Signal processing or wireless communication networks
• FPGA design

It is not required but a plus, if you have experience with some of the following tools: (Embedded) Linux, Python, PyTorch, Brevitas, FINN, C/C++, Vitis HLS, Vivado, PYNQ, Matlab, Simulink, HDL Coder, Vitis Model Composer, or GNU Radio.

You will experience one of the fastest growing research topics, deep learning, from an exciting new angle. Co-designing model and hardware accelerator will broaden your horizons and give you a feel for the challenges associated with constrained embedded systems and lightweight neural networks. More specifically, you will build up skills in the following areas:

• Soft skills, literature research, and experiment management
• Data analysis, feature extraction, and pre-processing
• Lightweight DNN training using quantization and sparsity
• Working with high-level FPGA compilers (i.e., the FINN framework)
• Integrating sensor and accelerator components into an end-to-end system
• Designing signal processing pipelines on cutting-edge SDR platforms

If you need further information, do not hesitate to contact Felix Jentzsch or Christoph Berganski. You can find the slide deck of the project group presentation here.