Competition-style exam drives student innovation

Teams of CS students enthusiastically searching the Siebel Center atrium for hidden spy cameras could only mean one thing: it was the middle of Professor Elahe Soltanaghai’s CS 437 mid-term exam.

Over two class periods on October 17 and 19, teams of students in her Internet of Things (IoT) course searched the atrium, vying to be the first to locate hidden spy cameras by tracking wireless traffic signals sent from the camera to the router with custom-made localization algorithms. 

Before the midterm, students attended a series of in-class labs in preparation. Their course studies taught them how to build smart wireless security cameras. Then, they learned to manipulate these cameras into spy devices. Finally, they collaborated in small teams to construct wireless localization algorithms that would intercept wireless signal strength emanating from the camera’s WiFi packets to detect the presence of a spy camera and its location. 

“With the advances in smart systems, we are seeing more and more instances of these IoT devices being misused. This has been particularly problematic in hotel rooms and Airbnbs, with many reported incidents of cameras or microphones with wireless connectivity being used to spy on guests. So, the competition's main goal was to build a spy camera finder based on the wireless traffic of IoT devices,” Soltanaghai said. 

One or more cameras were hidden in the competition area on each exam day. Teams used a Raspberry Pi equipped with several sensors to collect data and were given one minute to search the atrium. Then, they returned to the classroom to analyze their data and make any necessary adjustments. At the end of each session, they reported the location of the camera that their algorithm estimated.

Each of the two sessions presented different challenges and varying results.

“The first day, the assumption was that the camera constantly transmitted traffic. So they didn’t have to trigger the camera since it was always sending traffic. They used the signal strength of the packets coming from the camera to locate it,” Soltanaghai said.

The first-day competition winners were Keerthana Nallamotu and Snigdha Gupta. The distinguishing aspect of their winning approach was the way they corrected errors in the raw IMU data.

"We designed a localization algorithm that used IMU data to detect user steps and joystick triggers from the user to detect turns. We could accurately locate the user's movement around the area from this information. We then found the time and location that the user walked closest to the spy camera based on the corresponding signal strengths coming from the hidden camera," Nallamotu said.

The second day of the competition presented a different challenge. This time, a camera only sent packets when it detected motion in its sight line. As a result, teams needed to locate cameras with incomplete RSSI information.

“Day two was a bit more challenging. This time, the cameras only transmitted network traffic when they detected motion. Students had to strategize how to walk around the building in real-time. For example, if you see traffic, that means you’re in the field of view of the camera, so they had to optimize the collection of information to create their localization algorithm,” Soltanaghai said.

Avery Plote and Nikunj Tyagi won on the second day of the competition. Plote described his team’s two-part winning approach.

"To optimize our data collection, we used the LED on the Raspberry Pi to indicate packet reception and strength. The color of the LED panel indicated how close we were. Then, we developed an algorithm to estimate the potential orientation of the camera and used it to narrow down the hidden camera location," Plote said. 

From this mapped data, Tyagi created an algorithm that enabled the team to estimate the camera's actual location.

Raymond Chen and Aryaman Dwivedi were the runner-up winners on both days.

Throughout the competition, students encountered distinct obstacles. Some challenges were expected, like working with noisy sensing data, typical of IoT devices. Different groups created varying strategies for solving this problem. Some used other auxiliary sensors, like a joystick on the Raspberry Pi, to filter out noises, while others used landmarks in the scene to improve the drifts in the sensors.

The exam produced unexpected challenges as well. On the second day, Soltanaghai added several decoy spying cameras to the scene. “The cues in the scene misled them in the way they collected the data. Instead of relying on the wireless data, they relied on seeing the decoys we had placed there. The groups that trusted their sensor functionalities and their developed algorithms won the competition," Soltanaghai said.

Soltanaghai hoped to meet course objectives by implementing a competition-style exam format while giving students practical, hands-on experience. 

“I truly enjoy working with our undergraduate students. They are motivated and like to engage with research and solve real-world problems. My goal was to design an undergraduate class that integrates my group’s research and advanced topics in wireless sensing and communication with course materials but in an accessible way. In fact, the competition topic was based on one of our research papers published at the 31st USENIX Security Symposium in August 2022,” Soltanaghai said.

In creating this exam, she hoped to bridge the gap between knowledge acquisition and practical application. “I was experimenting with whether it’s possible to design the class such that someone with zero background in wireless networking or embedded systems could still take the material, digest it, and participate in more advanced research. I am glad to see that we successfully built up interest in a couple of students who have signed up to do research in my group in the following semester,” Soltanaghai said.

Donations from industry partners Bosch Research and Texas Instruments enabled students to build the technology necessary for the challenge. Bosch Research donated the hardware kits needed for the labs, including the Raspberry Pi kits. Texas Instruments also donated resources for the final projects, including single-chip radar sensors.

Ultimately, the students enjoyed and benefitted from the competitive structure of the exam. 

“The competition element drove students–they wanted to win. I tested the same learning objectives that are typical of a midterm exam, but the format motivated students to push themselves and their teams to think deeply about the course materials and apply them toward solving real-world problems. At the same time, they had fun while studying and preparing for this exam,” Soltanaghai said.

Luke Jacobs is a first-year grad student in ECE and supports Professor Soltanaghai as a teaching assistant in her CS 437 course. He was impressed with the students’ innovation and dedication to solving the challenges during the competition.

“The groups that put in the effort to design a creative approach, whether that was a unique walking pattern or a system of communication between group partners, really owned their ideas. Whether or not their approach was successful, the groups that had thought deeply about the problem beforehand were anxious to see it work,” Jacobs said.

He found the competition exam format to be especially fitting for a lab-heavy class, like CS 437. 

“There aren’t any practice exams that students can take to prepare for an exam like this, so it forces students to think deeper about how the course content connects to the problem. In a traditional exam, the best outcome for students is a good grade, but for this type of exam, the best outcome is the satisfaction of knowing they can tackle a real-world problem,” Jacobs said.

Plote agreed and enjoyed seeing the systems they built in action. “Participating in this midterm was a lot of fun! This class has been great for digging into hardware and working with sensor systems. I would recommend this class to other students interested in WiFi sensing, radars, and IoT in general,” he said.