Innovation by Design 2021 Awards Recognize HCII Projects

October 20, 2021
side by side images of Karan holding cellphone and sensor-generated version

 

Three HCII projects were recognized with four of Fast Company’s Innovation by Design 2021 Awards.

Researchers from the HCII’s Future Interfaces Group (FIG Lab) and Smart Sensing for Humans (SmaSH) Lab collaborated on these interdisciplinary projects, which explore new applications of sensors to enhance smart environments. 

“Current smart devices are oblivious to the user’s placement and pose, which can be extremely valuable to enable applications such as directional privacy-sensitive voice interfaces, full-body Animojis and embodied gaming interactions,” said Karan Ahuja, HCI PhD student who worked on two of the projects. “In our research, we aim to digitize the user’s body using sensor fusion techniques (Pose-on-the-Go) or sensing the acoustics of the environment (Direction-of-Voice).”

Learn more about the three projects below. 

Pose-on-the-Go: Approximating Partial User Pose with Smartphone Sensor Fusion and Inverse Kinematics 

Student Design finalist, Experimental Design honorable mention 

Pose-on-the-Go is a full-body pose estimation system that utilizes the sensors already found in today’s smartphones. This is in contrast to prior systems, which require additional worn or external sensors. The project uses extensive sensor fusion and leverages a phone’s front and rear cameras, the user-facing depth camera, touchscreen, and IMU. 

Researchers: Karan Ahuja, Sven Mayer, Mayank Goel, Chris Harrison

Project details: Pose-on-the-Go 

 

Direction-of-Voice (DoV) Estimation for Intuitive Speech Interaction with Smart Devices Ecosystems 

Student Design honorable mention

Modern homes and offices have a variety of IoT devices, many of which are activated by voice. In addition to receiving and processing spoken content, this work proposes that these devices can also infer the Direction of Voice (DoV). Such DoV estimation innately enables voice commands with addressability, in a similar way to gaze, but without the need for cameras. 

Researchers: Karan Ahuja, Andy Kong, Mayank Goel, Chris Harrison

Project details: Direction-of-Voice Estimation project 

 

City-Scale Sensing With Laser Vibrometry

Experimental Design finalist 

Although smart cities can provide rich data, it usually comes with the added costs of installing and maintaining new sensors. This project explores a novel application of laser vibrometry while utilizing the city’s existing retroreflective materials (commonly found as street signs, construction barriers, road studs and license plates) at very long ranges and use them as unpowered accelerometers for use in a wide variety of sensing applications. 

Researchers: Yang Zhang, Sven Mayer, Jesse T. Gonzalez, Chris Harrison

Project details: City-Scale Sensing With Laser Vibrometry