Problem

The addition of a camera to the TV ecosystem corresponding to the recent popularity of video conferencing provided an opportunity to introduce another medium of interaction with the television. Leveraging hands-free method of control can be beneficial for quick actions and alternatives to physical TV remotes. However, situating the easy use of air and motion driven hand gestures in noisy spaces is complex and must be calibrated carefully, such as considering the situational triggers, environmental contexts, and other interrupting factors.

Solution

Along with extensive research on current practices, successful use cases, and major trends across gesture control, we are working with sign language experts, educators and learning from ASL to design responsive and accurate gestures. While determining the UX and interaction flow of each movement between [trigger, engagement, and outcome], we considered other critical components of includes a feedback indicator during gesture recognition and confirmation of success. Upon testing this limited library at first to ease in users, we are continuing to expand functionality to a greater set of gestures.