Date of Graduation

Summer 2024

Degree

Master of Science in Computer Science

Department

Computer Science

Committee Chair

Razib Iqbal

Abstract

Voice-controlled smart assistants have received widespread popularity. It plays a pivotal role in smart homes by providing a natural and convenient interface for interacting with smart devices. However, these assistants are unable to serve persons with physical disabilities and speech impairments. Therefore, non-verbal communication methods, such as eye tracking, gesture recognition, and context awareness can complement and overcome some of these limitations to enhance user experience in smart homes. To address this issue, I am investigating non-verbal communication methods to make smart home technology more accessible and intuitive. In this research, I focus on proxemics, i.e., the study of distance between smart home users and surrounding objects, to enable spatial awareness and intuitive automation in smart homes. I apply scene graphs to provide a structured representation, such as positions, relationships, and properties, of the static and moving objects in indoor home environments. The novelty of this approach lies in the application of proxemics via scene graph generation for extracting contextual information and scene understanding to automate smart home actions. This work adds a distance attribute to the scene graph predicate for quantifying human and object relationships. The key contribution lies in leveraging proxemics through scene graph generation to extract contextual information, facilitating the automation of intelligent actions.

Keywords

distance calculation, human-computer interaction, non-verbal communication, object detection, proxemics, scene analysis, scene graphs, smart home, stereo vision

Subject Categories

Computer Sciences | Graphics and Human Computer Interfaces

Copyright

© Debaleen Das Spandan

Download

Open Access

Share

COinS