ProxeGraph: Scene Graph Generation Utilizing Proxemics for Smart Homes

Abstract

Voice-controlled smart assistants have gained widespread popularity, playing a pivotal role in smart homes by providing a natural and convenient interface for interacting with devices. However, these assistants are not usable by individuals with physical disabilities and speech impairments. Non-verbal communication methods such as gesture recognition, eye tracking, and context awareness can complement these technologies and mitigate some limitations, thereby enhancing the user experience in smart homes. This paper introduces a novel concept of proxemics-enhanced scene graph generation to make human-computer interaction more accessible and intuitive in smart homes. We focus on proxemics, which is the study of the distance between users and surrounding objects, to enable spatial awareness and intuitive automation. Scene graphs offer a structured representation of the positions, relationships, and properties of both static and moving objects in indoor environments. The novelty of this approach lies in the application of proxemics to scene graph generation, extracting spatial information, and understanding scenes to automate smart home actions. We have incorporated a distance attribute into the scene graph predicate to quantify human-object relationships.

Department(s)

Computer Science

Document Type

Conference Proceeding

DOI

10.1109/MIPR62202.2024.00024

Keywords

distance calculation, human-computer interaction, non-verbal communication, object detection, scene analysis

Publication Date

1-1-2024

Journal Title

Proceedings of the International Conference on Multimedia Information Processing and Retrieval Mipr

Link to Full Text

Share

COinS