Design and optimization of human-machine interaction interface for the intelligent Internet of Things based on deep learning and spatial computing

Wencong Wang, Ke Wang, Hui Du,
Design and optimization of human-machine interaction interface for the intelligent Internet of Things based on deep learning and spatial computing,
Egyptian Informatics Journal,
Volume 30,
2025,
100685,
ISSN 1110-8665,
https://doi.org/10.1016/j.eij.2025.100685.
(https://www.sciencedirect.com/science/article/pii/S1110866525000787)
Abstract: The Intelligent Internet of Things (IoT) is transforming interactions with smart devices, especially in a home environment, with lighting, security, and entertainment systems. Designing user-friendly interfaces for IoT systems presents difficulties, particularly for individuals with severe disabilities such as Amyotrophic Lateral Sclerosis (ALS), spinal cord injuries, and cerebral palsy. Existing user interfaces frequently restrict the capacity of individuals, particularly those with severe disabilities, to operate smart home gadgets efficiently. The study proposes that the NeuroSpatialIOT system solve this problem by combining 2D spatial mapping, deep learning, and eye tracking. The system’s innovative approach accurately interprets user intent through natural eye gaze and provides relevant controls based on the user’s viewpoint and environment. NeuroSpatialIOT leverages deep learning to gather data on eye movements, the 2D spatial configuration of the room, and user objectives. NeuroSpatialIOT functions by tracking eye movements, comprehending the room’s 2D layout, and employing deep learning to predict user intentions. The system then displays appropriate controls in the user’s field of view, enabling intuitive interaction with IoT devices. Testing on non-disabled and severely disabled people yielded positive findings. Non-disabled participants scored 88.9% and disabled individuals 91.5%, indicating great system usability. Subsequently took 40% less time for non-impaired users to complete tasks and 60% less for disabled users. With NeuroSpatialIOT, altering room temperature or illumination takes 10–15 s instead of 30–45. The results show that the system can improve autonomy and quality of life for varied users in IoT-enabled settings by making home operations easier to handle.
Keywords: Deep learning; Eye tracking; Human-machine interaction (HMI); Internet of things (IoT); Smart home devices; Spatial computing; User intent interpretation