CIT: A combined approach of improving inference and training phases of deep learning for IoT applications

Amir Masoud Rahmani, Seyedeh Yasaman Hosseini Mirmahaleh,
CIT: A combined approach of improving inference and training phases of deep learning for IoT applications,
Expert Systems with Applications,
Volume 280,
2025,
127554,
ISSN 0957-4174,
https://doi.org/10.1016/j.eswa.2025.127554.
(https://www.sciencedirect.com/science/article/pii/S0957417425011765)
Abstract: Memory demand and processing power restrictions are critical factors in IoT-based applications as technology evolves to support a wide range of devices. Deep learning (DL) techniques have emerged as solutions to address the complexities associated with computations and demand management in IoT systems. However, challenges such as energy efficiency, bandwidth requirements, and limited memory capacity persist for devices with restricted storage and processing power. This research aims to tackle these challenges by customizing deep learning techniques through methods such as bit reduction, weak weight pruning, neuron pruning, and layer pruning in multi-layer perceptron models of deep neural networks (DNNs). The primary focus is on enhancing the efficiency of both inference and training phases of DNNs to accommodate IoT devices with constrained memory and energy consumption. By pruning weak neurons, weights, and layers, this study seeks to optimize the performance of DNNs in IoT applications. Additionally, the research involves combining similar weights within each layer and aligning them with corresponding locations in the subsequent layer. This approach creates opportunities for weight reusability, thereby improving memory utilization by reducing redundant weight storage. The results demonstrate significant improvements in latency and memory demands, with reductions of approximately 95% and 43%, respectively, compared to traditional DL models. These findings underscore the potential of customized deep learning techniques in effectively addressing memory and power constraints in IoT applications.
Keywords: Internet of Things (IoT); Deep Learning (DL); Weight combination; Pruning; Bit reduction