Resource-aware and robust image processing for intelligent sensor systems

Abstract

The objective of this research is to design resource-aware and robust image processing algorithms, system architecture, and hardware implementation for intelligent image sensor systems in the Internet-of-Things (IoT) environment. The research explores the design of a wireless image sensor system with low-overhead pre-processing, which is integrated with a reconfigurable energy-harvesting image sensor array to implement a self-powered image sensor system. For reliable delivery of region-of-interest (ROI) under dynamic environment, the research designs low-power moving object detection with enhanced noise robustness. The system energy is further optimized by a low-power ROI-based coding scheme, whose parameters are dynamically controlled by a low-power rate controller to minimize required buffer size with minimum computational overhead. To enable machine learning based intelligent image processing at the IoT edge devices, the research proposes resource-efficient neural networks. The storage demand is reduced by compressing the neural network weights with an adaptive image encoding algorithm, and the computation demand is optimized by mapping the entire network parameters and operations into the frequency domain. To further improve the energy-efficiency and throughput of the edge device, the research explores inference partitioning of a DNN between the edge and the host platforms.