Design methodology for 3d-stacked imaging systems with integrated deep learning

Abstract

The Internet of Things (IoT) revolution has brought along with it billions of always on, always connected devices and sensors, associated with which are huge amounts of data that must be transmitted to an off-chip host for classification. However, sending these large volumes of unprocessed data incurs large latency and energy penalties which impairs the energy efficiency of resource constrained IoT systems. Moving computations to the sensor offers the potential to improve performance and energy efficiency of the end application. The objective of the presented research is to explore sensor integrated computing which allows the deployment of smart sensors capable of performing computations in-field. Initially, we introduce the design of a 3D-stacked image sensor with integrated deep learning, which uses the advantages of 3D integration to increase sensor fill factor, simplify routing, increase parallelism, and enhance memory capacity. Through an exploration of the design space we investigate how the system architecture and resource constraints can dictate system metrics such as the optimum energy efficiency configuration and accuracy-throughput tradeoffs. Next, we examine technology based solutions to further enhance system performance through the use of 3D stacked digital sensors with in-pixel ADCs, and explore how emerging device based processing-in-memory neural accelerators can offer superior energy efficiency. Furthermore, the various circuit issues involved with the design of these sensor based systems are investigated through the discussion of post-silicon results from an image sensor SOC with integrated energy harvesting. The dissertation concludes with a discussion on how energy harvesting sensors can be used to achieve energy neutral self-powered systems capable of operating solely with harvested energy.