Development of a smart camera system using a system on module FPGA

Abstract

Imaging systems can now produce more data than conventional PCs with frame grabbers can process in real-time. Moving real-time custom computation as close as possible to the image sensor will alleviate the bandwidth bottle-neck of moving data multiple times through buffers in conventional PC systems, which are also computation bottlenecks. An example of a high bandwidth, high computation application is the use of hyperspectral imagers for sorting applications. Hyperspectral imagers capture hundreds of colors ranging from the visible spectrum to the infrared. This masters thesis continues the development of the hyperspectral smart camera by integrating the image sensor with a field programmable gate array (FPGA) and by developing an object tracking algorithm for use during the sorting process, with the goal of creating a single compact embedded solution. An FPGA is a hardware programmable integrated circuit that can be reprogrammed depending on the application. The prototype integration involves the development of a custom printed circuit board to connect the data and control lines between the sensor, the FPGA, and the control code to read data from the sensor. The hyperspectral data is processed on the FPGA and is combined with the object edges to make a decision on the quality of the object. The object edges are determined using a line scan camera, which provides data via the Camera Link interface, and a custom object tracking algorithm. The object tracking algorithm determines the horizontal edges and center of the object while also tracking the vertical edges and center of the object. The object information is then passed to the air jet sorting subsystem which ejects bad objects. The solution is to integrate the hyperspectral image sensor, the two processing algorithms, and Camera Link interface into a single, compact unit by implementing the design on the Intel Arria 10 System on Module with custom printed circuit boards.