Development of a smart camera system on an FPGA

Abstract

In recent years, hyperspectral cameras have been appearing in many applications that need more information than what conventional color cameras can provide. A hyperspectral camera is able to capture data ranging in wavelengths from the visible spectrum all the way into the infrared. In this way, it is able to 'see' hundreds of colors, much more than the human eye or any standard camera that typically uses only 3 spectral values (corresponding to the standard red, green, and blue colors). Due to the large amount of data that these cameras can generate at increasingly faster frame rates, conventional computers are not able to perform all the necessary processing in real-time. Because of this limitation, a new system is needed to perform the image processing. This master's thesis is meant to contribute to the development of a smart camera targeted for hyperspectral image processing using a Field Programmable Gate Array (FPGA) and object sorting with a prototype waterfall system. Through the use of a Hardware Description Language (HDL), a currently used image processing algorithm has been implemented to classify pixels. Additionally, design and test of an architecture for full object classification has been developed for the FPGA. High-speed transceivers are used to move data between multiple FPGA development boards. When paired with a hyperspectral camera and a monochrome line scan camera, this prototype system is capable of scanning objects in freefall and deciding within milliseconds whether or not to keep the object. This decision will dictate the action of air jets to displace unwanted objects. This full system is potentially of interest to small businesses or farms as it will enable farmers to perform their own premium bulk sorting in a cost effective manner.