Real-time kernel processing for convolutional neural networks on SoC FPGAs

Abstract

This work supports the SMART FIRES project, which is an EPSCoR track 1 program with the goal of helping wildfire management. The SMART FIRES initiative aims to develop technologies for wildfire prediction and mitigation, with a particular focus on supporting prescribed fire planning and burning. This thesis contributes to these efforts by enabling the real-time analysis of hyperspectral imagery to identify fuel sources and their conditions directly in the field. This project presents a custom hardware accelerator for convolutional neural networks on Field-Programmable Gate Arrays, implementing memory controllers, data unpacking, and multiply-accumulate operations to process hyperspectral datacubes in real time. The hardware design uses circular buffering and specialized memory allocation to efficiently handle 3D convolution operations on hyperspectral data. An embedded Linux software stack using the Yocto Project creates a custom Board Support Package with bootloader configurations, kernel modifications, device drivers, and deployment tools. This work demonstrates effective hardware-software co-design for embedded systems, allowing for effective development and deployment of hardware and software for embedded systems.