An Overlay for Rapid FPGA Debug of Machine Learning Applications

FPGAs show promise as machine learning accelerators for both training and inference. Designing these circuits on reconfigurable technology is challenging, especially due to bugs that only manifest on-chip when the circuit is running at speed. In this paper, we propose a flexible debug overlay family that provides software-like debug times for machine learning applications. At compile time, the overlay is added to the design and compiled. At debug time, the overlay can be configured to record statistical information about identified weight and activation matrices; this configuration can be changed between debug iterations allowing the user to record a different set of matrices, or record different information about the observed matrices. Importantly, no recompilation is required between debug iterations. Although the flexibility of our overlay suffers some overhead compared to fixed instrumentation, we argue that the ability to change the debugging scenario without requiring a recompilation may be compelling and outweigh the disadvantage of higher overhead for many applications.