FPGA Implementation of Q-RTS for Real-Time Swarm Intelligence Systems

We propose an architectural blueprint to implement Q-RTS, Q-Learning Real-Time Swarm Reinforcement Learning algorithm, on FPGA. The design solution is built on FPGA-based Centralized RL Processing Units (CRLPU). A CRLPU processes local and global state-action matrices and exchanges information frames with low-power Microcontroller-based Agents. The novel architecture implementation, for up to 32 Agents with up to 512 states, on a Xilinx Ultrascale device shows low resource requirements in terms of CLB (7%) and memory (2% FF and 22% BRAM). Performance metrics show that the required energy per generated action is always lower than 1 µJ.