Model-free control for dynamic-field acoustic manipulation using reinforcement learning

Dynamic-field acoustic manipulation techniques benefit numerous applications in microsystem assembly, pattern formation, biological research, tissue engineering, and lab-on-a-chip. These techniques generally rely on a theoretical dynamic model of particle motion in the acoustic field. Accordingly, success of the manipulation task highly depends on the accuracy of the employed dynamic model. However, modelling such dynamic behavior is a great challenge in more advanced acoustic manipulation devices and requires significant simplifications. Here, we introduce a model-free control method based on reinforcement learning for highly-dynamic acoustic manipulation devices. In our method, the controller does not need a prior knowledge of the acoustic field and learns the optimal control policy for each manipulation task by merely interacting with the acoustic field. As a proof-of-concept, we apply our method to a classic acoustic manipulation device, a Chladni plate consisting of a centrally-actuated vibrating plate. By employing the controller, we demonstrate successful manipulation of single and multiple particles towards target locations on the plate surface. The model-free control method is not limited to the Chladni plate and can be potentially applied to a broad range of acoustic manipulation devices as well as other forms of field-based micromanipulation systems, where accurate theoretical modelling of the field is challenging.