Position control of an acoustic cavitation bubble by reinforcement
  learning
Kálmán Klapcsik Bálint Gyires-Tóth Juan Manuel Rosselló Ferenc Hegedűs
A control technique is developed via Reinforcement Learning that allows
arbitrary controlling of the position of an acoustic cavitation bubble in a
dual-frequency standing acoustic wave field. The agent must choose the optimal
pressure amplitude values to manipulate the bubble position in the range of
$x/\lambda_0\in[0.05, 0.25]$. To train the agent an actor-critic off-policy
algorithm (Deep Deterministic Policy Gradient) was used that supports
continuous action space, which allows setting the pressure amplitude values
continuously within $0$ and $1\, \mathrm{bar}$. A shaped reward function is
formulated that minimizes the distance between the bubble and the target
position and implicitly encourages the agent to perform the position control
within the shortest amount of time. In some cases, the optimal control can be 7
times faster than the solution expected from the linear theory.