HyperPPO: A scalable method for finding small policies for robotic
  control
Shashank Hegde Zhehui Huang Gaurav S. Sukhatme
Models with fewer parameters are necessary for the neural control of
memory-limited, performant robots. Finding these smaller neural network
architectures can be time-consuming. We propose HyperPPO, an on-policy
reinforcement learning algorithm that utilizes graph hypernetworks to estimate
the weights of multiple neural architectures simultaneously. Our method
estimates weights for networks that are much smaller than those in common-use
networks yet encode highly performant policies. We obtain multiple trained
policies at the same time while maintaining sample efficiency and provide the
user the choice of picking a network architecture that satisfies their
computational constraints. We show that our method scales well - more training
resources produce faster convergence to higher-performing architectures. We
demonstrate that the neural policies estimated by HyperPPO are capable of
decentralized control of a Crazyflie2.1 quadrotor. Website:
https://sites.google.com/usc.edu/hyperppo