Fast model inference and training on-board of Satellites
Vít Růžička Gonzalo Mateo-García Chris Bridges Chris Brunskill Cormac Purcell Nicolas Longépé Andrew Markham
Artificial intelligence onboard satellites has the potential to reduce data
transmission requirements, enable real-time decision-making and collaboration
within constellations. This study deploys a lightweight foundational model
called RaVAEn on D-Orbit's ION SCV004 satellite. RaVAEn is a variational
auto-encoder (VAE) that generates compressed latent vectors from small image
tiles, enabling several downstream tasks. In this work we demonstrate the
reliable use of RaVAEn onboard a satellite, achieving an encoding time of
0.110s for tiles of a 4.8x4.8 km$^2$ area. In addition, we showcase fast
few-shot training onboard a satellite using the latent representation of data.
We compare the deployment of the model on the on-board CPU and on the available
Myriad vision processing unit (VPU) accelerator. To our knowledge, this work
shows for the first time the deployment of a multi-task model on-board a
CubeSat and the on-board training of a machine learning model.