PixelGen: Rethinking Embedded Camera Systems
Kunjun Li Manoj Gulati Steven Waskito Dhairya Shah Shantanu Chakrabarty Ambuj Varshney
Embedded camera systems are ubiquitous, representing the most widely deployed
example of a wireless embedded system. They capture a representation of the
world - the surroundings illuminated by visible or infrared light. Despite
their widespread usage, the architecture of embedded camera systems has
remained unchanged, which leads to limitations. They visualize only a tiny
portion of the world. Additionally, they are energy-intensive, leading to
limited battery lifespan. We present PixelGen, which re-imagines embedded
camera systems. Specifically, PixelGen combines sensors, transceivers, and
low-resolution image and infrared vision sensors to capture a broader world
representation. They are deliberately chosen for their simplicity, low bitrate,
and power consumption, culminating in an energy-efficient platform. We show
that despite the simplicity, the captured data can be processed using
transformer-based image and language models to generate novel representations
of the environment. For example, we demonstrate that it can allow the
generation of high-definition images, while the camera utilises low-power,
low-resolution monochrome cameras. Furthermore, the capabilities of PixelGen
extend beyond traditional photography, enabling visualization of phenomena
invisible to conventional cameras, such as sound waves. PixelGen can enable
numerous novel applications, and we demonstrate that it enables unique
visualization of the surroundings that are then projected on extended reality
headsets. We believe, PixelGen goes beyond conventional cameras and opens new
avenues for research and photography.