PixelGen | Dhairya J. Shah

ACM MobiCom '24 ImmerCom WorkshopACM/IEEE IPSN '24 Demo

Best Demo Runner-Up, IPSN '24

Conventional embedded cameras capture only visible light and spend most of their energy on the image sensor and on streaming pixels. PixelGen instead senses the environment with a handful of low-power sensors, including audio, magnetic and other modalities, and transmits that compact description at a fraction of the energy. A multimodal generative model on the receiving side reconstructs a rich visual representation, which can be viewed on a mixed-reality headset.

PixelGen was published at the ImmerCom workshop at MobiCom ‘24, and its demonstration received the Best Demo Runner-Up award at IPSN ‘24.

Publications

PixelGen: Rethinking Embedded Cameras for Mixed-Reality

Kunjun Li^*, Manoj Gulati^*, Dhairya Shah^*, Steven Waskito, Shantanu Chakrabarty, and Ambuj Varshney

ACM ImmerCom (MobiCom) ’24 Workshop

DOI arXiv Bib PDF

@inproceedings{pixelgen_mobicom24,
  author = {Li, Kunjun and Gulati, Manoj and Shah, Dhairya and Waskito, Steven and Chakrabarty, Shantanu and Varshney, Ambuj},
  title = {PixelGen: Rethinking Embedded Cameras for Mixed-Reality},
  year = {2024},
  isbn = {9798400704895},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3636534.3696216},
  doi = {10.1145/3636534.3696216},
  booktitle = {Proceedings of the 30th Annual International Conference on Mobile Computing and Networking (MobiCom '24)},
  pages = {2128–2135},
  numpages = {8},
  keywords = {embedded systems, networking, large language models},
  location = {Washington D.C., DC, USA},
  series = {ACM ImmerCom (MobiCom) '24}
}

Demo Abstract: PixelGen: Rethinking Embedded Camera Systems for Mixed-Reality

Kunjun Li^*, Manoj Gulati^*, Dhairya Shah, Steven Waskito, Shantanu Chakrabarty, and Ambuj Varshney

ACM/IEEE IPSN ’24 Demo 🏅 Best Demo Runner-Up Award

DOI Bib

@inproceedings{pixelgen_ipsn24demo,
  author = {Li, Kunjun and Gulati, Manoj and Shah, Dhairya and Waskito, Steven and Chakrabarty, Shantanu and Varshney, Ambuj},
  booktitle = {2024 23rd ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)},
  title = {Demo Abstract: PixelGen: Rethinking Embedded Camera Systems for Mixed-Reality},
  year = {2024},
  volume = {},
  number = {},
  pages = {271-272},
  series = {ACM/IEEE IPSN '24},
  keywords = {Headphones;Visualization;Power demand;Magnetic sensors;Mixed reality;Virtual reality;Sensor phenomena and characterization;Embedded Camera Systems;Mixed Reality;Multimodal AI;Low-power Systems},
  doi = {10.1109/IPSN61024.2024.00036}
}

All projects