arxiv:2503.15475

Cube: A Roblox View of 3D Intelligence

Published on Mar 19

· Submitted by

j4kn on Mar 20

Upvote

Authors:

Nishchaie Khanna ,

Tinghui Zhou ,

Yiheng Zhu ,

Xiaoxia Sun ,

Anirudh Sudarshan ,

Daiqing Li ,

Kangle Deng ,

Jean-Philippe Fauconnier ,

Tijmen Verhulsdonck ,

Maneesh Agrawala ,

Kayvon Fatahalian ,

Alexander Weiss ,

Christian Reiser ,

Ravi Kiran Chirravuri ,

Ravali Kandur ,

Abstract

Foundation models trained on vast amounts of data have demonstrated remarkable reasoning and generation capabilities in the domains of text, images, audio and video. Our goal at Roblox is to build such a foundation model for 3D intelligence, a model that can support developers in producing all aspects of a Roblox experience, from generating 3D objects and scenes to rigging characters for animation to producing programmatic scripts describing object behaviors. We discuss three key design requirements for such a 3D foundation model and then present our first step towards building such a model. We expect that 3D geometric shapes will be a core data type and describe our solution for 3D shape tokenizer. We show how our tokenization scheme can be used in applications for text-to-shape generation, shape-to-text generation and text-to-scene generation. We demonstrate how these applications can collaborate with existing large language models (LLMs) to perform scene analysis and reasoning. We conclude with a discussion outlining our path to building a fully unified foundation model for 3D intelligence.

View arXiv page View PDF GitHub repository Add to collection

Community

j4kn

Paper author Paper submitter 1 day ago

Abstract: Foundation models trained on vast amounts of data have demonstrated remarkable reasoning and generation capabilities in the domains of text, images, audio and video. Our goal at Roblox is to build such a foundation model for 3D intelligence, a model that can support developers in producing all aspects of a Roblox experience, from generating 3D objects and scenes to rigging characters for animation to producing programmatic scripts describing object behaviors. We discuss three key design requirements for such a 3D foundation model and then present our first step towards building such a model. We expect that 3D geometric shapes will be a core data type and describe our solution for 3D shape tokenizer. We show how our tokenization scheme can be used in applications for text-to-shape generation, shape-to-text generation and text-to-scene generation. We demonstrate how these applications can collaborate with existing large language models (LLMs) to perform scene analysis and reasoning. We conclude with a discussion outlining our path to building a fully unified foundation model for 3D intelligence.