Dictate3D: Natural-language control for a 3D editor

The experiment

Dictate3D tests a different way to operate a 3D editor: describe an action in ordinary language and have the scene update in response. Commands can create, move, delete, or modify objects without navigating a traditional tool panel.

System design

• React Three Fiber and Three.js render the interactive scene.
• Next.js provides the surrounding application.
• A Flask service hosts the command-processing models.
• BERT classifies command intent.
• Supporting models and utilities extract values such as colors, quantities, and object references.

Engineering focus

The central problem was turning an open-ended instruction into a predictable scene operation. The command pipeline separates intent, target, and parameters before applying a change, making failures easier to identify and individual models easier to improve.

Docker packages the Python model service so the inference environment remains consistent across development and deployment.

Result

Dictate3D demonstrates how language can become an input layer for a visual tool. It also provided an early test bed for coordinating several specialized models inside one interactive product.