3D Vision & Generative AI

Digit3D: 3D Multimodal MNIST Benchmark

Transforming 2D MNIST into rich 3D geometries (watertight meshes, sparse SDFs, and 6D normal-oriented point clouds) with continuous flow matching generative pipelines.

Khoi DO
Conquer3D Research & Open-Source Geometry Engine
Code Conquer3D Engine Overview Unconditional Generation Class-Conditional Generation Image-Conditional Generation Arbitrary-Resolution Generation Sparse Voxel Reconstruction Image-Conditional Mesh Generation PointCloud-Conditional Mesh Generation Limitations
Experiment 5

Two-Stage Sparse Voxel 3D Mesh Generation

High-fidelity 3D mesh synthesis from 2D MNIST input images via a cascaded sparse geometric flow framework. Stage 1 (Structure DiT MeanFlow) predicts 3D active voxel occupancy grids, followed by Stage 2 (Sparse Vertex SDF Rectified Flow) predicting continuous 1D Signed Distance Fields decoded into watertight triangle meshes via Differentiable Marching Cubes.

0 Digit 0 (Sample #000) 1.34s
2D Input Digit 0 2D Input
Stage 1 Voxels
Stage 2 Mesh
1 Digit 1 (Sample #001) 420ms
2D Input Digit 1 2D Input
Stage 1 Voxels
Stage 2 Mesh
2 Digit 2 (Sample #002) 737ms
2D Input Digit 2 2D Input
Stage 1 Voxels
Stage 2 Mesh
3 Digit 3 (Sample #003) 851ms
2D Input Digit 3 2D Input
Stage 1 Voxels
Stage 2 Mesh
4 Digit 4 (Sample #004) 678ms
2D Input Digit 4 2D Input
Stage 1 Voxels
Stage 2 Mesh
5 Digit 5 (Sample #005) 997ms
2D Input Digit 5 2D Input
Stage 1 Voxels
Stage 2 Mesh
6 Digit 6 (Sample #006) 734ms
2D Input Digit 6 2D Input
Stage 1 Voxels
Stage 2 Mesh
7 Digit 7 (Sample #007) 620ms
2D Input Digit 7 2D Input
Stage 1 Voxels
Stage 2 Mesh
8 Digit 8 (Sample #008) 1.10s
2D Input Digit 8 2D Input
Stage 1 Voxels
Stage 2 Mesh
9 Digit 9 (Sample #009) 780ms
2D Input Digit 9 2D Input
Stage 1 Voxels
Stage 2 Mesh

Cascaded 3D Generation: Left: 2D Input • Middle: Stage 1 Active Voxel Quads • Right: Stage 2 Watertight Mesh