Digit3D: 3D Multimodal MNIST Benchmark

Transforming 2D MNIST into rich 3D geometries (watertight meshes, sparse SDFs, and 6D normal-oriented point clouds) with continuous flow matching generative pipelines.

Khoi DO
Conquer3D Research & Open-Source Geometry Engine

PointCloud-Conditional 3D Mesh Generation

Continuous 3D mesh synthesis conditioned directly on sparse input point clouds ($N = 512$ with oriented normals). Stage 1 (Point-Conditioned Structure DiT MeanFlow) predicts active voxel occupancy grids, followed by Stage 2 (Sparse Vertex SDF Rectified Flow) predicting continuous 1D Signed Distance Fields decoded into watertight triangle meshes via Differentiable Marching Cubes.

0 Digit 0 (Sample #000) 1.02s
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
1 Digit 1 (Sample #001) 354ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
2 Digit 2 (Sample #002) 477ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
3 Digit 3 (Sample #003) 542ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
4 Digit 4 (Sample #004) 427ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
5 Digit 5 (Sample #005) 655ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
6 Digit 6 (Sample #006) 469ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
7 Digit 7 (Sample #007) 409ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
8 Digit 8 (Sample #008) 718ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh
9 Digit 9 (Sample #009) 497ms
Input Point Cloud
Stage 1 Voxels
Stage 2 Mesh

Two-Stage Geometric Pipeline: Input Point Cloud ($N=512$, $6\text{D}$) $\to$ Stage 1 Predicted Sparse Voxels $\to$ Stage 2 Watertight Triangle Mesh via Differentiable Marching Cubes.