Transforming 2D MNIST into rich 3D geometries (watertight meshes, sparse SDFs, and 6D normal-oriented point clouds) with continuous flow matching generative pipelines.
While Digit3D and its sparse continuous flow generative models achieve efficient 3D synthesis, several geometric and empirical limitations remain to be addressed in future research.
The dataset's 3D meshes are constructed from 2D MNIST boundary contours with parametric extrusion, triangulation, and smoothing. While optimal for rapid algorithm development and differentiable benchmarking, the synthetic assets exhibit constrained topological diversity (e.g., restricted genus, lack of complex internal cavities, and uniform thickness) compared to unconstrained real-world 3D object repositories (such as ShapeNet, Objaverse, or OmniObject3D).
Operating under compact model parameterizations (6-layer Structure DiT and 6-layer Sparse Vertex Transformers, $D = 256$) trained for modest schedules (15–20 epochs on single consumer GPUs), the generative pipeline occasionally produces localized non-manifold artifacts—such as boundary self-intersections, isolated floating zero-crossing facets, or non-manifold edge junctions—when predicted vertex SDFs lack global continuity across neighboring sparse voxel cells.
The $32^3$ volumetric grid resolution enables ultra-fast $\mathcal{O}(B)$ variable-length attention with zero dummy padding overhead, but sets a physical upper bound on fine corner sharpness and high-frequency structural boundaries. Future extensions will incorporate multi-resolution hierarchical octrees or adaptive tetrahedral complexes to completely eliminate topological defects while preserving high inference throughput.
@article{do2026digit3d,
title = {Digit3D: 3D Multimodal MNIST Benchmark},
author = {Khoi DO},
journal = {arXiv preprint},
year = {2026},
url = {https://github.com/KhoiDOO/digit3d}
}