Rigid 3D Object Alignment: Optimization vs. Feed-Forward Prediction

We study rigid 3D asset alignment: placing an independently generated 3D asset onto a base object via a rigid transformation, without modifying geometry. We contrast two paradigms: (i) an optimization baseline based on differentiable rendering, which is accurate but requires reference views and minutes of compute per asset, and (ii) a feed-forward Vision–Language–Action (VLA) model that regresses the transformation in a single sub-second forward pass, optionally without any reference views. To enable this study, we curate a large-scale dataset of 905k asset–base pairs with ground-truth transformations derived from PartVerse-XL. Our analysis reveals a pronounced accuracy–efficiency trade-off: the feed-forward model matches the optimization baseline on translation while falling substantially short on rotation, and we analyze the causes of this gap. Our results position feed-forward prediction as a fast, reference-free initialization for downstream refinement, and the dataset as a foundation for future alignment research.

@misc{khomenko-shamrai-2026-opensun3d-3d-alignment,
  author = {Ostap Khomenko and Maksym Shamrai},
  title  = {Rigid {3D} Object Alignment: Optimization vs. Feed-Forward Prediction},
  month  = {September},
  year   = {2026},
  note   = {\emph{Accepted to OpenSUN3D @ ECCV 2026.} \url{https://research.macpaw.com/publications/3d-alignment}},
}