hn.today

Show HN: DRS-VPT. Localize a photo in a 3D point cloud without an initial pose

github.com9 points0 comments
Screenshot of Show HN: DRS-VPT. Localize a photo in a 3D point cloud without an initial pose

DRS-VPT is a vision-and-point-cloud model for placing a camera image in a 3D LiDAR scan or point-cloud map without needing an initial camera pose. It progressively estimates and refines the pose through transformer blocks, and can also predict camera intrinsics when they are unknown. The same model is presented for outdoor driving scans and indoor maps. Beyond single-image localization, it can encode a map once for repeated image queries, estimate a camera-to-LiDAR transform from synchronized image-scan pairs without calibration targets, or refine an existing calibration.

The released code includes an online demo, Python APIs, sample data, evaluation and training tools, and pretrained weights totaling 2.9 GB. Reported tests on KITTI and nuScenes show 99.71% and 97.62% recall, respectively, within 2 m and 5°; mean translation and rotation errors are 0.08 m and 0.28° on KITTI, and 0.31 m and 0.80° on nuScenes. The README also documents how to reproduce those evaluations, including randomly transforming scans to test localization without a pose prior. Running the model locally requires PyTorch and sparse-convolution dependencies; GPU instructions target NVIDIA CUDA, with CPU and macOS options also provided.

Read on github.com0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Science

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.