Skip to content

3DCV Workshop 03 - 3D Object Reconstruction

Workshop Overview

In this workshop, we'll reconstruct small 3D objects and compare various methods and devices to realize this task. The goal is to understand the strengths and weaknesses of different approaches, including traditional Structure From Motion methods and model-based methods.

Workshop Goals

  • Reconstruct small 3D objects using various methods.
  • Compare results from different devices and techniques.
  • Reflect on the effectiveness of traditional versus model-based methods.

Procedure

The parts do not need to be done by everyone. Form teams and pairs that work on each part and present the status and results every hour.

Parts to do:

  • 3D Scanning: Use a 3D scanner or a smartphone app that supports 3D scanning.
  • Collect images, use a smartphone or webcam on a tripod and the rotating turntable. Block the background with the black cloth.
  • Eventually use a tool like SAM to segment the object from the background or create image masks.
  • Reconstruct the object using one of the following methods:
    • Structure From Motion: Use software like Meshroom or COLMAP to create a 3D model from images.
    • Model-based Reconstruction: Explore visual geometric models like VGGT, DUNE or Mast3r that can reconstruct 3D objects from images.

Expected Outcomes

  • A reconstructed 3D model of a small object, exported in a common format (e.g., OBJ, PLY).
  • A comparison of the results from different methods and devices.
  • A reflection on the effectiveness of traditional versus model-based methods.
  • A written report summarizing the findings.
  • A discussion on the future of 3D object reconstruction in the context of AI advancements.
  • A reproducible workflow that can be shared with others.

Using a scanner or smartphone app

We have a Moose 3D Scanner. You can download the scanning software JMStudio.

3D Reconstruction from Real-World Image Data

Capturing

Capture RGB images of some small object that you want to reconstruct. Use a smartphone or webcam on a tripod and a rotating turntable. Block the background with a black cloth to avoid noise in the images.

Hints for capturing

Work like a scientist. Think before you act:

  • Which properties should the object have, so that the reconstruction is easy?
  • Capture scenes with good lighting conditions. Consider lens distortion and reflections.
  • Note for all your samples your expectations. Name and sort all captured scenes and images in a folder structure that allows you to automatically process all images using a script.

Image preprocessing

Inspect the captured images and prepare them for reconstruction. If necessary, use a tool like SAM to segment the object from the background or create image masks.

Reconstructing the 3D Object

Pick one of the following tools to reconstruct the 3D object from the captured images:

Hints for working with tools

Work like a scientist. Think before you act:

  • Check the documentation of the tool you selected.
  • Compare the system requirements of the tools with your computer.
  • Start with the example images provided by the tool to ensure it works correctly. If not available, use synthetic images from the NeRF dataset (available on the mobile hard drive) or similar.

3D Reconstruction with AI Models

Pick one or more of the following models to run 3D reconstruction on RGB images:

You can also check out more models from this list of Awesome DUSt3R Resources

Hints for working with scientific tools

Work like a scientist. Think before you act:

  • Check the documentation (GitHub readme) of the tool you selected.
  • Compare the system requirements of the tools with your computer. Usually this is not stated prominently, so you may need to check the code or issues. And be very precise about the hard- and software (driver versions, CUDA version) you have available.
  • Start with the example images provided by the tool to ensure it works correctly. If not available, use synthetic images from the NeRF dataset (available on the mobile hard drive) or similar.

Discussion and Reflection

Key Question

Do we still need COLMAP (feature detection) or are the AI models good enough?

Points to Consider

  • Accuracy and consistency of AI models across different examples
  • Computational requirements of AI models
  • Limitations in specific use cases
  • Real-time capabilities

Task

Create the following deliverables and upload them to the FELIX folder:

  • A reconstructed 3D model of a small object, exported in a common format (e.g., OBJ, PLY).
  • A comparison of the results from different methods and devices.
  • A reflection on the effectiveness of traditional feature-based versus model-based methods.
  • A written report summarizing the findings.
  • A discussion on the future of 3D object reconstruction in the context of AI advancements.
  • A reproducible workflow that can be shared with others.

Final Thoughts

By completing this workshop, you have:

  • Captured RGB data (aka images) for a small research project,
  • Organized and structured data to efficiently process it,
  • Dealt with classic photogrammetry tools as well as scientific code that uses AI models,
  • and presented and discussed your results in a scientific format.

Grading

For this workshop and the next one the outcome of task needs to be handed in. Upload the resulting document to the FELIX folder until one week after the workshops. It should either contain the files in a zip or a link to the project repository. If you worked in teams, state who is responsible for each part.

Your results need to be reproducible and contain references to all used sources and tools.