Skip to content

Workshop 3+4 Noam Hartmann

Data Sets

I tested multiple objects, knowing that some might not perform well, such as the clock with its reflective surfaces. I also chose objects that I expected to reconstruct successfully. Additionally, some of the sample photos taken vary in quality (as I am a bad photographer).

Name Mimic Paper Bug Clock
Sample Size 30 images total 25 images total 17 images total
Visual Mimic Front View PaperBug Front View Clock Front View
Source Taken by an iPhone 16 (credit to Maximilian) Taken by Samsung Galaxy A53s Taken by Samsung Galaxy A53s

Meshroom

Meshroom is a 3D reconstruction application that uses photogrammetry to generate 3D models from a set of input images. The process starts by analyzing the images to detect common features, matches these features across multiple views, and then reconstructs the object’s geometry. Meshroom automates much of this process, making it user-friendly, as the user only needs to provide a set of overlapping photos for it to begin building the 3D model.

Meshroom1 Meshroom2

Figures: Pictures taken in Meshroom

Installation

The installation was relatively simple as it was a .exe I just executed and then it was ready to go.

Results

Name Mimic Paper Bug Clock
Visual Mimic Front View
Mimic Back View		 N/A N/A
Input/Processing Time 30 images total / 21:28 min 25 images total / N/A 17 images total / N/A

Problems

Meshroom automatically removes pictures that don’t work for its reconstruction process, often without explaining why. This made it quite unpredictable when testing with my datasets. As a result, two out of three datasets failed and produced no model, only a blank canvas.

Dune

Problems

DUNE, despite the relatively short instructions provided, is not a simple tool to install or use for 3D reconstruction. This is because DUNE itself is not a 3D reconstruction tool, but rather can be used as one when combined with other AI model like in this case Mast3r. Initially, this made the installation and setup quite difficult, as I approached the project under the assumption that DUNE was a standalone 3D reconstruction tool (similar to the other AI models provided to us). This misunderstanding was partly my own mistake, as I was trying to get 3D reconstruction software to work that didn’t exist. Nonetheless, DUNE should be viewed more as an extension or enhancement of the Mast3r model, since it operates through applications based on Mast3r.

Another problem worth mentioning is that the partial support for Dune on the Mast3r model was added on the second day of the workshop which addition made the installation much easier later on.

Installation (For Mast3r):

Followed the Mast3r GitHub instructions for installation and then followed them further under the “Dune model” section.

Results

Name Mimic Paper Bug Clock
Mesh (Front and Back) Mimic Front View
Mimic Back View		 PB Front View
PB Back View		 Clock Front View
Clock Back View
Point Cloud (Front and Back) Mimic Front View
Mimic Back View		 PB Front View
PB Back View		 Clock Front View
Clock Back View
Input/Processing Time 15 Images / 8-10 min 25 Images / 12mins 17 Images/ 6 mins

Other Things

Dune’s PCA Visualiser

DUNE provides an algorithm that produces patch features, which are not directly useful for this 3D reconstruction workshop but are interesting nonetheless, so I thought I include it here.

Installation

  1. Follow the GitHub README Instructions:
  2. Create and activate a Conda environment with Python 3.12
  3. Install the required Python packages

  4. You may also need to manually install additional packages during later steps

  5. Install the teacher models:

  6. Follow the instructions provided in the GitHub repository to download the teacher models.

  7. Download the checkpoint:

  8. Download dune_vitbase14_448_paper.pth and place it in the /dune folder.

  9. Run the PCA Visualizer:

  10. Use this command to avoid a certain bug: 

    python -m scripts.pca_vis

  11. Test with images:

  12. Use the images provided in the assets folder to run your tests.

Results

Mimic Random Scene Picture Paper Bug
PCA1 PCA2
PCA3		 PCA4

Mast3rs Matching

Mast3r provides a matching algorithm that shows the user the how many matching features the Mast3r model find between two inputs.

Results (at least some of them)

View Top Front Back
Inputs Matching1 Matching2 Matching3
Outputs Matching4 Matching5 Matching6
Matches Detected 638 1622 1680

Conclusion

From my experience, AI models like Dune+Mast3r are already good enough for most 3D reconstruction tasks. Dune + Mast3r worked more or less well with all my datasets, even when the image quality wasn’t perfect. On the other hand, Meshroom (a feature-based tool) failed in two out of three cases and often didn’t make it clear why.

Additionally, it's worth mentioning that I used different hardware throughout the process of getting the software to work (as my laptop gave me some trouble). Because of this, I can't exactly determine whether the AI models are faster or slower compared to Meshroom.

Nonetheless, while I think tools like Meshroom still have their place, especially in precise, controlled setups, I personally found the AI models to be more reliable, easier to work with, and likely faster overall.

For me, AI models, altough not perfect, feels like the better alternative for 3D reconstruction.