Projects and Research
Although we live in a three-dimensional environment, most media are presented on two-dimensional output devices.
The challenge of processing three-dimensional (3D) data for visualization including human interaction has always excited me.
- How can we capture our 3D world?
- How can we make it accessible for various applications?
- How do we handle the loss of information when projecting 3D data onto two-dimensional (2D) surfaces?
- How can we restore 3D information from 2D images?
Those questions drive me in my research.
Past Focus Areas
During my PhD work at the research group for Computer Graphics at Technische Universität Berlin, I investigated the use of time-of-flight cameras for 3D graphics and interaction. I developed methods for real-time depth image processing and explored their applications in Computer Graphics and Human-Computer Interaction. As the data captured by time-of-flight cameras at this time was often noisy and incomplete, I focused on developing algorithms that could handle these issues using techniques from Computer Vision. My research focused on the challenges of capturing, processing, and visualizing 3D data in real-time, as well as the development of novel interaction techniques that leverage depth information. After finalizing my PhD, I decided to continue to work on 3D sensing in the industry, where I could apply my research to real-world applications.
The SICK AG in Waldkirch provided the perfect environment for this work and me personally to grow. I started as software developer for the Graphical User Interface of the Visionary 3D camera product line. Later, as software group lead, I supported multiple research projects at SICK AG, also through the supervision of students from universities like the Karlsruhe Institute of Technology. At the core of this work was the processing and analysis of image data, and the question of how these data can be enriched through additional information from time-of-flight or other 3D snapshot cameras.
An important part of this research was machine learning and its applications in Industry 4.0. On the one hand, many inspection tasks were be addressed effectively with classic machine learning or image processing methods. On the other hand, the use of autonomous vehicles was increasing rapidly, and for many resulting tasks, such as localization, networking, navigation, and collision avoidance, Deep Learning based methods showed promising results.
A major industry challenge was and still is to develop machine or deep learning models that can be trained with as few negative samples as possible. This is necessary because, in the target applications, reproducing failure cases is often impossible or at least very expensive. In addition, many applications require reliable detection, classification, and localization of people and other objects. In my group, we developed and evaluated solutions to these problems with the help of 3D sensing devices.
Current Topics
In 3D sensing, a recurring challenge is to visualize captured 3D camera data in a way that allows users to align their own perception with the recorded data. The core problem is that visualizing 3D content on 2D output devices inevitably introduces information loss, which can be compensated through interaction with the visualization system. On the other hand, VR headsets have evolved to a point where they can be used in industrial as well as consumer applications. They allow users to perceive 3D content, however, the optical design has its flaws that limits the usage to a couple of hours or even less if the application is not well designed. In my research, I am exploring ways to make 3D content more accessible and intuitive to use, both on traditional 2D output devices and in VR environments.
While 3D camera devices never reached a break-through in the industry, there are now numerous emerging approaches to reconstruct the 3D world simply from 2D camera images. The basic idea is simple: instead of always trying to reconstruct the 3D world each time from scratch, artificial neural networks are used to memorize the concept of 3D scenes and objects. These so called 3D geometric foundation models (e.g. DepthAnything3 or VGGT-Omega) allow a fast and simple way to create 3D models of the real world. My research goal is to make those 3D spaces accessible and intuitive to utilize in various applications. Important open questions remain, especially in the scope of user experience with such systems as well as about data protection and privacy, and I am highly motivated to address them.
Connecting Research and Teaching
In my academic work, I have gained excellent experience in collaborative project work with students. I bring current research topics into teaching and involve students directly in active research projects. I have very positive experiences with building interactive prototypes (such as the Smart Mirror and Multi-touch table listed below). This enables students to gain hands-on experience with emerging technologies and supports their direct transition into later development projects with, or within, industrial partners.
Network
In my current role, I am well connected within Hochschule Furtwangen, as well as international partners (e.g a student exchange to Kanagawa University in Yokohama, Japan was very successful and lead to a joined publication). I also maintain a large network with former colleagues from SICK AG (especially the research team at Linköping, Sweden), TU Berlin (now in locations including Paris, Zurich, and New York) and former fellow students from Weimar (now in Berlin, Köln, and at other institutions including Stanford). In addition, I am about to start a research visit at the French-German Research Institute of Saint-Louis (ISL) in France, where I will work on event based imaging and its applications in the context of autonomous drones.
My objective is to leverage the existing network to cultivate professional and academic development opportunities for students. Furthermore, I intend to expand this network to initiate collaborative research projects with colleagues who share my passion in 3D technologies.
Reflection
At the beginning of my PhD I was fascinated by the academic world of publishing and eager to attend many conferences and workshops. My goal was to publish at least one paper at each of the main conferences in Computer Graphics, Computer Vision and Human Computer Interaction. I realized that this approach was very ambitious, but my curiosity and fascination for 3D was so strong that I wanted to explore all aspects of this field. I learned a lot about the different research communities, their approaches and their expectations. Finally, I got a paper rejected at each of the leading conferences (SIGGRAPH, CVPR, CHI).
I decided to join the industry, also because I became a father and wanted to have a more stable work environment. I was lucky to find a position at SICK AG, where I could transfer my research on 3D sensing into a real product. I learned a lot about the challenges of product development, and I realized that my fascination for 3D was not only about publishing papers, but also about creating real-world applications together with experts from different fields. It was a very rewarding experience to discuss open problems of 3D sensing with colleagues having a PhD in electrical engineering, mechanical engineering or optics. However, the industry is very different from academia, and I missed the freedom to explore new ideas and develop applications for the greater good instead of maximizing profits.
Back in academia at Hochschule Furtwangen, I realized two things: First, working with undergrad students at a university of applied sciences is different than working with experts in a research and development department at a company. I realized that it is hard work to teach students and I reflected a lot about my own learning path in the context of the completely changed landscape of media technologies. Those findings are addressed in my teaching portfolio, where I describe my approach to teaching and learning.
Second, the research landscape has changed dramatically in the last decade. The amount of publications in the field of machine learning and computer vision has increased tremendously, and it is impossible to keep track of all new developments. With the help of tools like ResearchRabbit or ScholarInbox, it is possible to keep track of the most relevant publications and to find related work. In addition, recent advances in agentic AI lead to the best practices of research experiments being executed by AI agents, which can be trained to perform specific tasks and to explore the research landscape in a more efficient way. I have first experiences with this approach, but we are very limited by the availability of computation power at Hochschule Furtwangen. I am very motivated to explore this topic further while I also have strong doubts whether this approach will lead to a better understanding of the underlying research questions.
In the field of 3D sensing, the already mentioned geometric foundation models appear to have a similar impact as ChatGPT had in the field of natural language processing. The ability to reconstruct 3D scenes from 2D images is a game changer for many applications, and I am very motivated to explore the potential of these models in my research, while also critically reflecting on their limitations.
In the following current and past projects as well as other research oriented activitiesare listed.
Hochschule Furtwangen University (since 2021)
Furtwangen Media Students Days 2026
I gave a small workshop on “3D reconstruction” at the Furtwangen Media Students Days 2026 in June 2026. The workshop was attended by about 30 students and we had a lot of fun together. We captured together photos of the room and reconstructed a 3D model of the room with the help of several tools like COLMAP, DepthAnything3 and VGGT-Omega. The results were quite good and the students were very happy with the outcome.
IMeRT Courage Hub
I am a funding member of the “Immersive Media in Research and Teaching (IMeRT) Courage Hub”. We connect across locations and faculties researchers and teachers who work together on theoretical and practical questions on a wide range of topics, such as: Mixed Reality (MR), Virtual Reality (VR), Augmented Reality (AR) and 3D audio environments. We offer a platform for joint projects, research, communication, and the use of resources and infrastructure. IMeRT brings together expertise in research, development and teaching in the field of extended reality at our locations in Furtwangen, Schwenningen, Freiburg and Tuttlingen.
We are also the central point of contact for industry partners, companies and institutions seeking collaboration in the field of immersive media.
Smart Mirror
A new tandem project together with Prof Dr. Angela Dieterich, PhD in which we develop a smart mirror prototype that is able to capture 3D data of the user in front of a screen. The mirror is intended to be used in therapy settings as well as information and advertising scenarios. The project is still ongoing.
MünsterAR
In this project, we develop an Augmented Reality application for the Freiburger Münsterbauverein. The application focuses on visualizing 3D models of the gargoyles. It started with the master thesis of Moreli Paredes in 2023. Using NeRF based 3D capturing methods, we were able to create several 3D models of the gargoyles. The next step is to integrate the models into an AR application that allows users to explore the gargoyles in detail.
Keep your eyes - a point&click mystery
In this student project about creating a point&click adventure game initiated by students we developed a first tutorial level of a horror mystery point&click adventure game. See the documentation and check out the code for further details.
Free3D
A master students research project about exploring technologies to create 3D videos. The input are be three Azure Kinect cameras from which a 3D scene is captured and reconstructed. 3D Gaussian Splatting and NeRF provide highly potential options. See our project page for further results.
News: We presented the project at the European Machine Vision Forum 2024 in Mulhouse.
Teaching3D
The IAF Tandem Project Teaching3D aimed to integrate 3D technologies into teaching in order to improve the presentation of complex, spatial content. The focus was particularly on the use of time-of-flight cameras for 3D capture of teachers. This technology makes it possible to capture teachers in real time and create virtual 3D models. This allows augmented reality (AR) content to be used to visualize difficult concepts and hybrid teaching formats to be created in which teachers can appear virtually at any location.
The project led to significant advances, including the development of software frameworks for integrating Azure Kinect data into AR applications and research into current 3D technologies such as Neural Radiance Fields (NeRF). Challenges such as high computational requirements and dependence on specialized GPU hardware were also identified.
Projection Mapping in the wild
A student project that I supervised together with Regina Reusch and VIOSO. The final project results can are shown on this website.
VISMO
My first student project at Furtwangen University. The project page is not available any more, and you have to read the final report (in German) for the details and results.
Past projects
SICK AG Waldkirch (2012 - 2021)
Most projects with SICK have been confidential.
Smart 3D cameras
Our work on smart 3D cameras was presented at the “77. Heidelberger Bildverarbeitungsforum” by my team member Andreas Richert.
Change detection
We patented a method for change detection that was implemented in one of our products.
Hololens
A joint work with our subsidary in Linköping (Sweden) using the Hololens for 3D user interaction was presented and published at the var² conference in Chemnitz in 2017.
TU Berlin (2006 - 2012)
These were the main two research topics I was working on at TU Berlin:
Combining time-of-flight with stereo
I work on this topic for about two years. It all started with a student project “Scanning real world objects without worries”. Subsequently, I supervised the diploma thesis of Antoine Mischler who focusses on real-time stereo and depth estimation for augmented reality applications. Don’t hesitate to contact me if you are interested in my work.
Multi-touch
I also do some research on multi-touch. We built a multi-touch table during a course project with a couple of students. Sorry, but the project website is offline, but feel free to contact me in order to get more information. See the publications page for a poster that has been presented at Siggraph 2008.
Side projects
Kinect
When the first Kinect for XBox camera was first released in 2010, it became a hot topic in the Computer Vision and Graphics research and artist scene. It was very cheap and an early reverse engineering hack allowed 3D data access via OpenNI. As a side project I supported Markus Wambsganss in the creation of 200-DOF. A short film from 2012 that was shot with DSLR, Kinect and Super8 technology.
Weimar (2000 - 2006)
This is a list of projects and seminar works I realized during my studies at Bauhaus University Weimar.
Diploma thesis: “Weighting in Laplacian Mesh Editing”
This diploma thesis has been created in cooperation with DaimlerChrysler. A software has been developed that supports the assembly simulations performed by design engineers. An assembly simulation examines the collision of units. These virtual units are presented as triangular meshes and they are currently solid and undeformable in contrast to real world objects. This software simulates the natural solidness of the units by allowing to deform the virtual objects in a natural manner. To do this, a Laplacian mesh editing scheme has been implemented. Laplacian mesh editing allows to deform 3D objects while their surface details are preserved. Detail preservation is an important feature because it allows the simulation of realistic deformations.
The thesis focuses on examining the effects of different weighting schemes for a Laplacian mesh editing method. This method supports the assembly simulation which is performed during the phase of construction in automotive design. Thus, it has been adapted to the special requirements of such an application by the use of different weights. The deformations are computed by the help of a minimisation method. The present work describes the usage of alternative weights within these minimisation and its effects. In addition, an automatic weighting method has been developed that allows rapid editing tasks.
Slides (PDF), Full text (PDF), Video: Bunny (AVI), Video: Sphere (AVI)
Optical Tracking
During two projects we worked on tracking input devices for virtual reality applications. In the first part, we tracked LEDs an reconstructed their position in 3D space with a multi view setup of several small firewire cameras. In the second part, we worked on tracking colored objects with a single camera and on computing the position of a rack of laser pointers by tracking their light spots and calculate back their origin.
Final presentation slides of the second part. (PDF)
Gasblasen
In industry gas bubbles are used to clean oil. The path of these bubbles want to be predictable. In our project we analyzed the path of a bubble in a water tank captured by two video cameras. We described the path as a spiral in order to minimze the error while having a simple description of the path of any further bubble in any other liquid. Unfortunately, we never received more data to prove our system.
German slides (PDF), German documenation (PDF)
Holographics
First, I did some research on digital holography. In order to print a hologram we took a picture of a car’s headlamp from each viewing angle (see video). The whole process is described in the term paper and the slides of our presentation. I joined the holographics research project in 2004. The aim of the project was to combine holograms with interactive computer graphics.