Master
Diffusion as View-Set Optimization for Active 3D Reconstruction
Joint MSc thesis project in collaboration with Assistant Professor Marija Popović (TU Delft)
Autonomous 3D scanning requires robots to select informative camera views that jointly reveal an unknown object while avoiding redundant observations and unnecessary motion. Conventional next-best-view approaches typically make local decisions one step at a time, which can lead to short-sighted scanning behaviour.
Recent work has shown two complementary directions. ScanDP [1] uses diffusion policies to generate short-horizon camera motions for adaptive 3D scanning, but primarily learns from demonstrated scanning trajectories. In contrast, SCVP [2] formulates view planning as a set-covering problem and predicts a compact set of globally complementary viewpoints, but operates over predefined discrete candidates.
In this project, we aim to combine these ideas by using diffusion directly for global view-set optimization. Starting from noisy camera poses, a conditional diffusion model will generate and optionally refine a compact set of complementary 6-DoF viewpoints, guided by reconstruction objectives such as surface coverage, visibility, redundancy, and feasibility. The resulting view set will then be converted into an efficient UAV scanning trajectory.
The framework will be evaluated for UAV-based reconstruction of large indoor static objects, such as sculptures or vehicles, with final validation on a real UAV platform at the TU Delft Cyberzoo.
Requirements:
- Enrolled in Computer Science, Robotics, or a related MSc program.
- Strong proficiency in Python; experience with C++ is a plus.
- Experience with robot learning and deep learning frameworks such as PyTorch.
- Familiarity with 3D perception, including point clouds, occupancy maps, or RGB-D reconstruction.
- Experience with ROS and robotic simulation environments is highly desirable.
- Solid background in optimization, motion planning, or active perception is a plus.
- Willingness to travel to TU Delft for real-world UAV experiments and system validation in the Cyberzoo.
- Enthusiasm for hands-on robotic deployment and aiming for a scientific publication.
Related Work:
[1] I. Hirako, R. Hakoda, Y. Liu, M. Hwang, Y. Sato, and T. Oishi, “ScanDP: Generalizable 3D Scanning with Diffusion Policy,” arXiv:2603.10390, 2026.
[2] S. Pan, H. Hu, and H. Wei, “SCVP: Learning One-Shot View Planning via Set Covering for Unknown Object Reconstruction,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1463–1470, 2022.