Master
Lab Course Humanoid Robots
Robots are versatile systems, that provide vast opportunities for active research and various operations. Humanoid robots, for example, have a human-like body, and thus can act in environments designed for humans. They are able to, e.g., climb stairs, walk through cluttered environments, and open doors. Mobile robots with a wheeled base are designed to operate on flat grounds to perform, e.g., cleaning and service tasks. Robotic arms are able to grasp and manipulate objects.
Participants will work in group of 2 or 3 on one of the possible topics.
At the end of the semester each group will give a presentation and demonstration of their project accompanied by oral moderation. The whole presentation should be approximately 10 minutes long. Every member of the group should present his/her part in the development of the system in a few sentences/slides. When the presentation is complete, each group will be asked a few questions by the HRL staff members or preferably the other students. Everyone is required to be present and to watch the presentation of the other groups.
Project codes must be pushed to the group's git repository before the lab presentation.
The grade of the lab will depend on the final presentation and how well the assigned task was solved (30/70).
Participants are expected to have Ubuntu Linux installed on their personal computers. The specific requirements for each project are quoted below.
The mandatory Introductory Meeting take place in person (see important dates below).
Links:
Lecturers:
Important dates:
All interested students have to attend the Introductory Meeting. In the Introductory Meeting, we will present the projects, the schedule, the registration process, and answer your questions.
After the Introductory Meeting, each participant arranges an individual schedule with the respective supervisor.
Registration
The registration will be open soon.
Presentation template
Please use the following template for midterm and final presentation:
[Presentation template]
Projects:
Open Vocabulary Mobile Manipulation: Semantic Rearrangement from Natural Language Instructions using Scene Graphs, LLMs and VLA
Supervisor: Rohit Menon
This project aims to convert natural language instructions for semantic rearrangement into task-level plans using scene graphs and LLMs. The final manipulation is then carried out using a custom VLA. The project is for 3 students
Project Description: ovmm.pdf
VisTaR: Visuo-Tactile In-Hand Object Pose Refinement for a Multi-Finger Robot Hand
Supervisor: Rohit Menon, Benno WIngender
This project aims to first estimate the pose of an unknown object using few-shot RGB pose estimator and then refine it during grasping using kinematic constraints and tactile sensing on a multi-fingered hand. This project is for 3 students.
Project Description: vistar.pdf
Privacy-constrained robot navigation using VLM
Supervisor: Xuying Huang
This project aims to use a vision language model (VLM) to achieve robot navigation under privacy constraints. The robot should be able to understand spoken instructions, process the command using the VLM, and convert it into specific actions (such as moving to the kitchen).
This project is for 2 students.
Language-Guided Robot Rearrangement
Supervisor: Ahmed Shokry
This project aims to develop a language-guided robotic rearrangement framework that learns reusable manipulation behaviors from separate sub-task demonstrations rather than complete task trajectories. A vision-language model will interpret the scene and user instruction to define an object-relation goal, while a goal-conditioned critic evaluates candidate action chunks generated by a Flow Matching policy and coordinates behaviors such as opening drawers, picking objects, and placing them at the requested target
Project Description: PDF
Socially NEAT Evolutionary Learning for Social Robot Navigation
Supervisor: Subham Agrawal, Rohit Menon
This lab project investigates the combination of neuroevolution, imitation learning, and reinforcement learning for socially compliant mobile robot navigation. The students will develop largely independent work packages with clearly defined standalone outcomes and subsequently integrate their components into a common social-navigation pipeline. All controller training, reinforcement learning, and evolutionary optimization are performed in simulation. Real-world experiments are restricted to deployment and evaluation of already trained controllers on a Husarion mobile robot.
Project Description: PDF
Autonomous Racing
Supervisor: Subham Agrawal, Shahram Khorshidi
This Lab will involve fixed meetings where the basics of the car will be explained, with exercise sheets to be completed. This will lead to a final project in which the task is to enable a racing car to navigate a given track autonomously.
Project Description: PDF
Find the viewpoint!
Supervisor: Sicong Pan
The task is to find the viewpoint of the given RGB image within the context of an eye-in-hand tabletop configuration.
Project Description: PDF
Whole Body Control for Opening Furnitures
Supervisors: Rohit Menon, Shahram Khorshidi
Opening articulated objects requires the end effector to remain constrained by the drawer or hinge while the mobile base preserves reachability and avoids collisions. Whole-body mobile manipulation addresses these coupled arm–base motions rather than treating navigation and manipulation independently.