Semester and Thesis Projects
The ETH AI Center offers a wide range of semester and thesis projects for students at ETH Zurich, as well as other universities. Please see the list below for projects that are currently available.
Are you a student? Check out our Semester and Thesis projects below!
Need help? If you have any questions, feel free to request them at .
Online Safe Locomotion Learning in the Wild
Reinforcement learning (RL) can potentially solve complex problems in a purely data-driven manner. Still, the state-of-the-art in applying RL in robotics, relies heavily on high-fidelity simulators. While learning in simulation allows to circumvent sample complexity challenges that are common in model-free RL, even slight distribution shift ("sim-to-real gap") between simulation and the real system can cause these algorithms to easily fail. Recent advances in model-based reinforcement learning have led to superior sample efficiency, enabling online learning without a simulator. Nonetheless, learning online cannot cause any damage and should adhere to safety requirements (for obvious reasons). The proposed project aims to demonstrate how existing safe model-based RL methods can be used to solve the foregoing challenges.
Keywords
safe mode-base RL, online learning, legged robotics
Labels
Master Thesis
Description
Contact Details
More information
Published since: 2024-05-03
Organization ETH Competence Center - ETH AI Center
Hosts Li Chenhao , Li Chenhao , Li Chenhao , Li Chenhao
Topics Engineering and Technology
Lifelike Agility on ANYmal by Learning from Animals
The remarkable agility of animals, characterized by their rapid, fluid movements and precise interaction with their environment, serves as an inspiration for advancements in legged robotics. Recent progress in the field has underscored the potential of learning-based methods for robot control. These methods streamline the development process by optimizing control mechanisms directly from sensory inputs to actuator outputs, often employing deep reinforcement learning (RL) algorithms. By training in simulated environments, these algorithms can develop locomotion skills that are subsequently transferred to physical robots. Although this approach has led to significant achievements in achieving robust locomotion, mimicking the wide range of agile capabilities observed in animals remains a significant challenge. Traditionally, manually crafted controllers have succeeded in replicating complex behaviors, but their development is labor-intensive and demands a high level of expertise in each specific skill. Reinforcement learning offers a promising alternative by potentially reducing the manual labor involved in controller development. However, crafting learning objectives that lead to the desired behaviors in robots also requires considerable expertise, specific to each skill.
Keywords
learning from demonstrations, imitation learning, reinforcement learning
Labels
Master Thesis
Description
Contact Details
More information
Published since: 2024-03-25
Organization ETH Competence Center - ETH AI Center
Hosts Li Chenhao , Li Chenhao , Klemm Victor
Topics Information, Computing and Communication Sciences
Learning Real-time Human Motion Tracking on a Humanoid Robot
Humanoid robots, designed to mimic the structure and behavior of humans, have seen significant advancements in kinematics, dynamics, and control systems. Teleoperation of humanoid robots involves complex control strategies to manage bipedal locomotion, balance, and interaction with environments. Research in this area has focused on developing robots that can perform tasks in environments designed for humans, from simple object manipulation to navigating complex terrains. Reinforcement learning has emerged as a powerful method for enabling robots to learn from interactions with their environment, improving their performance over time without explicit programming for every possible scenario. In the context of humanoid robotics and teleoperation, RL can be used to optimize control policies, adapt to new tasks, and improve the efficiency and safety of human-robot interactions. Key challenges include the high dimensionality of the action space, the need for safe exploration, and the transfer of learned skills across different tasks and environments. Integrating human motion tracking with reinforcement learning on humanoid robots represents a cutting-edge area of research. This approach involves using human motion data as input to train RL models, enabling the robot to learn more natural and human-like movements. The goal is to develop systems that can not only replicate human actions in real-time but also adapt and improve their responses over time through learning. Challenges in this area include ensuring real-time performance, dealing with the variability of human motion, and maintaining stability and safety of the humanoid robot.
Keywords
real-time, humanoid, reinforcement learning, representation learning
Labels
Master Thesis
Description
Contact Details
More information
Published since: 2024-03-25
Organization ETH Competence Center - ETH AI Center
Hosts He Junzhe , Li Chenhao , Li Chenhao
Topics Information, Computing and Communication Sciences
Continuous Skill Learning with Fourier Latent Dynamics
In recent years, advancements in reinforcement learning have achieved remarkable success in teaching robots discrete motor skills. However, this process often involves intricate reward structuring and extensive hyperparameter adjustments for each new skill, making it a time-consuming and complex endeavor. This project proposes the development of a skill generator operating within a continuous latent space. This innovative approach contrasts with the discrete skill learning methods currently prevalent in the field. By leveraging a continuous latent space, the skill generator aims to produce a diverse range of skills without the need for individualized reward designs and hyperparameter configurations for each skill. This method not only simplifies the skill generation process but also promises to enhance the adaptability and efficiency of skill learning in robotics.
Keywords
representation learning, periodic autoencoders, learning from demonstrations, policy modulating trajectory generators
Labels
Master Thesis
Description
Contact Details
More information
Published since: 2024-03-25
Organization ETH Competence Center - ETH AI Center
Hosts Li Chenhao , Rudin Nikita
Topics Information, Computing and Communication Sciences , Engineering and Technology
Universal Humanoid Motion Representations for Expressive Learning-based Control
Recent advances in physically simulated humanoids have broadened their application spectrum, including animation, gaming, augmented and virtual reality (AR/VR), and robotics, showcasing significant enhancements in both performance and practicality. With the advent of motion capture (MoCap) technology and reinforcement learning (RL) techniques, these simulated humanoids are capable of replicating extensive human motion datasets, executing complex animations, and following intricate motion patterns using minimal sensor input. Nevertheless, generating such detailed and naturalistic motions requires meticulous motion data curation and the development of new physics-based policies from the ground up—a process that is not only labor-intensive but also fraught with challenges related to reward system design, dataset curation, and the learning algorithm, which can result in unnatural motions. To circumvent these challenges, researchers have explored the use of latent spaces or skill embeddings derived from pre-trained motion controllers, facilitating their application in hierarchical RL frameworks. This method involves training a low-level policy to generate a representation space from tasks like motion imitation or adversarial learning, which a high-level policy can then navigate to produce latent codes that represent specific motor actions. This approach promotes the reuse of learned motor skills and efficient action space sampling. However, the effectiveness of this strategy is often limited by the scope of the latent space, which is traditionally based on specialized and relatively narrow motion datasets, thus limiting the range of achievable behaviors. An alternative strategy involves employing a low-level controller as a motion imitator, using full-body kinematic motions as high-level control signals. This method is particularly prevalent in motion tracking applications, where supervised learning techniques are applied to paired input data, such as video and kinematic data. For generative tasks without paired data, RL becomes necessary, although kinematic motion presents challenges as a sampling space due to its high dimensionality and the absence of physical constraints. This necessitates the use of kinematic motion latent spaces for generative tasks and highlights the limitations of using purely kinematic signals for tasks requiring interaction with the environment or other agents, where understanding of interaction dynamics is crucial. We would like to extend the idea of creating a low-level controller as a motion imitator to full-body motions from real-time expressive kinematic targets.
Keywords
representation learning, periodic autoencoders
Labels
Master Thesis
Description
Contact Details
More information
Published since: 2024-03-25
Organization ETH Competence Center - ETH AI Center
Hosts Li Chenhao , Li Chenhao , Li Chenhao
Topics Information, Computing and Communication Sciences , Engineering and Technology
Humanoid Locomotion Learning and Finetuning from Human Feedback
In the burgeoning field of deep reinforcement learning (RL), agents autonomously develop complex behaviors through a process of trial and error. Yet, the application of RL across various domains faces notable hurdles, particularly in devising appropriate reward functions. Traditional approaches often resort to sparse rewards for simplicity, though these prove inadequate for training efficient agents. Consequently, real-world applications may necessitate elaborate setups, such as employing accelerometers for door interaction detection, thermal imaging for action recognition, or motion capture systems for precise object tracking. Despite these advanced solutions, crafting an ideal reward function remains challenging due to the propensity of RL algorithms to exploit the reward system in unforeseen ways. Agents might fulfill objectives in unexpected manners, highlighting the complexity of encoding desired behaviors, like adherence to social norms, into a reward function. An alternative strategy, imitation learning, circumvents the intricacies of reward engineering by having the agent learn through the emulation of expert behavior. However, acquiring a sufficient number of high-quality demonstrations for this purpose is often impractically costly. Humans, in contrast, learn with remarkable autonomy, benefiting from intermittent guidance from educators who provide tailored feedback based on the learner's progress. This interactive learning model holds promise for artificial agents, offering a customized learning trajectory that mitigates reward exploitation without extensive reward function engineering. The challenge lies in ensuring the feedback process is both manageable for humans and rich enough to be effective. Despite its potential, the implementation of human-in-the-loop (HiL) RL remains limited in practice. Our research endeavors to significantly lessen the human labor involved in HiL learning, leveraging both unsupervised pre-training and preference-based learning to enhance agent development with minimal human intervention.
Keywords
reinforcement learning from human feedback, preference learning
Labels
Master Thesis
Description
Goal
Contact Details
More information
Published since: 2024-03-25
Organization ETH Competence Center - ETH AI Center
Hosts Li Chenhao , Li Chenhao , Chen Xin , Li Chenhao
Topics Information, Computing and Communication Sciences , Engineering and Technology
Deep Learning and Data Collection in Speech Recognition for Individuals with Complex Congenital Disorders
Complex congenital disorders often result in speech and motor skill impairments, posing communication challenges. Existing non-English speech recognition tools struggle with non-standard speech patterns, compounded by a lack of large training datasets. This project aims to create a personalized framework for training German speech recognition models, catering to the unique needs of individuals with congenital disorders. You will learn to collect data and apply machine learning or deep learning models.
Keywords
Machine Learning, Deep Learning, Speech recognition, Natural Language Processing, Speech Recognition
Labels
Master Thesis
Description
Contact Details
More information
Published since: 2024-03-12
Organization ETH Competence Center - ETH AI Center
Hosts Vo Anh
Topics Information, Computing and Communication Sciences , Engineering and Technology , Behavioural and Cognitive Sciences
Towards AI Safety: Adversarial Attack & Defense on Neural Controllers
The project is collaborating between SRI and RSL/CRL lab and aims to investigate the weakness of the neural controller based on the state-of-the-art [3] attacking method.
Keywords
Adversarial attack; safe AI; Reinforcement learning
Labels
Semester Project , Master Thesis
PLEASE LOG IN TO SEE DESCRIPTION
More information
Published since: 2024-03-06 , Earliest start: 2024-03-06 , Latest end: 2024-09-30
Applications limited to ETH Zurich , EPFL - Ecole Polytechnique Fédérale de Lausanne
Organization Robotic Systems Lab
Hosts Shi Fan , Shi Fan , Shi Fan
Topics Information, Computing and Communication Sciences , Engineering and Technology
Learning Diverse Adversaries to Black-box Learning-based Controller for Quadruped Robots
The project aims to leverage the latest unsupervised skill discovery techniques to validate the state-of-the-art black-box learning-based controllers in diverse ways.
Keywords
Diversity in RL, Trustworthy AI
Labels
Semester Project , Master Thesis
PLEASE LOG IN TO SEE DESCRIPTION
More information
Published since: 2024-03-02 , Earliest start: 2024-03-02 , Latest end: 2024-08-28
Applications limited to ETH Zurich , [nothing]
Organization Robotic Systems Lab
Hosts Shi Fan , Shi Fan , Shi Fan
Topics Information, Computing and Communication Sciences
Towards interpretable learning pipeline: A visual-assisted workflow for locomotion learning
Current reinforcement learning (RL)-based locomotion controllers have shown promising performance. However we are still not clear about what is learned during the training process. In this project, we investigate the proper metrics and visualisation techniques to interactively steer the locomotion learning tasks.
Keywords
Reinforcement learning; visualization; interpretable AI
Labels
Semester Project , Master Thesis
PLEASE LOG IN TO SEE DESCRIPTION
More information
Published since: 2024-02-28 , Earliest start: 2024-02-26 , Latest end: 2024-08-26
Organization ETH Competence Center - ETH AI Center
Hosts Zhang Xiaoyu , Shi Fan , Wang April , Shi Fan , Shi Fan
Topics Engineering and Technology
Misestimation of CT-perfusion output in acute stroke due to attenuation curve truncation
In this master's thesis project, we are looking for a candidate to apply machine learning techniques to correct and predict signals of incomplete CT perfusion imaging for ischemic stroke. We hope to use machine learning techniques to de-noise and correct for the truncation in CT perfusion signals. In particular, we aim to infer the true attenuation curve after the truncation time-point.
Keywords
machine learning; CT perfusion imaging; ischemic stroke; contrast-media attenuation time-curves;
Labels
Master Thesis
Description
Goal
Contact Details
More information
Published since: 2024-02-22 , Earliest start: 2024-06-01
Organization Bjoern Menze
Hosts Davoudi Neda , Yang Kaiyuan
Topics Medical and Health Sciences , Information, Computing and Communication Sciences