Intelligent agents such as robots require the ability to learn and adapt within their environment. Our group investigates novel methods for learning to understand dynamic 3D scenes and their functioning, and uses this knowledge to perform complex tasks such as autonomous navigation and object manipulation. Traditional approaches often integrate perception and control components engineered for specific tasks and scenarios. In contrast, we aim at systems that learn to act and perceive from raw sensor measurements such as images or tactile information and action experience acquired in their environment. We investigate computer vision methods and end-to-end trainable architectures for learning task-relevant representations that allow agents to plan their actions.
Computer Vision for Embodied Agents: Interpreting visual information plays a key role in autonomous systems that act purposefully in their environment. We develop approaches for visual scene reconstruction and understanding in intelligent systems. Specifically, we are interested in computer vision methods for simultaneous localization and mapping, and 3D and dynamic scene understanding. We also investigate approaches for self-supervised learning and adaptation in order to increase the flexibility of systems.
Learning and Perception of Dynamics Models for Control and Planning: In recent years, learning-based approaches have gained traction due to their prospects of enabling robotic agents to learn their skills through interaction and exploration in the environment. This way, robots could adapt faster to novel situations and learn more generalizable representations than manually engineered approaches. In contrast to model-free approaches, which learn policies and value functions with an implicit understanding of the environment, our paradigm is to make environment models explicit and learn these models for model-based control and planning. Challenges in this regard are to learn generalizable models which transfer to a variety of environments and systems, to achieve sample-efficient learning, and to enable robust approaches which are aware of the uncertainty of models.