People

Marc Toussaint

Principal Investigator

Robotics

TU Berlin

 

Email:

 

Photo: SCIoI

← People Overview

Marc Toussaint

Marc Toussaint

Photo: SCIoI

Marc Toussaint leads the Learning & Intelligent Systems Lab. For SCIoI, he represents the synthetic disciplines at the intersections of AI planning, machine learning, and robotics. In his view, a key in understanding and creating intelligence is the interplay of learning and reasoning, where learning becomes the enabler for strongly generalizing reasoning and acting in our physical world. Within SCIoI, he is interested in studying computational methods and representations to enable efficient learning and general purpose physical reasoning, and demonstrating such capabilities on real-world robotic systems.

Projects

Marc Toussaint is member of:

• Project 30: Analyzing human physical reasoning and strategy exploration on physical puzzles
• Project 39: Physical manipulation planning with differentiable closed-loop manipulation primitives
• Project 46: Mouse lock box 2
Grote, P., Ortiz-Haro, J., Toussaint, M., & Oguz, O. S. (2023). Neural Field Representations of Articulated Objects for Robotic Manipulation Planning. arXiv: 2309.07620. https://doi.org/10.48550/arXiv.2309.07620
Zhou, H., Schubert, I., Toussaint, M., & Oguz, O. S. (2023). Spatial Reasoning via Deep Vision Models for Robotic Sequential Manipulation. IROS 2023. https://doi.org/10.1109/iros55552.2023.10342010
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., Huang, W., Chebotar, Y., Sermanet, P., Duckworth, D., Levine, S., Vanhoucke, V., Hausman, K., Toussaint, M., Greff, K., … Florence, P. (2023). PaLM-E: An Embodied Multimodal Language Model. ICML 2023. https://doi.org/10.48550/arXiv.2303.03378
Ortiz-Haro, J., Karpas, E., Katz, M., & Toussaint, M. (2022). A Conflict-driven Interface between Symbolic Planning and Nonlinear Constraint Solving. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2022.3191948
Ha, J.-S., Driess, D., & Toussaint, M. (2022). Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Input. IEEE Robotics and Automation Letters. https://doi.org/10.1109/LRA.2022.3194955
Toussaint, M., Harris, J., Ha, J.-S., Driess, D., & Hönig, W. (2022). Sequence-of-Constraints MPC: Reactive Timing-Optimal Control of Sequential Manipulation. IROS 2022. https://doi.org/10.1109/IROS47612.2022.9982236
Ortiz-Haro, J., Ha, J.-S., Driess, D., & Toussaint, M. (2022). Structured deep generative models for sampling on constraint manifolds in sequential manipulation. CoRL 2021. https://argmin.lis.tu-berlin.de/papers/21-ortiz-CORL.pdf
Kamat, J., Ortiz-Haro, J., Toussaint, M., Pokorny, F. T., & Orthey, A. (2022). BITKOMO: Combining Sampling and Optimization for Fast Convergence in Optimal Motion Planning. IROS 2022. https://doi.org/10.1109/IROS47612.2022.9981732
Harris, J., Driess, D., & Toussaint, M. (2022). FC3: Feasibility-Based Control Chain Coordination. IROS 2022. https://doi.org/10.1109/IROS47612.2022.9981758
Driess, D., Huang, Z., Li, Y., Tedrake, R., & Toussaint, M. (2022). Learning Multi-Object Dynamics with Compositional Neural Radiance Fields. CoRL 2022. https://doi.org/10.48550/arXiv.2202.11855
Driess, D., Schubert, I., Florence, P., Li, Y., & Toussaint, M. (2022). Reinforcement Learning with Neural Radiance Fields. NeurIPS 2022. https://doi.org/10.48550/arXiv.2206.01634
Driess, D., Ha, J.-S., & Toussaint, M. (2021). Learning to solve sequential physical reasoning problems from a scene image. The International Journal of Robotics Research, 40(12–14), 1435–1466. https://doi.org/10.1177/02783649211056967
Roy, N., Posner, I., Barfoot, T., Beaudoin, P., Bengio, Y., Bohg, J., Brock, O., Depatie, I., Fox, D., Koditschek, D., Lozano-Perez, T., Mansinghka, V., Pal, C., Richards, B., Sadigh, D., Schaal, S., Sukhatme, G., Therien, D., Toussaint, M., & Panne, M. V. de. (2021). Nicholas Roy; Ingmar Posner; Tim Barfoot; Philippe Beaudoin; Yoshua Bengio; Jeannette Bohg; Oliver Brock; Isabelle Depatie; Dieter Fox; Dan Koditschek; Tomas Lozano-Perez; Vikash Mansinghka; Christopher Pal; Blake Richards; Dorsa Sadigh; Stefan Schaal; Gaurav Sukhatme; Denis Therien; Marc Toussaint; Van de Panne, Michiel. arXiv. https://doi.org/10.48550/arXiv.2110.15245
Schubert, I., Driess, D., Oguz, O. S., & Toussaint, M. (2021). Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics. NeurIPS 2021. https://doi.org/10.48550/arXiv.2111.07908
Driess, D., Ha, J.-S., Toussaint, M., & Tedrake, R. (2021). Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning. CoRL 2021. https://doi.org/10.48550/arXiv.2110.00792

Research

An overview of our scientific work

See our Research Projects