Marc Toussaint
Photo: SCIoI
Marc Toussaint leads the Learning & Intelligent Systems Lab. For SCIoI, he represents the synthetic disciplines at the intersections of AI planning, machine learning, and robotics. In his view, a key in understanding and creating intelligence is the interplay of learning and reasoning, where learning becomes the enabler for strongly generalizing reasoning and acting in our physical world. Within SCIoI, he is interested in studying computational methods and representations to enable efficient learning and general purpose physical reasoning, and demonstrating such capabilities on real-world robotic systems.
Projects
Marc Toussaint is member of:
• Project 30: Analyzing human physical reasoning and strategy exploration on physical puzzles
• Project 39: Physical manipulation planning with differentiable closed-loop manipulation primitives
• Project 46: Mouse lock box 2
• Project 39: Physical manipulation planning with differentiable closed-loop manipulation primitives
• Project 46: Mouse lock box 2
Publications +
Levit, S., & Toussaint, M. (2025). Regrasp Maps for Sequential Manipulation Planning. IROS 2025. https://doi.org/10.48550/arXiv.2507.12407
Levit, S., Ortiz-Haro, J., & Toussaint, M. (2024). Solving Sequential Manipulation Puzzles by Finding Easier Subproblems. IEEE International Conference on Robotics and Automation. https://doi.org/10.48550/arXiv.2405.02053
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., Huang, W., Chebotar, Y., Sermanet, P., Duckworth, D., Levine, S., Vanhoucke, V., Hausman, K., Toussaint, M., Greff, K., … Florence, P. (2023). PaLM-E: An Embodied Multimodal Language Model. ICML 2023. https://doi.org/10.48550/arXiv.2303.03378
Zhou, H., Schubert, I., Toussaint, M., & Oguz, O. S. (2023). Spatial Reasoning via Deep Vision Models for Robotic Sequential Manipulation. IROS 2023. https://doi.org/10.1109/iros55552.2023.10342010
Grote, P., Ortiz-Haro, J., Toussaint, M., & Oguz, O. S. (2023). Neural Field Representations of Articulated Objects for Robotic Manipulation Planning. arXiv: 2309.07620. https://doi.org/10.48550/arXiv.2309.07620
Driess, D., Schubert, I., Florence, P., Li, Y., & Toussaint, M. (2022). Reinforcement Learning with Neural Radiance Fields. NeurIPS 2022. https://doi.org/10.48550/arXiv.2206.01634
Driess, D., Huang, Z., Li, Y., Tedrake, R., & Toussaint, M. (2022). Learning Multi-Object Dynamics with Compositional Neural Radiance Fields. CoRL 2022. https://doi.org/10.48550/arXiv.2202.11855
Harris, J., Driess, D., & Toussaint, M. (2022). FC3: Feasibility-Based Control Chain Coordination. IROS 2022. https://doi.org/10.1109/IROS47612.2022.9981758
Ortiz-Haro, J., Karpas, E., Katz, M., & Toussaint, M. (2022). A Conflict-driven Interface between Symbolic Planning and Nonlinear Constraint Solving. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2022.3191948
Ha, J.-S., Driess, D., & Toussaint, M. (2022). Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Input. IEEE Robotics and Automation Letters. https://doi.org/10.1109/LRA.2022.3194955
Kamat, J., Ortiz-Haro, J., Toussaint, M., Pokorny, F. T., & Orthey, A. (2022). BITKOMO: Combining Sampling and Optimization for Fast Convergence in Optimal Motion Planning. IROS 2022. https://doi.org/10.1109/IROS47612.2022.9981732
Toussaint, M., Harris, J., Ha, J.-S., Driess, D., & Hönig, W. (2022). Sequence-of-Constraints MPC: Reactive Timing-Optimal Control of Sequential Manipulation. IROS 2022. https://doi.org/10.1109/IROS47612.2022.9982236
Ortiz-Haro, J., Ha, J.-S., Driess, D., & Toussaint, M. (2022). Structured deep generative models for sampling on constraint manifolds in sequential manipulation. CoRL 2021. https://argmin.lis.tu-berlin.de/papers/21-ortiz-CORL.pdf
Driess, D., Ha, J.-S., & Toussaint, M. (2021). Learning to solve sequential physical reasoning problems from a scene image. The International Journal of Robotics Research, 40(12–14), 1435–1466. https://doi.org/10.1177/02783649211056967
Driess, D., Ha, J.-S., Toussaint, M., & Tedrake, R. (2021). Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning. CoRL 2021. https://doi.org/10.48550/arXiv.2110.00792
Roy, N., Posner, I., Barfoot, T., Beaudoin, P., Bengio, Y., Bohg, J., Brock, O., Depatie, I., Fox, D., Koditschek, D., Lozano-Perez, T., Mansinghka, V., Pal, C., Richards, B., Sadigh, D., Schaal, S., Sukhatme, G., Therien, D., Toussaint, M., & Van de Panne, M. (2021). From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence. arXiv. https://doi.org/10.48550/arXiv.2110.15245
Schubert, I., Driess, D., Oguz, O. S., & Toussaint, M. (2021). Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics. NeurIPS 2021. https://doi.org/10.48550/arXiv.2111.07908
Driess, D., Ha, J.-S., & Toussaint, M. (2020). Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image. arXiv. https://doi.org/10.48550/arXiv.2006.05398
VDI Nachrichten – May 2023 - Robotik- und KI- Forschung sind von ganzheitlichem Design noch weit entfernt
Deutschlandfunk – February 2021 - Big Data, Big Theory – welcher Weg führt zu Erkenntnis?
TU Intern – December 2020 - Wenn Roboter Physik lernen
