MLL Lab
Multimodal Learning and Language
We develop intelligent language + X (vision, robotics, etc) models that reason, plan, and interact with the physical world.
Latest News
We co-organized the KnowFM: Towards Knowledgeable Foundation Models Workshop at ACL 2026.
We organized the FAGEN: Failure Modes in Agentic AI Workshop at ICML 2026, co-organized by Manling Li and Zihan Wang.
Our RAGEN-2 received the Best Paper Award (Top 1%) at the CVPR 2026 MMRAgI Workshop!
We organized the 2nd Foundation Models Meet Embodied Agents (FMEA) Workshop at CVPR 2026.
Our RAGEN-2: Reasoning Collapse in Agentic RL is accepted to ICML 2026 as an Oral Presentation (Top 0.7%).
Our ENACT received two Outstanding Paper Awards at the ICLR 2026 World Models and Lifelong Agents Workshops.
Our FedAgent received the Best Paper Award (TrustAgent @ AAAI 2026) and an Outstanding Paper Award (PerFM @ AAAI 2026)!
Multiple papers are accepted to ICLR 2026, including ENACT, MindCube, Theory of Space and ODE-Steer!
We co-organized the ResponsibleFM: Socially Responsible and Trustworthy Foundation Models Workshop at NeurIPS 2025.
Our VAGEN is accepted to NeurIPS 2025, and featured by MIT Tech Review and Stanford HAI.
Prof. Manling Li is named to MIT Technology Review's Innovators Under 35 (Global List, 2025)!
Our MindCube received the Best Paper Award at the ICCV 2025 SP4V Workshop, and was selected as The Best of ICCV by Voxel51.
Our EmbodiedBench is accepted to ICML 2025 as an Oral Presentation (Top 1%).
Will organize the CVPR 2025 Workshop on Foundation Models for Embodied Agents, with Ruohan Zhang, Yunzhu Li, Jiayuan Mao and Wenlong Huang, Qineng Wang, Yonatan Bisk, Shenlong Wang, Fei-Fei Li and Jiajun Wu. Calling for papers!
Will organize the tutorial on Foundation Models for Embodied Agents at AAAI 2025 and NAACL 2025, with Yunzhu Li, Jiayuan Mao and Wenlong Huang.
Prof. Manling Li is selected as AAAI 2025 New Faculty Highlights.
Our Embodied Agent Interface has been selected as Best Paper Award (top 0.4%) at SoCal NLP 2024!
Our Embodied Agent Interface has been selected as Oral Presentation (top 0.6% over D&B track, top 0.4% over all tracks) at NeurIPS 2024!
Selected Publications
2026
- Embodied First Exam
Kangrui Wang*, Qineng Wang*, Changhe Chen*, Pingyue Zhang*, Chi Gui*, Feishi Wang, Canyu Chen, Wenli Xiao, Xiaoyu Huang, Tonghe Zhang, Linjie Li, Zihan Wang, Minyeong Kim, Yiyou Sun, Mingkai Deng, Jinyu Hou, Eric P. Xing, Han Liu, Qi Zhu, Huaxiu Yao, Zhiwen Fan, Yuejiang Liu, Dawn Song, Pieter Abbeel, Jiajun Wu, Yuke Zhu, Guanya Shi+, Ruohan Zhang+, Manling Li+
2026
- RAGEN-2: Reasoning Collapse in Agentic RL
Zihan Wang*, Chi Gui*, Xing Jin*, Qineng Wang*, Licheng Liu*, Kangrui Wang, Shiqi Chen, Linjie Li, Zhengyuan Yang, Pingyue Zhang, Yiping Lu, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, Manling Li
ICML 2026
Oral (Top 0.7%) Best Paper (Top 1%) at CVPR 2026 MMRAgI WorkshopHugging Face #2 Paper of the Day, Invited Talk at MIT Media Lab
- Spatial Mental Modeling from Limited Views
Qineng Wang*, Baiqiao Yin*, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu+, Li Fei-Fei+, Manling Li+
ICLR 2026
Best Paper Award at ICCV 2025 SP4V Workshop Best Paper Honorable Mention at NeurIPS 2025 LAW WorkshopThe Best of ICCV (featured by Voxel51), Adopted by Gemini 3 Pro
- ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
Qineng Wang*, Wenlong Huang*, Yu Zhou, Hang Yin, Tianwei Bao, Jianwen Lyu, Weiyu Liu, Ruohan Zhang+, Jiajun Wu+, Li Fei-Fei+, Manling Li+
ICLR 2026
Outstanding Paper Award at ICLR 2026 World Models Workshop Outstanding Paper Award at ICLR 2026 Lifelong Agents Workshop Spotlight at NeurIPS 2025 LAW Workshop - Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?
Pingyue Zhang*, Zihan Huang*, Yue Wang*, Jieyu Zhang*, Letian Xue, Zihan Wang, Qineng Wang, Keshigeyan Chandrasegaran, Ruohan Zhang, Yejin Choi, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Manling Li
ICLR 2026
Spotlight at ACL 2026 Workshop on Advances in Language and Vision ResearchFeatured by MIT Tech Review
2025
- RAGEN (RL-Agent): Training Agents by Reinforcing Reasoning
Zihan Wang*, Kangrui Wang*, Qineng Wang*, Pingyue Zhang*, Linjie Li*, Zhengyuan Yang, Kefan Yu, Minh Nhat Nguyen, Monica Lam, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, Manling Li
MMLS 2025 (Midwest Machine Learning Symposium)
Best Poster Award 2.7k+ Github Stars, Featured by MIT Tech Review, Lambda Partner Spotlight, VentureBeat, Medium, AI News, MarkTechPost, Business Leaders Review, etc. - VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents
Kangrui Wang*, Pingyue Zhang*, Zihan Wang*, Yaning Gao*, Linjie Li*, Qineng Wang, Chi Wan, Hanyang Chen, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Yejin Choi, Manling Li
NeurIPS 2025
Featured by MIT Tech Review, Stanford HAI
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, Heng Ji, Huan Zhang, Tong Zhang
ICML 2025
Oral (Top 1%)
2024
- Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
Manling Li*, Shiyu Zhao*, Qineng Wang*, Kangrui Wang*, Yu Zhou*, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, Jiajun Wu
NeurIPS 2024 D&B Track
Oral (Top 0.6%) Best Paper Award at SoCal NLP 2024, Top 0.4%