Welcome to the

Workshop on Whole-body Control and Bimanual Manipulation: Applications in Humanoids and Beyond

at CoRL 2024


Read More

About

WCBM @ CoRL 2024

Motivation

Humanoid robots have long held promise to be seamlessly deployed in our daily lives. Despite the rapid progress in humanoid robots' hardware (e.g. Boston Dynamics Atlas, Tesla Optimus, Unitree H1, 1X Neo, Agility Digit), their software is fully or partially hand-designed for specific tasks. The goal of Whole-body Control and Bimanual Manipulation (WCBM) workshop is to provide a platform for roboticists and machine learning researchers to discuss the past achievements of whole-body control and manipulation as well as future research directions on this topic, especially on the problem of enabling autonomous humanoid robots. We invited a group of speakers who are world-renowned experts to present their work on whole-body control for humanoid robots and bimanual robotic systems. We also want to provide researchers with the opportunity to present their latest research by accepting workshop papers. We will review and select the best submissions for spotlight talks and interactive poster presentations. We have also planned guided panel discussions to encourage debate among the invited speakers and workshop participants. In the discussion, we would like to contrast the viewpoints of machine learning researchers and roboticists on the past and future of this research topic.

If you have any questions, feel free to contact us.

Program

Workshop schedule (tentative)

🚪Room "Taurus 2"

TimeEvent
  8:30  -   8:35Opening Remark
  8:35  -   9:15Jitendra Malik / Toru Lin (UC Berkeley)
  9:15  -   9:40Yuke Zhu (UT Austin, NVIDIA)
  9:40  -  10:05Manuel Galliker (1X Technologies)
  10:05  -  10:30Vikash Kumar (CMU)
10:30  -  11:00Coffee Break, Posters, Robot Demos
11:00  -  11:20Xingxing Wang (Unitree)
11:20  -  11:40Quentin Rouxel on behalf of Serena Ivaldi (Inria)
11:40  -  12:00Ziwen Zhang (Tsinghua University)
12:00  -  13:45Lunch, Robot Demos
13:45  -  14:10Xiaolong Wang (UCSD)
14:10  -  14:35Moritz Baecher (Disney Research)
14:35  -  15:00Chelsea Finn (Stanford, PI)
15:00  -  15:30Coffee Break, Posters, Robot Demos
15:30  -  15:55Jonathan Hurst (Oregon State Univ., Agility Robotics)
15:55  -  16:20Ashish Kumar (Tesla)
16:20  -  16:45Deepak Pathak (CMU)
16:45  -  17:10Scott Kuindersma (Boston Dynamics)
17:10  -  18:00Panel Discussion
18:00  -  18:10Closing Remarks / Best Paper Award

Talks

Invited Speakers

Chelsea Finn

Chelsea Finn is an Assistant Professor in Computer Science and Electrical Engineering at Stanford University, the William George and Ida Mary Hoover Faculty Fellow, and a co-founder of Physical Intelligence (Pi). Her research interests lie in the capability of robots and other agents to develop broadly intelligent behavior through learning and interaction. To this end, her work has pioneered end-to-end deep learning methods for vision-based robotic manipulation, meta-learning algorithms for few-shot learning, and approaches for scaling robot learning to broad datasets. Her research has been recognized by awards such as the Sloan Fellowship, the IEEE RAS Early Academic Career Award, and the ACM doctoral dissertation award, and has been covered by various media outlets including the New York Times, Wired, and Bloomberg. Prior to joining Stanford, she received her Bachelor's degree in Electrical Engineering and Computer Science at MIT and her PhD in Computer Science at UC Berkeley.

Scott Kuindersma

Scott Kuindersma is the Senior Director of Robotics Research at Boston Dynamics where he leads behavior research on Atlas. Prior to joining Boston Dynamics, he was an Assistant Professor of Engineering and Computer Science at Harvard. Scott’s research explores intersections of machine learning and model-based control to improve the capabilities of humanoids and other dynamic mobile manipulators.

Yuke Zhu

Yuke Zhu is an Assistant Professor in the Computer Science department of UT-Austin, where he directs the Robot Perception and Learning (RPL) Lab. He is also a core faculty at Texas Robotics and a senior research scientist at NVIDIA. He focuses on developing intelligent algorithms for generalist robots and embodied agents to reason about and interact with the real world. His research spans robotics, computer vision, and machine learning. He received his Master's and Ph.D. degrees from Stanford University. His works have won several awards and nominations, including the Best Conference Paper Award in ICRA 2019, Outstanding Learning Paper at ICRA 2022, Outstanding Paper at NeurIPS 2022, and Best Paper Finalists in IROS 2019, 2021, and RSS 2023. He received the NSF CAREER Award and the Amazon Research Awards.

Jitendra Malik

Jitendra Malik is Arthur J. Chick Professor of EECS at UC Berkeley. His research has spanned computer vision, machine learning, modeling of human vision, computer graphics, and most recently robotics. He has advised more than 70 Ph.D. students and postdocs, many of whom are now prominent researchers. His honors include numerous best paper prizes, the 2013 Distinguished Researcher award in computer vision, the 2016 ACM/AAAI Allen Newell Award, the 2018 IJCAI Award for Research Excellence in AI, and the 2019 IEEE Computer Society’s Computer Pioneer Award for “leading role in developing Computer Vision into a thriving discipline through pioneering research, leadership, and mentorship”. He is a member of the National Academy of Sciences, the National Academy of Engineering, and the American Academy of Arts and Sciences.

Toru Lin

Toru Lin is a second-year PhD student at Berkeley AI Research (BAIR), advised by Jitendra Malik and Alexei Efros. Previously, she obtained her BSc and MEng from MIT EECS, under the supervision of Phillip Isola. During her undergraduate years at MIT, she was also advised by Jiajun Wu and Antonio Torralba. Before transferring to MIT, she was an undergraduate student at The University of Tokyo. She also has interned at DeepMind, Facebook, and Google. She is building real-world robotic systems that can improve themselves through interaction with the environment.

Ashish Kumar

Ashish Kumar is a research engineer at Tesla working on Tesla Optimus. He received his Ph.D. from UC Berkeley, advised by Prof. Jitendra Malik. Before that, he was a Research Fellow at Microsoft Research India, where he worked with Dr. Manik Varma and Dr. Prateek Jain on developing Resource Efficient Machine Learning algorithms. He is broadly interested in Robotics, with a focus on developing efficient deployable algorithms for locomotion and navigation. He completed his undergraduate study at Indian Institute of Technology Jodhpur, with a major in Computer Science and Engineering.

Xingxing Wang

Xingxing Wang the Founder/CEO/CTO of Unitree, a technology supremacist. During his master’s degree, he pioneered the development of XDog, a high-performance quadruped robot driven by a low-cost external rotor brushless motor, pioneering the technology of low-cost, high-performance footed robots. After graduating in 2016, joined DJI. At the same time, the XDog robot was reported by many media at home and abroad, and received a huge response in the global robotics circle. He then resigned and founded Unitree in August 2016. It became the first company in the world to publicly retail high-performance quadruped robots, leading global sales year after year, significantly promoting the commercialization process of global high-performance quadruped robots. He has been invited to give speeches at the ICRA 2018~2022 Footed Robot Forum, the top international robotics conference, and has applied for more than 150 domestic and foreign patents. Led the company to obtain multiple rounds of investment from Sequoia, Shunwei, Matrix Partners, etc. He once led a team to have the company’s products appear on the CCTV Year of the Ox Spring Festival Gala stage, the opening ceremony of the Winter Olympics, and the Super Bowl. 2023 Fortune China’s 40 business elites under 40 years old.

Deepak Pathak

Deepak Pathak is a faculty in the School of Computer Science at Carnegie Mellon University. He received his Ph.D. from UC Berkeley and his research spans computer vision, machine learning, and robotics. He is a recipient of the faculty awards from Google, Samsung, Sony, GoodAI, and graduate fellowship awards from Facebook, NVIDIA, Snapchat. His research has been featured in popular press outlets, including The Economist, The Wall Street Journal, Quanta Magazine, Washington Post, CNET, Wired, and MIT Technology Review among others. Deepak received his Bachelor's from IIT Kanpur with a Gold Medal in Computer Science. He co-founded VisageMap Inc. later acquired by FaceFirst Inc.

Moritz Baecher

Moritz Bächer is a Research Scientist at Disney Research, where he leads the Computational Design and Manufacturing group. He is deeply passionate about solving real-world problems in computational robotics, fabrication, and architecture. His core expertise is the development of differentiable simulators to tackle complex design, control, and characterization problems in (soft) robotics, architecture, and computer graphics. Before joining Disney, he received a Ph.D. from the Harvard School of Engineering and Applied Sciences and graduated with a master’s from ETH Zurich.

Junathan Hurst

Jonathan Hurst is a Professor of Robotics, co-founder of the Oregon State University Robotics Institute, and Chief Technology Officer and co-founder of Agility Robotics. He holds a B.S. in mechanical engineering and an M.S. and Ph.D. in robotics, all from Carnegie Mellon University. His university research focuses on understanding the fundamental science and engineering best practices for legged locomotion. Investigations range from numerical studies and analysis of animal data, to simulation studies of theoretical models, to designing, constructing, and experimenting with legged robots for walking and running, and more recently, using machine learning techniques merged with more traditional control to enable highly dynamic gaits. Agility Robotics is extending this research to commercial applications for robotic legged mobility, working towards a day when robots can go where people go, generate greater productivity across the economy, and improve quality of life for all.

Vikash Kumar

Vikash Kumar is an adjunct professor at the Robotics Institute, CMU. He finished his Ph.D. from the University of Washington with Prof. Emo Todorov and Prof. Sergey Levine, where his research focused on imparting human-level dexterity to anthropomorphic robotic hands. He continued his research as a post-doctoral fellow with Prof. Sergey Levine at University of California Berkeley where he further developed his methods to work on low-cost scalable systems. He also spent time as a Research Scientist at OpenAI and Google-Brain where he diversified his research on low-cost scalable systems to the domain of multi-agent locomotion. He has also been involved with the development of the MuJoCo physics engine, now widely used in the fields of Robotics and Machine Learning. His works have been recognized with the best Master's thesis award, best manipulation paper at ICRA’16, best workshop paper ICRA'22, CIFAR AI chair '20 (declined), and have been widely covered with a wide variety of media outlets such as NewYorkTimes, Reuters, ACM, WIRED, MIT Tech reviews, IEEE Spectrum, etc.

Ziwen Zhuang

Ziwen Zhuang is a PhD student at IIIS, Tsinghua University. He is also working as a research assistant with Professor Hang Zhao at Shanghai Qi Zhi Institute. He completed his Master’s degree at ShanghaiTech University, advised by Professor Soeren Schwertfeger. Ziwen’s research focuses on robot learning, especially athletic intelligence on legged robots. He published Robot Parkour Learning at CoRL 2023, which achieved Best System Award Finalist. Prior to that, he was a research intern at Carnegie Mellon University working with Professor David Held, and he was the algorithm leader of ShanghaiTech RoboMaster team.

Manuel Yves Galliker

Manuel Yves Galliker is the team lead for controls and embedded at 1X Technologies. He holds a B.Sc. and M.Sc. in mechanical engineering with a focus on robotics, systems and controls from ETH Zurich. Manuel's research interests range from mechantronics to optimal, data-driven and reinforcement learned controls with a focus on locomotion and loco-manipulation. During his academic tenure, he dedicated his efforts of his master thesis to researching online gait generation for bipedal robots using Whole-Body Dynamics in a Nonlinear Model Predictive Control approach, contributing at ETH's Robotics Systems Lab and as a visiting researcher at Caltech's AMBER lab. This work culminated in a publication presented at the IEEE Humanoids 2022 conference, where it was distinguished as a finalist for the Best Paper Award. In his current role he is leading the R&D efforts on controls and embedded for the new bipedal Android NEO. In particular the team is aiming to develop whole body control and planning algorithms, ranging from Centroidal MPC to Whole-Body MPC and Reinforcement Learning, to enable consecutively more and more general loco-manipulation behaviors.

Serena Ivaldi

Serena Ivaldi is a tenured senior research scientist (director of research) in Inria, France, leading the humanoid and human-robot interaction activities of Team Larsen in Inria Nancy, France. She obtained her Ph.D. in Humanoid Technologies in 2011 at the Italian Institute of Technology and University of Genoa, Italy, and the French Habilitation to Direct Research in 2022 at the University of Lorraine, France. Prior to joining Inria, she was a post-doctoral researcher at UPMC in Paris, France, then at the University of Darmstadt, Germany. In 2023 she also spent two months at the European Space Agency as visiting expert. She is currently Editor in Chief of the International Journal of Social Robotics and has been serving as Associate Editor for IEEE Robotics and Automation Letters. She was Program Chair of the conferences IEEE/RAS Humanoids 2019, IEEE ARSO 2023, Tutorial Chair for CORL 2021, and is the General Chair of IEEE/RAS Humanoids 2024. She has been serving IEEE RAS as Associate Vice-president of MAB and co-chair of the ICRA Steering Committee. She was a proud judge of the ANA Avatar Xprize competition in 2022. She received the Suzanne Zivi Prize for excellence in research, the 2021 IEEE RA-L Distinguished Service Award as Outstanding Associate Editor, and was nominated in the 50 Women in robotics you need to know about in 2021.

Xiaolong Wang

Xiaolong Wang is an assistant professor at UC San Diego in the ECE department. He is affliated with the CSE department, Center for Visual Computing, Contextual Robotics Institute, and Artificial Intelligence Group. He is a member of the Robotics team in the TILOS NSF AI Institute. He was a postdoctoral fellow at UC Berkeley with Alexei Efros and Trevor Darrell. He received his Ph.D. in robotics from Carnegie Mellon University, at where he worked with Abhinav Gupta.

Organization

Workshop Organizers

Pieter Abbeel

Pieter Abbeel

Professor at UC Berkeley
Carlo Sferrazza

Carlo Sferrazza

Postdoc at UC Berkeley
Xingyu Lin

Xingyu Lin

Postdoc at UC Berkeley
Youngwoon Lee

Youngwoon Lee

Assistant Professor at Yonsei Unviersity

Program Committee

  • Carlo Sferrazza (UC Berkeley)
  • Youngwoon Lee (Yonsei University)
  • Xingyu Lin (UC Berkeley)

Calls

Call for papers

We welcome submissions of full papers as well as work-in-progress and accept submissions of work recently published or currently under review.

In general, we encourage two types of papers:

  • Empirical paper: Submissions should focus on presenting original research, case studies or novel implementations in the fields related to the workshop (see potential topics below).
  • Position paper: Authors are encouraged to submit papers that discuss critical and thought-provoking topics within the scientific community.
Potential topics include:
  • Reinforcement learning for whole-body control and bimanual manipulation
  • Teleoperation systems for humanoid robots (or other complex robotic systems) and imitation learning
  • Learning models (e.g. dynamics, perception) and planning for complex, mobile robotic systems
  • Benchmark and task proposals for whole-body control and manipulation
  • Multimodal, whole-body sensing and perception
  • Simulation to real world transfer
  • Learning from human videos
Important Dates
  • Submission deadline: October 11 18, 2024
  • Notification of acceptance: October 18 25, 2024
  • Camera-ready papers due: November 1, 2024
  • All deadlines are AoE time.
Submission Guidelines

The WCBM workshop will use OpenReview as the review platform.

Accepted papers will be presented as posters, and a subset of them will be selected for oral presentation.

The paper template and style files can be found at here (adapted based on CoRL 2024 template). There is no page limit but recommended 4-8 pages (excluding references and appendix). Submissions must follow the template and style and should be properly anonymized.

Dual Submission Policy

We welcome papers that have never been submitted, are currently under review, or recently published. Accepted papers will be published on the workshop homepage, but will not be part of the official proceedings and are to be considered non-archival.