Postdoctoral Research Fellow - VLM

At Michigan Engineering, we develop the talent and technologies that move society forward and serve our state and national interests. Through discovery and innovation, we create the foundational knowledge and practical technologies to solve today's most pressing challenges and power industries to change lives. Our programs and community promote personal well-being and achievement, enabling everyone to unlock their potential and contribute with confidence.

About the role

Robotic manipulation in manufacturing environments, particularly pick-and-place operations, remains challenging when parts and environmental conditions vary. Conventional robotic systems rely on fixed object categories and require extensive manual reprogramming each time a new part or variant is introduced, limiting scalability and slowing deployment in real-world production settings.

The Connected and Intelligent Manufacturing Systems (CIMS) Lab at the University of Michigan, in partnership with General Motors, is developing a Vision-Language Model (VLM)-based adaptive perception framework for robotic manipulation. This framework aims to enable robots to interpret semantic task instructions and generalize to new or variant parts with minimal reprogramming, reducing the time and cost of deploying perception models while improving robustness to real-world variability such as lighting changes and occlusions. The project spans model benchmarking, perception algorithm development, and integration and validation on physical robotic hardware.

We are looking for a postdoctoral researcher to lead the technical execution of this project, with work performed on-site at the sponsor's facility.

Responsibilities

• Evaluate and benchmark vision-language models for adaptive robotic perception in industrial applications
• Develop robust perception methods for object segmentation and pose estimation under real-world conditions such as occlusion, lighting variation, and diverse part geometries
• Integrate perception components into a real-time pipeline and validate on robotic manipulation hardware
• Prepare technical reports, manuscripts, and presentations documenting research outcomes
• Advise student research assistants supporting the project

Requirements

• PhD in Mechanical Engineering, Robotics, Computer Science, Electrical Engineering, or a related field
• Strong background in computer vision and/or machine learning, with hands-on experience in deep learning frameworks
• Strong demonstrated communication skills

Additional Information

The Connected and Intelligent Manufacturing Systems (CIMS) Lab is an interdisciplinary team dedicated to advancing the intelligence, quality, and efficiency of manufacturing. Alumni have pursued successful careers in academia (e.g., City University of Hong Kong) and industry (e.g., Meta, OpenAI, Google, KLA, Intel, Apple, 3M, C3 AI, Solventum, Milwaukee Tool, Lucid, Seagate, and MathWorks).

More details about the group are available at https://shaolab.engin.umich.edu/.

Work mode is at the discretion of the hiring department and subject to annual review.