Skip to Content

Deep learning enhanced inference of human motor intentions for human-robot collaboration

PhD project title

Deep learning enhanced inference of human motor intentions for human-robot collaboration

Outline description, including interdisciplinary, intersectoral and international dimensions 

The role of robots in our society is ever-increasing. Robots are devised to autonomously perform many tasks currently performed by humans. In designing robots, situations involving human-robot collaboration – very likely to occur for instance on factory floors that cannot yet be completely automated – specific challenges remain. The control of robot movement must be finely coordinated with the movements of the human counterparts. Both the robot and human must infer intentions from the observed movements of their counterparts. Indeed, humans can infer much about another biological entity from rudimentary features of its movements (called biological motion). For human-robot interactions, additional feedback systems could inform the human about a robot’s upcoming (intended) movements, since these intentions can be accessed. The reverse is not true: a robot does not have access to the movement planning stages of the human, which occur inside the brain. For a robot to infer the intentions of their human counterpart, reliance on principles of biological motion thus seems essential. Indeed, human movements – particularly in constrained tasks – are relatively stereotypical, which may help in predicting upcoming (intended) movements. This project aims to use modern computational intelligence methods (machine learning, deep learning) to predict human (arm) movements based on features extracted from vision (e.g., optical motion tracking, video cameras) with the specific aim to improve performance of a constrained task that requires human-robot motor coordination (i.e., cooperation). Where possible and needed, the algorithms will be constrained or informed by knowledge about the human ability to extract intention from biological motion. This project is interdisciplinary at its core, given the key role of computer science, engineering, motor neuroscience, and psychology. This is strongly reflected in the supervisory team.


Key words/descriptors

Machine learning, movement control, decoding intentions, biological motion

Fit to CITI-GENS theme(s)

The project sits at the interface between the ‘Advanced Manufacturing’ and ‘Information Technology’ themes.   It draws on information technology, specifically AI and computer vision, as tools to enable robots to become more intelligent and human like in their interaction with humans.  While this has broad applicability, with the likely increasing prevalence of automation in our society, the specific context and need that motivates the research is collaborative tasks in a manufacturing environment, for example, assembly of complex parts.

Supervisor Information



First Supervisor:        Prof. Seán McLoone                                                               School:  EEECS, QUB

Second Supervisor:   Dr.  Joost C. Dessing                                                         School: Psychology, QUB

Third Supervisor:       Dr. Vien Ngo,  Research Scientist                                  Company: Bosh Centre for AI (Germany)

Name of non-HEI partner(s)

Bosh Centre for AI, Renningen, Germany


Contribution of non-HEI partner(s) to the project:



The fellow will have the opportunity to work in the robotics team at Bosch Corporate Research (Bosch Centre for AI) during his/her placement and will have access to several robot platforms within the Centre that perform the manipulation tasks being investigated in the project.  The industrial supervisor, Dr Vien Ngo is an expert in machine learning for robotics, and will contribute to the supervision of the PhD student in this regard.


Non-HEI supervisor: Dr Vien Ngo

Dr. Vien Ngo is a research scientist at The Bosch Centre for Artificial Intelligence (BCAI). His team's research interest is focused on AI and ML theories for robotic manipulation. The team is in cooperation with Cyber Valley, which is Europe’s largest research consortium in the field of artificial intelligence with partners from science and industry. The state of Baden-Württemberg, the Max Planck Society with the Max Planck Institute for Intelligent Systems, the Universities of Stuttgart and Tübingen, as well as Amazon, BMW AG, Daimler AG, IAV GmbH, Porsche AG, Robert Bosch GmbH, and ZF Friedrichshafen AG are the founding partners of this initiative. Prior to joining BCAI, he was an early career Lecturer at EEECS/ECIT/DSSC at Queen's University Belfast in the UK from September 2017 to Feb 2020. He received the B.S. degree in Computer Engineering from Hanoi University of Science and Technology, Vietnam, in 2005, and the Ph.D degree in Computer Engineering from Kyung Hee University, Republic of Korea, in 2009. He was a postdoctoral researcher at the National University of Singapore from 2009 to 2011, at the Ravensburg-Weingarten University of Applied Sciences in Germany from 2011-2013, and a group leader at the Machine Learning and Robotics lab at the University of Stuttgart from 2013-2017. His research interests include Machine Learning, and Robotics.



 Profile of the non-HEI partner 


Robert Bosch GmbH, or Bosch, is a German multinational engineering and electronics company headquartered in Gerlingen, near Stuttgart, Germany. It is the world's largest supplier of automotive components measured by 2011 revenues. Bosch Cooperate Research: International research network of the corporate sector for research and advance engineering, comprising 12 locations in 8 countries (Palo Alto, Pittsburgh, Boston, Hildesheim, Renningen, St. Petersburg, Moscow, Bangalore, Tel Aviv, Tokyo, Shanghai, and Singapore); other engineering activities related to the development of products and services at some 120 locations worldwide. Bosch Research in Renningen where the fourth supervisor is based, Germany. This goal is to encourage interdisciplinary collaboration (automobile, robotic, medicine), and in this way further enhance its innovative strength. At the new centre in Renningen for research and advance engineering on the outskirts of Stuttgart, some 1,700 creative minds are doing applied industrial research.


Research centre / School

Centre for Intelligent Autonomous Manufacturing Systems

Subject area

Deep Learning and Robotics