Human-robot collaboration as end-to-end learning

  • Human-robot collaboration as end-to-end learning

School of Electronics, Electrical Engineering and Computer Science
& ECIT Global Research Institute

Proposed Project Title: Human-robot collaboration as end-to-end learning

Principal Supervisor: Vien Ngo                                    Second Supervisor:  Neil Robertson

Project Description:

Recently, there have been rapid development in both computing power and robotic hardwares. We now have faster robot brains and more precise, robust and compliant robotic sensor, actuators and devices. On the other hands with a very significant advancements of artificial intelligence and deep learning, we also have more powerful algorithms in learning and data processing such as via end-to-end learning techniques. The robots now are able to perform more challenging tasks and to master complex and dexterous skills more quickly.

However there is still one big gap to make robots living more feasibly in human-friendly unstructured environment and working alongside with human. It is necessary to come up with a new learning paradigm to allow the robot able to learn/communicate/collaborate/reason with human. To make a smooth and better collaboration, the robot is required to a) understand common-sense knowledge, task purpose and context, human as collaboration party intention; b) then to decide what to query, ask, or give offers to the human.

Previous work has done this via separate processes: parse commands (in texts or speech using NLP or spoken language undestanding) from the human into symbolic sentences, map symbolic sentences into machine-understanding state or action representation, learn/plan a new policy based on this new information, translate the policy into low-level force control for execution.

In this project, we aim at constructing a human-robot collaboration framework to let a robot learning by natural instructions (via texts, speech). Learning is expected to be end-to-end which is less manual pre-defining or with saparate stages. This study will transfer all advanced techniques from NLP, speech understanding, learning from demonstration, and end-to-end robot learning into one framework.

Contact details

Supervisor Name: Vien Ngo                                                                             Tel: +44 (0)28 9097 1824
QUB Address: 03-032 CSB                                                                               Email: