Human-robot collaboration as end-to-end learning

  • Human-robot collaboration as end-to-end learning

Human-robot collaboration as end-to-end learning

Principal Supervisor: Dr. Vien Ngo

Second Supervisor:  Prof. Neil Robertson

+ Project Description

Recently, there have been rapid development in both computing power and robotic hardwares. We now have faster robot brains and more precise, robust and compliant robotic sensor, actuators and devices. On the other hands with a very significant advancements of artificial intelligence and deep learning, we also have more powerful algorithms in learning and data processing such as via end-to-end learning techniques. The robots now are able to perform more challenging tasks and to master complex and dexterous skills more quickly. However there is still one big gap to make robots living more feasibly in human-friendly unstructured environment and working alongside with human. It is necessary to come up with a new learning paradigm to allow the robot able to learn/communicate/collaborate/reason with human. To make a smooth and better collaboration, the robot is required to a) understand common-sense knowledge, task purpose and context, human as collaboration party intention; b) then to decide what to query, ask, or give offers to the human. Previous work has done this via separate processes: parse commands (in texts or speech using NLP or spoken language undestanding) from the human into symbolic sentences, map symbolic sentences into machine-understanding state or action representation, learn/plan a new policy based on this new information, translate the policy into low-level force control for execution.

In this project, we aim at constructing a human-robot collaboration framework to let a robot learning by natural instructions (via texts, speech). Learning is expected to be end-to-end which is less manual pre-defining or with saparate stages. This study will transfer all advanced techniques from NLP, speech understanding, learning from demonstration, and end-to-end robot learning into one framework.

+ How to Apply

Applicants should apply electronically through the Queen’s online application portal at:

+ Contact Details

Supervisor Name: Dr. Vien Ngo

03-032 CSB      



+44 (0)28 9097 1824