Deep relational reinforcement learning

  • Deep relational reinforcement learning

School of Electronics, Electrical Engineering and Computer Science
& ECIT Global Research Institute

Proposed Project Title: Deep relational reinforcement learning

Principal Supervisor:   Vien Ngo                     Second Supervisor:  Neil Robertson

Project Description:

Deep reinforcement learning (DRL) has shown many great successes  such as demonstrations by Google Deepmind in Atari video games and the game of Go, robotic manipulations. The idea of DRL is to bring the power of deep learning to operate on reinforcement learning (trial-and-error learning). However, existing DRL methods have many shortcomings which are also inherent in any current deep learning techniques. Specifically, they always require a massive amount of data to work effectively, this causes a very slow adoption of DRL in practical tasks such as sequential decision making and robotics. Moreover, they clearly fail in making reasoning on an abstract level, which makes it difficult for applications in tasks of relational learning, transfer learning, hierarchical learning, hypothesis-based reasoning, and common-sense knowledge learning.

In tasks of sequential decision making and robotics, we often see that the tasks can be designed hierarchically or with abstraction. For example in a navigation task in a city, at task-level a human would first choose to navigate through a sequence of junctions. After that, at low-level he will make decisions to select how to accelerate and stop in order to navigate from one junction to next ones such that minimizing the cost of energy and travel time and maximizing his comfort. Similar to a robotic manipulation task, at abstract-level the robot would first decide on task-level how to move from one location to the others, and decide to work with a certain object (grasp or place), before he would choose how to exert forces to move his body or to control his arms at low-level control.

There has been recent effort in making deep relational learning or symbolic neural networks, but very little in realizing deep relational reinforcement learning. In this project, the student is expected to investigate how to bring advances and advantages of deep learning, relational learning and reinforcement learning together. The project is built on previous work by the supervisor ‘s research on relational reinforcement learning and deep reinforcement learning.



Contact details

Supervisor Name: Vien Ngo                                                                             Tel: +44 (0)28 9097 1824
QUB Address: 03-032 CSB                                                                               Email: v.ngo@qub.ac.uk