Wide area multi-target tracking over multiband imaging sensor networks for military applications

  • Wide area multi-target tracking over multiband imaging sensor networks for military applications
EPSRC Industrial Cooperative Awards in Science & Technology (ICASE) PHD

Proposed Project Title:

  • Wide area multi-target tracking over multiband imaging sensor networks for military applications

Principal Supervisor(s):
  • Jesus Martinez del Rincon (QUB) & Alf Casement (Thales UK)

Project Description:

Technological advances in sensing have fully transformed military operations, allowing remote monitoring and intervention while reducing the dangers for pilots and soldiers.  In particular, the use of thermal and RGB cameras, both on fix-ground location or on vehicles, as well as on Unmanned Aerial Vehicles (UAVs) with imaging capabilities, have leveraged the acquisition of critical information on potential threats, intelligence and surveillance.  In this context, target detection and tracking is a fundamental task required for tactical operations and reconnaissance.  The increasing ubiquity of these sensors on the field increases not only the coverage but its success at the cost of increasing the resources to process the large data flow.

The automatisation of target tracking allows addressing efficiently this scenario while overcoming the human limitations, such as the need of multitasking or the decrease of performance during long missions.  In recent years, advances in research have made staggering progress on multi-target tracking in both visible spectrum aerial cameras and infrared imaginery for both pedestrians  and vehicles. Initial attempts of improving tracking by combining both modalities have been already proposed, although both sensors are limited to be mounted in the same vehicle and perspective. While satisfactory results can be achieved for traffic and vehicle monitoring, given the more predictable nature of their movement, pedestrians still remain a challenge, particularly in environments where visual clues are not always discriminative, as it is the case in military applications. More importantly, most existing research in the field assumes tracking is confined within a single sensor’s field of view, rather than addressing the complexity of tracking over a network of non-overlapping sensors. This not only reduced coverage but also assumes no interruption of visual contact happens during the duration of the mission, resulting on increasing chances of losing a critical target.

In this PhD project, we aim to tackle the previous limitations of the state of the art by proposing a holistic view of tracking, where target detection, multi target tracking and target re-identification across sensors are solved together in a unified framework. The scenario to be solved comprises multiple objects being observed and transiting between multiple non-overlapping cameras, both infrared and RGB, mounted on the ground and/or on UAVs.


To achieve this goal, we propose a unified wide area tracking framework based on Deep Neural networks. We aim to explore three complementary aspects:

  • Multi-band re-identification algorithm: to develop a re-identification system able to preserve the identity of the targets when they move across non overlapping sensors, as well as reappearances after long occlusions. The proposed system is based on Deep learning Siamese architectures  that enable the join optimisation of feature extraction and metric learning for this task. Given the particularities of military scenarios, the network will be trained using different combinations of RGB, infrared and motion channels –such as optical flow- in order to avoid over-dependency on visual clues and allow the re-identification when only partial information is given by the sensor.
  • Multi target tracking based on LSTM and recurrent networks: based on the novel use of Deep Learning for multi-target tracking for solving the inherent data association and optimisation problem involved in multi-target tracking. Very preliminary attempts to tracking using neural networks have been recently proposed in the literature, but limited to single-target tracking or using simulated results. We aim to extend those approaches to an effective multi-target tracking system.
  • Wide area tracking framework: to combine the previous research outcomes in a unique Deep Neural architecture. This will be possible thanks to the use of neural networks in both components that can be here joined into a unified architecture.  This will allow an end-to-end learning paradigm that will maximise the performance of all the components. The result of this will be the first proposed wide-area framework based on deep learning.


Academic Requirements:

Students entering the programme will normally be required to have a 2.1 BSc/BEng in Computer Science, Electrical and Electronic Engineering, or a maths based engineering or physical science degree, or equivalent qualification recognised by the University.

Students holding an appropriate MEng or MSc (Software conversion) will normally be required to have a 2.1 or commendation (distinction) respectively. Furthermore, additional criteria may be applied. All applicants must have significant mathematical and programming experience.

General Information:

This 4 year PhD studentship is funded by Engineering and Physical Sciences Research Council (EPSRC), the UKs main agency for funding research in engineering and the physical sciences, and Thales UK.

Full funding during the 4 years is provided (£19,000 stipend + fees included per annum).

Student will also benefit of close collaboration and supervision from the industrial partner (Thales) as well as an internship of a minimum of 3 months within Thales. Full conditions (https://www.epsrc.ac.uk/skills/students/coll/icase/intro/) and eligibility (https://www.epsrc.ac.uk/skills/students/help/eligibility/) are available online.

Applicants should apply electronically through the Queens online application portal at: https://dap.qub.ac.uk/portal/

Further information available at: http://www.qub.ac.uk/schools/eeecs/Study/PostgraduateResearch/

Contact details:

Supervisor Name: Jesus Martinez del Rincon

Queens University of Belfast
School of EEECS,
NI Science Park,
Queen’s Road,
Queen’s Island,

Email: j.martinez-del-rincon@qub.ac.uk
Tel: +44 (0)28 9097 1779