CEMPS funded PhD Studentship in Computer Science: Understanding Recurrent Neural Networks Dynamics Ref: 2543

About the award

Location: Streatham Campus, University of Exeter, EX4 4QJ

Primary Supervisor: Dr Lorenzo Livi

Project Description:

Many dynamical systems are naturally representable as networks characterized by time-variant properties, which are related to the topology and/or features associated with vertices and edges. Of particular interest are systems that also perform a computation when driven by an external input signal. Recurrent neural networks (RNNs) constitute an important example in this direction. RNNs are universal approximators of dynamical systems with the capability to generate complex dynamics, store the history of input signals in their transient dynamics and perform inference accordingly. Favoured by their biological plausibility, RNNs are also gaining increasing attention in neuroscience.

Nonetheless, RNNs depend on several (hyper-)parameters, whose tuning highly affect the performance on the application of interest. Moreover, the recurrent nature of their architecture complicates training and hampers interpretability of internal dynamics. This poses constraints on the applicability of such models, which are usually treated as black-box and related hyper-parameters are tuned by cross-validation. The interpretability issue is particularly important, since scientific knowledge can be acquired from experiments only if such models allow for an interpretation. Cross-validation, although effective in practice, comes with shortcomings, since it requires to evaluate the performance on a validation set, hence affecting the overall computational complexity. Moreover, in real-life applications the amount of data may be limited and/or supervised information not always available.

Unsupervised learning offers an alternative framework to address such problems. Several unsupervised methods for tuning RNN (hyper-)parameters have been proposed in the literature. Of particular interest are methods aiming at identifying the so-called edge of criticality, namely the set of (hyper-)parameters for which RNNs denote a dynamical behaviour between order and chaos. Experimental results show that, once operating in such configurations, RNNs usually achieve the best performance in terms of prediction accuracy and memory capacity, which are the quantities generally monitored by supervised approaches.

The goal of this project is to take advantage of the notion of criticality to design unsupervised learning methods contextualized within a graph-theoretical framework. Graph-theoretical approaches are expected to:

1) Provide a way to interpret RNN behaviour by using the language of graph theory

2) Offer a formal framework suitable for the identification of critical behaviour in RNNs and design of related learning mechanisms.

Contact for informal enquiries:

For more information about the project and informal enquiries, please contact Dr Lorenzo Livi (l.livi@exeter.ac.uk)

Application criteria:

Competitive candidates should be highly motivated and possess a BSc or MSc degree in computer science, electrical engineering, applied mathematics, or physics, with a strong background in mathematics and algorithms. Prior research experience in machine learning and related fields is desirable. Applicants are expected to have also a solid background in programming and computational techniques. Successful candidates must be self-motivated and demonstrate an interest to work in interdisciplinary research environments.

If English is not your first language you will need to meet the English language requirements and provide proof of proficiency. Click here for more information and a list of acceptable alternative tests.


Application deadline:4th June 2017
Number of awards:1
Value:3.5 year studentship including UK/EU/International tuition fees plus a stipend equivalent to the RCUK rate (£14,553 for 2017/18)
Duration of award:per year
Contact: Postgraduate Research Officeemps-pgr-ad@exeter.ac.uk

How to apply

To apply, complete the online form. You will be asked to submit some personal details and upload a full CV, covering letter and details of two academic referees. Your covering letter should outline your academic interests, prior research experience and reasons for wishing to undertake this project. 

Interviews will be held at Streatham Campus week commencing 12 June 2017. This studentship is to start in September 2017.

Please quote reference 2543 on your application and in any correspondence about this studentship.