My main interest is in the ability of a system to adapt to its surroundings and improve its performance through exposure to a problem. Systems of this type are commonplace in nature, including animal brains, immune systems, evolution of genetic material, the laws and languages of a society, and even the way in which a plant grows towards a light. These and related ideas, have been modelled extensively by researchers in a variety of areas including machine learning, genetic algorithms, reinforcement learning, neuroscience, psychology, ethology, biology, computer science and physics.

The field is large and further distinctions are useful. For example, is adaptation in-lifetime or outside-lifetime? Is learning supervised, unsupervised or reinforced? Is the system natural or artificial? If artificial, is the model symbolic or sub-symbolic? The level of description is also a key distinction - neuronal, behavioural, psychological, social etc.

I am specifically interested in in-lifetime, sub-symbolic approaches to learning with an emphasis on unsupervised and reinforcement techniques applied to the lower levels of description such as those at the focus of behavioural and neuronal modelling. My academic background is in Artificial Intelligence, Cognitive Science and Computer Science, and I am particularly interested in multi-disciplinary approaches to increasing our understanding of intelligence and learning.

My current interests revolve around building computational models of mesolimbic dopamine function. Dopamine is a major neuromodulator in the brain and is implicated in a number of psychiatric disorders including psychosis (schizophrenia), Attention Deficit/ Hyperactivity Disorder (ADHD), Parkinson's disease, Tourette's syndrome and drug addiction. All but drug addiction are conditions for which the first line treatment involves dopamine manipulation. Tourette's and Parkinson's are more likely to involve the Nigrostriatal dopaminergic pathways, but the others are all likely to depend heavily on the mesolimbic dopaminergic system, with particular emphasis on the nucleus accumbens. The diagram below indicates the target brain regions of the mesolimbic dopaminergic system (thought to be implicated in psychosis for example).

• Salamone et. al. (1997) demonstrate that under dopamine blockade, rats prefer a small but easily obtained reward over a previously preferred larger reward that is harder to obtain. If the easy reward is removed then the animal returns to the more inaccessible reward.


• In Attention Deficit/Hyperactivity Disorder (ADHD), the patient is prone to impulsivity. For example, the patient prefers immediate payoffs over holding back for a greater payoff at a later time. The treatment for ADHD involves enhancing dopamine levels.


• Incentive Salience. A key finding presented in Berridge (1998) is that if a rat is deprived of almost all its mesolimbic dopamine then it will stop eating, even though it is apparently capable of producing the motoric responses necessary for doing so, and even though the animal is still apparently capable of feeling the pleasure of the food (if artificially fed).


• A basic Instrumental response finding is that an animal trained to press a lever for a reward, presses more under dopamine agonists and less under dopamine antagonists. Moreover, lever pressing is blockaded at doses which still allow the animal to eat if presented with the food directly. This finding is generalised across different reward types and different instrumental responses.


• The Conditioned Avoidance Response (CAR) paradigm has been used extensively to test the potential effectiveness of antipsychotics for the treatment of schizophrenia. A rat that has learned to avoid a shock (US) by fleeing a conditioned stimulus (CS), then fails to respond appropriately to that stimulus under dopamine blockade. Under enough dopamine blockade, the rat will fail to even escape while being shocked, which may be due to motoric incapacitation, extreme motivational incapacitation, or both.


• Incentive Learning. Balleine and Dickinson et. al. (1995) present experimental observations that if a rat is first trained on an instrumental chain leading to a food reward, and then tested in a sated state, only the animal's motivation for the action proximal to the reward is altered. However, in an interesting twist, the motivation for the distal action is also modulated if the animal is re-exposed to the food in the new motivational state.


• Pavlovian Instrumental Transfer (PIT) refers to the ability of a CS that has previously been paired with a reward, to increase motivation for an instrumental response for that same reward. Interestingly, the CS may still increase motivation for an action that is directed at obtaining a reward different to that predicted by the CS. PIT appears to be affected by dopamine manipulation.