PLOS Computational Biology: Neurobiological Models of Two-Choice Decision Making Can Be Reduced to a One-Dimensional Nonlinear Diffusion Equation
Neurobiological Models of Two-Choice Decision Making Can Be Reduced to a One-Dimensional Nonlinear Diffusion Equation
Alex Roxin, Anders Ledberg
Abstract
The response behaviors in many two-alternative choice tasks are well described by so-called sequential sampling models. In these models, the evidence for each one of the two alternatives accumulates over time until it reaches a threshold, at which point a response is made. At the neurophysiological level, single neuron data recorded while monkeys are engaged in two-alternative choice tasks are well described by winner-take-all network models in which the two choices are represented in the firing rates of separate populations of neurons. Here, we show that such nonlinear network models can generally be reduced to a one-dimensional nonlinear diffusion equation, which bears functional resemblance to standard sequential sampling models of behavior. This reduction gives the functional dependence of performance and reaction-times on external inputs in the original system, irrespective of the system details. What is more, the nonlinear diffusion equation can provide excellent fits to behavioral data from two-choice decision making tasks by varying these external inputs. This suggests that changes in behavior under various experimental conditions, e.g. changes in stimulus coherence or response deadline, are driven by internal modulation of afferent inputs to putative decision making circuits in the brain. For certain model systems one can analytically derive the nonlinear diffusion equation, thereby mapping the original system parameters onto the diffusion equation coefficients. Here, we illustrate this with three model systems including coupled rate equations and a network of spiking neurons.
Author Summary
The brain holds a central position in scientific theories of rational behavior. For example, brain activity is thought to stand in a causal relation to the decision making behavior observed in two-choice perceptual discrimination tasks. Although a lot is known about both the brain activity and the response behavior during these tasks, the relationships between the two are not fully understood. In particular, how can one relate the high-dimensional dynamic activity of the brain to the low-dimensional descriptions of response behavior such as performance and reaction-times? Our approach to this question is to relate existing neurobiological models of brain activity to existing models of response behavior. In this paper we establish a formal link between standard, winner-take-all models of brain activity during two-choice tasks and a family of one-dimensional behavioral models known as diffusion models. Our analysis demonstrates a universal functional dependence between the external inputs to the neural populations in the neurobiological model on the one hand, and reaction times and performance in the one-dimensional model on the other. Importantly, we show that experimentally measured performance and reaction-times can be predicted through changes in these external inputs alone.
Citation: Roxin A, Ledberg A (2008) Neurobiological Models of Two-Choice Decision Making Can Be Reduced to a One-Dimensional Nonlinear Diffusion Equation. PLoS Comput Biol 4(3): e1000046. doi:10.1371/journal.pcbi.1000046
Editor: Lyle J. Graham, UFR Biomédicale de l'Université René Descartes, France
Received: June 12, 2007; Accepted: February 28, 2008; Published: March 28, 2008
Copyright: © 2008 Roxin, Ledberg. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: AR was funded by a Marie Curie Incoming Fellowship. AL was funded by a Juan de la Cierva Fellowship.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In perceptual two-choice decision making experiments one studies how sensory information influences response behavior. In each trial the experimental subject is presented with a stimulus and must use the information thus provided to choose one of two possible responses. The response behavior in these tasks, as defined by reaction times and performance, has been studied for over a hundred years [1]–[3] leading to a wealth of data and modeling results [4]. The reaction times are typically long compared to what would be expected based only on neuronal conduction times and vary considerably from trial to trial. Mean reaction times for error and correct trials are also, in general, found to be different. Moreover, subjects can be instructed to trade speed for accuracy. These facts are believed to reflect, at least in part, the decision making aspects of the tasks as opposed to sensory or motor aspects [1]–[5].
The aim of the work presented here is to account for the response behavior in two-choice decision making tasks in terms of the underlying neurobiology. In the remaining part of this section we will first describe one prominent family of behavioral models of response behavior, the sequential sampling models. Subsequently we will describe some neurophysiological findings, and models thereof, pertinent to our modeling framework.
...
Discussion
One-dimensional diffusion equations have long been used to model behavior in two-choice reaction-time tasks. Recently, researchers discovered that the trial-averaged single-unit activity recorded in areas of the brain which are implicated in generating this behavior closely resemble the dynamics of a linear diffusion process [21]. This suggests a correspondence between the neural activity in these areas and the decision making variable X in the linear diffusion equation. However, it remained unclear how the cortical activity might actually conspire to generate such a linear diffusion. On the other hand, it was soon demonstrated that some aspects of the neural activity could be captured in biophysically motivated winner-take-all network models [28]–[30]. Here we have shown, through the use of standard tools from nonlinear dynamics theory, that the dynamics in winner-take-all models relevant for two-choice decision making can be captured in a one-dimensional nonlinear diffusion equation, Equation 8. This suggests that the cortical circuits involved in decision making generically generate an effective nonlinear diffusion which in a limited parameter regime leads to behavior very similar to that predicted by the linear diffusion equation.
The dependence of the coefficients in Equation 8 on external inputs is explicit and independent of the details of the underlying model. This suggests that the functional dependence of behavioral measures in two-choice decision making on changes in inputs is universal. In particular, we predict that modulations of the input common to both populations can account for the speed-accuracy trade-off. This mechanism differs from that evoked by others previously, which consists of varying the threshold for detection of the decision (e.g. a higher threshold increases reaction times and increases performance), [6],[30]. The novel mechanism proposed here of speed-accuracy trade-off through modulations in the mean input predicts that pre-stimulus activity in LIP should be higher, on average, when the subject must respond more rapidly. Support for this comes from the observation that the baseline neuronal activity in monkeys varies in a task-dependent manner, see Figure 16 from [20], a phenomenon which has been interpreted as anticipatory activity. Indeed, increases in the baseline activity were found to correlate with more rapidly evolving post-stimulus activity. Equation 8 now provides us with an explanation for the functional role of this activity. This phenomenon could be further confirmed through comparison of the relative changes in the BOLD signal in fMRI studies of activity in brain areas in humans homologous to LIP during the pre-stimulus period in a task where the speed-accuracy trade-off is observed behaviorally.
While Equation 8 appears similar in form to other diffusion models which have been used to describe behavior in two-choice decision making [9], [12]–[14],[45], it is important to distinguish between their very distinct mathematical pedigrees. In particular, we have not evoked the nonlinear diffusion equation as a phenomenological model of behavior for two-choice decision making. Rather, it represents the correct asymptotic description of the dynamics in nonlinear winner-take-all models near the bifurcation to winner-take-all behavior. This observation has two consequences. Firstly, in as far as nonlinear winner-take-all models can successfully reproduce some qualitative features of the neuronal activity in brain areas implicated in the decision making process for two-choice decision making [28], i.e. LIP, the nonlinear diffusion equation also provides an approximate description of this activity. Secondly, if an actual nonlinear winner-take-all process is at work in the brain during such tasks, then this process will behave as an approximately one-dimensional diffusion process in the vicinity of the bifurcation to winner-take-all behavior. This process is described by the nonlinear diffusion equation Equation 8. Note also that the effective reduction in dimension of the dynamics in nonlinear systems in general only occurs at bifurcations. Thus nonlinear normal forms for bifurcations such as Equation 8 represent the only proper one-dimensional reduction of such a system.
As in the linear diffusion equations, bias in external inputs in the nonlinear diffusion equation appears to leading order as a constant drift term. In contrast, while reductions of linear connectionist models to the linear diffusion equation lead to a linear (Ornstein-Uhlenbeck) term proportional to the difference between intrinsic ‘leak’ and the effective cross inhibition, this is not the case in nonlinear systems. Rather, this term reflects the linear growth rate of the spontaneous state which, given that the input is the bifurcation parameter, is simply proportional to the distance of the common external input from the critical value at the bifurcation. Thus this term varies with modulations of the external input, unlike in the linear case. Finally, the cubic nonlinearity, which is the lowest order nonlinearity consistent with the reflection symmetry of the original system, leads to an inverted-U potential. This drives the activity to infinity in finite time, reflecting the escape from the spontaneous state to the ‘decision’ state. As illustrated in Figure 1, this renders the measurement of reaction-times and performance insensitive to the exact placement of a threshold as long as it is high enough. Setting relatively high thresholds therefore effectively eliminates one free parameter from the model, namely the threshold placement. Nonetheless, one could set low thresholds in the nonlinear system, i.e. very close to the spontaneous state [30]. It has been hypothesized that the threshold for detection of a decision in the brain may be set by downstream areas including superior colliculus [30] or the basal ganglia [46]
As it turns out, Equation 8 can account for behavioral data for the random moving dot task in monkeys and humans, c.f. Figures 3 and 4. As such Equation 8 seems to provide a correct description of both the neuronal activity and the behavior in this task, thereby linking the two. This, however, in no way contradicts the success of connectionist and linear diffusion models in fitting behavioral data. Indeed, a comparison of the nonlinear diffusion equation and the linear one, Equation 1, shows approximately equally good fits for correct reaction-times and performance for the data in Figures 3 and 4, see supporting material (Text S1). On the other hand error reaction-times in Figure 3, which are longer than correct ones, cannot be fit by the linear diffusion model unless variability in the initial condition and drift term across trials is introduced [9]. They are, however, correctly captured by the nonlinear diffusion equation. We note, furthermore, that several groups have derived reduced models for two-choice decision making. Wong and Wang performed a heuristic reduction of a spiking network model to a system of two coupled rate equations [29], and showed that it gave similar qualitative behavior. As a canonical model, Equation 8 qualitatively captures the dynamics of both the network and the rate models, also see fit in supporting material (Text S1). We note, however, that far from the bifurcation the full dimensionality of the system being studied will come into play and the dynamics will not be captured by Equation 8. Much of the phenomenology in [29] appears to occur in this regime. Once this is the case, the dynamics may depend crucially on the details and dimensionality of the system and, if so, cannot be generalized. Wong et al. have recently used their reduced model to explain the experimentally observed violation of time-shift invariance in the behavior of monkeys doing the random moving dot task [47], lending further support for the nonlinear, attractor network framework for LIP activity [31]. They also note that the inclusion of target inputs, which more faithfully reproduces the experimental paradigm, ‘renders the model behavior closer to a one-dimensional model in the decision process’ [31]. Interestingly, the presence of the unstable cubic term in the 1D nonlinear diffusion equation Equation 8 should lead to the experimentally observed violation of time-shift invariance for which perturbations arriving later in time have a lesser effect due to the nonlinear acceleration away from the spontaneous state. This remains to be tested quantitatively. Usher and McClelland derived a one-dimensional diffusion equation equivalent to an Ornstein-Uhlenbeck process from a neurobiologically motivated system of two coupled, threshold-linear equations [12]. This and other similar systems of linear equations were studied by Bogacz, Brown and collaborators [13],[14]. The linearity of the system in these studies allowed for an in-depth analytical characterization of the dynamics. Indeed, it has been argued that neurobiologically motivated models might, within certain parameter regimes, be reducible to an equivalent linear diffusion equation [14]. However, as we have shown here, if the underlying winner-take-all system exhibits any generic nonlinearities, as seems to be the case in neural systems, the correct dynamics are given by Equation 8.
Soltani and Wang [48] and Fusi et al. [49] have both investigated how synaptic plasticity might shape the response in winner-take-all decision making circuits. Soltani and Wang introduced a reward-dependent stochastic Hebbian rule for updated synaptic strengths which successfully reproduces the so-called ‘matching behaviorÂ’ while Fusi et al. have presented a model of flexible sensorimotor mapping in which reward-dependent synaptic plasticity shapes the output of a winner-take-all decision making circuit. In both cases, the performance depends on the difference in the fraction of potentiated synapses between the two populations Δc, i.e. the symmetry breaking occurs due to plastic changes in synaptic strength. In the context of Equation 8 this would lead to an additional term which is functionally equivalent to the symmetry-breaking term proportional to the difference in inputs. The effect of synaptic plasticity in two-choice decision making could therefore be studied by means of Equation 8 coupled with an appropriate learning rule.
The reduction to Equation 8 is strictly valid only in the immediate vicinity of the bifurcation. For this reason it might be argued that the current scenario is tantamount to fine-tuning and may not be biologically relevant. Three facts indicate this is not the case. (I) As we have shown here Equation 8 can be rigorously derived from model systems and can provide a quantitative match even away from the bifurcation. (II) Equation 8 can be fit to behavioral data, previously published model networks [28] and models in regimes far from the bifurcation where a quantitative match is no longer found. It thus provides a correct qualitative description of the dynamics. Furthermore these fits are made by varying physiologically meaningful parameters in ways that are either consistent with experimental findings or which lead to experimentally testable predictions. (III) Lastly, a large literature exists showing that human behavior in 2-choice decision making is well-described by one-dimensional sequential sampling models. A deep question is how such low-dimensional dynamics might arise from high-dimensional neuronal dynamics. We believe the most parsimonious explanation is that the neuronal circuits involved operate near the low dimensional manifold which arises naturally within a certain parameter range, i.e. near the bifurcation.