Critical tests of neurocognitive relationships

AgencyNational Science Foundation
PanelCognitive Neuroscience
LocationUniversity of California, Irvine
Start dateSeptember 2019
End dateAugust 2023
Budget$ 674,807.00
Agency code1850849
LeadJoachim Vandekerckhove
OtherRamesh Srinivasan
Project text pn-1850849.pdf


The link between patterns of activity in the brain and human actions has been studied for a long time. Careful study of these patterns has led to great scientific and technological progress. Brain-computer interfaces and brain-controlled prosthetic limbs are two examples. These technologies depend strongly on the power to tie brain signals to (intended) actions. In this project, we use state-of-the-art methods in cognitive science to develop a precise mathematical model of the link between brain activity and human behavior. Our theory now needs to be tested with new experimental data in adverse conditions. The challenges that will test whether this model can predict the brain-behavior link in new experiments, new behaviors, and new measures of brain activity. A strong model of brain and behavior will improve our knowledge of the brain and help future research and technological development. It can also improve the accuracy of science or technology that uses brain signals, including brain-computer interfaces. The project will benefit researchers outside our lab in other ways. We will document analyses and experiments as part of a series of video lectures. We will also freely share our data, code, and methods online. This will help other researchers to verify our findings and to educate members of the general public who have an interest in cognitive neuroscience. Finally, the project will involve junior scientists who will receive training and start a career in neuroscience or cognitive science.

The primary advantage of joint modeling is that it improves researchers' ability to estimate parameters of neural, cognitive, or behavioral models by using constraints imposed by one or more additional data modes. This has already allowed us to construct genuine neurocognitive models that are jointly informed by behavioral and neural data. We have developed a multimodal sequential accumulation model that makes predictions about the combination of reaction time, accuracy, and neuroelectric data, and that allows for conclusions not possible from either type of data individually. We will now test the generalizability of this model to other contexts. After first training a model on a relatively small data set, we will extrapolate its predictions to (a) new tasks by the same participants; (b) new participants in the same task; (c) new paradigms (i.e., tasks with new response modalities) by the same participants; and (d) new tasks by new participants. For a rigorous test of the linkage between the neural and behavioral data, the experiments will involve manipulations that selectively affect the cognitive components (visual preprocessing, motor preparation time, evidence accumulation) as well as corresponding human behavior (reaction times, accuracies, and choice behavior) and electrophysiological signals (ERP latencies and magnitudes and EMG muscle preparation signatures).


Nunez, M. D., Fernandez, K., Srinivasan, R., & Vandekerckhove, J. (in press). A tutorial on fitting joint models of M/EEG and behavior to understand cognition. Behavior Research Methods.
Villarreal, J., Chávez De la Peña, A. F., Mistry, P., Menon, V. E., Vandekerckhove, J., & Lee, M. D. (2024). Bayesian graphical modeling with the circular drift diffusion model. Computational Brain & Behavior, 7, 181-194.
Chwiesko, C., Janecek, J., Doering, S., Hollearn, M., McMillan, L., Vandekerckhove, J., Lee, M. D., Ratcliff, R., & Yassa, M. (2023). Parsing memory and non-memory contributions to age-related declines in mnemonic discrimination performance: A hierarchical Bayesian diffusion decision modeling approach. Learning & Memory, 30, 296-309.
Sun, J. Q., Vo, K., Lui, K. K., Nunez, M. D., Vandekerckhove, J., & Srinivasan, R. (2022). Decision SincNet: Neurocognitive models of decision making that predict cognitive processes from neural signals. Proceedings of the International Joint Conference on Neural Networks.
Vo, K., Vishwanath, M., Srinivasan, R., Dutt, N., & Cao, H. (2022). Composing graphical models with generative adversarial networks for EEG signal modeling. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 1231-1235.