Neural Reinforcement Learning Lab (NeuRLab)
Introduction
Living in an uncertain environment, we desire to pursue good things and to avoid bad things. We are interested in how the brain recognizes different situations and learns to make better decisions. Related questions are: How does the brain represent reward or punishment? How does the brain remember something good and pursue it? How does the brain choose one action out of multiple options? What makes one animal more intelligent than another animal? What can we learn about how the brain works from artificial intelligence?
Reinforcement learning (RL) theory provides theoretical and computational frameworks to these problems. Interestingly, it has been shown that dopamine activity in the brain resembles the teaching signal in one of reinforcement learning theories, temporal difference (TD) learning. However, the detailed neural mechanisms of adaptive behaviors remain elusive. We perform experiments using animals and analyze data using computational models derived from artificial intelligence (AI) to understand the biological mechanisms of reinforcement learning.
Selected Recent Publications
1. Kim HR*, Malik AM*, Mikhael JG, Bech P, Tsutsui-Kimura I, Sun F, Zhang Y, Li Y, Watabe-Uchida M, Gershman SJ, Uchida N (2020) A unified framework for dopamine signals across timescales. Cell (lead author)
2. Kim HR, Angelaki DE, DeAngelis GC (2017) Gain Modulation as a Mechanism for Coding Depth from Motion Parallax in Macaque Area MT. Journal of Neuroscience 37 (34), 8180-8197
3. Kim HR, Angelaki DE, DeAngelis GC (2015) A novel role for visual perspective cues in the neural computation of depth. Nature Neuroscience 18(1), 129-137.
Computational Learning & Memory Neurosciece Lab
(CLMN Lab)
Research interest
· Computational modeling of human movement control, learning, and memory
· Neuroscientific approach to modulating human learning & memory with non-invasive brain stimulation
· Brain inspired artificial intelligence (Reverse engineering the brain to understand learning and memory)
· Cognitive and neural mechanisms underlying decision making in the framework of reinforcement learning
Selected Recent Publications
1. Choi Y, Shin EY, Kim S*. Spatiotemporal dissociation of fMRI activity in the caudate nucleus underlies human de novo motor skill learning. Proceedings of National Academy of Sciences U. S. A., Vol. 117, Issue 38, 2020
2. Kim S, Nilakantan AS, Hermiller MS, Palumbo R, VanHaerents SA, Voss JL*. Selective and coherent activity increases due to stimulation indicate functional distinctions between episodic memory networks. Science Advances, Vol. 4, Issue 8, 2018
3. Kim S, Ogawa K, Lv J, Schweighofer N*, Imamizu H. Neural substrates related to motor memory with multiple time scales in sensorimotor adaptation. PLoS Biology, Vol. 13, Issue 12, 2015
4. Kim S, Callier T, Tabot GA, Gaunt RA, Tenore FV, Bensmaia SJ*. Behavioral assessment of sensitivity to intracortical microstimulation of primate somatosensory cortex. Proceedings of National Academy of Sciences U. S. A., Vol. 112, Issue 49, 2015
Detail
CAMIN lab (Computational Analysis for Multimodal Integrative Neuroimaging)
Introduction
We analyze multimodal brain MR imaging data using advanced connectomics and machine learning techniques to establish novel frameworks assessing multiscale brain organization. In particular, we aim to assess large-scale brain organization as well as structure-function coupling during typical and atypical development and young adults. Leveraging histology and imaging-genetics approaches, we provide biological underpinnings to the imaging findings. Methodologically, we study data mining, multimodal integration, and classification/prediction for big data.
Selected Recent Publications
B.-y. Park et. al., “Topographic divergence of atypical cortical asymmetry and atrophy patterns in temporal lobe epilepsy”, Brain, 2021.
B.-y. Park, H. Park, F. Morys, M. Kim, K. Byeon, H. Lee, S.-H. Kim, S. Valk, A. Dagher, B. C. Bernhardt, “Inter-individual body mass variations relate to fractionated functional brain hierarchies”, Communications Biology, 4:735, 2021.
B.-y. Park, S.-J. Hong, S. Valk, C. Paquola, O. Benkarim, R. A. I. Bethlehem, A. Di Martino, M. Milham, A. Gozzi, B. T. T. Yeo, J. Smallwood, and B. C. Bernhardt, “Differences in subcortico-cortical interactions identified from connectome and microcircuit models in autism”, Nature Communications, 12:2225, 2021.
B.-y. Park, R. A. I. Bethlehem, C. Paquola, S. Larivière, R. Rodríguez-Cruces, R. Vos de Wael, E. T. Bullmore, B. C. Bernhardt, “An expanding manifold in transmodal regions characterizes adolescent reconfiguration of structural connectome organization”, eLife, 10:e64694, 2021.
B.-y. Park, R. Vos de Wael, C. Paquola, S. Larivière, O. Benkarim, J. Royer, S. Tavakol, R. Rodríguez-Cruces, Q. Li, S. L. Valk, D. S. Margulies, B. Mišić, D. Bzdok, J. Smallwood, and B. C. Bernhardt, “Signal diffusion along connectome gradients and inter-hub routing differentially contribute to dynamic human brain function”, NeuroImage, 224:117429, 2021.
I am a brand-new Assistant Professor at Sungkyunkwan University (SKKU) in South Korea, studying how the brain generates complex and intelligent behaviors. I am affiliated with the Institute for Basic Science (IBS) - Center for Neuroscience Imaging Research and the Department of Biomedical Engineering.
Previously, I was a postdoctoral associate/research scientist at MIT, working with Mehrdad Jazayeri, and at Yale, working with Daeyeol Lee. I obtained my Ph.D. in neuroscience from Seoul National University, mentored by Sang-hun Lee, and my master’s/undergrad from KAIST, mentored by Jaeseung Jeong.
My area of research is cognitive and systems neuroscience. I have been investigating how the brain measures and processes time using multiple approaches: behavioral experiments, computational modeling (e.g., Bayesian theory), human neuroimaging (EEG/fMRI), and electrophysiology in non-human primates. In my new lab, I will combine these techniques to study how the prefrontal and posterior parietal cortices process information about magnitude (time, number, and space).
In my spare time (if I have any!), I enjoy spending time with my daughters outdoors (camping,skiing) and would love to adopt a dog.
Recent Updates
February 2023: I start my own lab at Sungkyunkwan University (SKKU), Department of Biomedical Engineering & Institute for Basic Science - Center for Neuroscience Imaging Research
January 2023: Manuel & Nico’s work titled “Parametric control of flexible timing through low-dimensional neural manifolds”, which I am a part of, is published in Neuron
November 2022: Reza & Andrew’s work titled “A large-scale neural network training framework for generalized estimation of single-trial population dynamics”, which I am a part of, is published in Nature Methods
October 2022: Jason’s work that I mentored is accepted as an oral presentation in NeurIPS workshop
October 2021: My review paper with Devika titled “Neural implementations of Bayesian inference” is published in Current Opinion in Neurobiology
June 2021: My work titled “Validating model-based Bayesian integration using prior–cost metamers” is published in PNAS