Research

In the ever-changing and complex world we live in, we are constantly bombarded with visual information, which we process and integrate to form our perception, memory, and understanding of the environment. Although the input information is only two-dimensional and processed in the retina, our ability to perceive and comprehend the world remains steadfast.

My research aims to

(1) shed light on the behavioral and neural mechanisms of visual processing by combining human psychophysics, eye-tracking, neuroimaging techniques such as fMRI and EEG, computational methods including linear regression, MVPA, RSA, IEM, and pRF, as well as deep learning models.

(2) map representations between human brains and artificial neural networks via representational alignment between human brains and models and image-to-brain encoding models.

Below are the two main topics I am interested in with several subtopics and projects I am working on:

Neural and behavioral meachnisms of visual perception

Object-location binding across saccades

Increasing studies have found interactions between object identity adn spatial location. Even when location is irrelevatn to the task, location can be bound to object representations. However, what's the reference framework of object-location binding? Is it retinotopic (gaze-centered) or spatiotopic (world-centered)?
I am investigating the dynamic framework of object-location binding across eye movements, using the Spatial Congruency Bias paradigm. This inverstigation is conducted from three perspectives:
(1) Observer's eye-movements: how dynamic saccade context (using multiple saccades) influences object-location binding.
(2) External environment: how the landmark influences object-location binding.
(3) Object's movements: how object-location binding happens of a moving object.

Depth perception and 3D integration

Visual input is initially captured in 2D on our retinas, however, we can know the location of an object in 3D spatial coordinates quickly. Our brains seamlessly integrate two-dimensional representations with various depth cues to form a three-dimensional perception. To better understand human depth perception, I am conducting research in two ways:
(1) Object depth and size representations in natural images: to explore the brain representations of preceived real-world depth, real-world size and relative size of the object in an image using EEG and computational approaches. (2) Spatiotemporal representations of 3D perception: to unfold the spatiotemporal neural mechanisms of depth perception and 3D integration using a fMRI-EEG fusion computational framework.

Generally spatial representation

As we move our eyes around the world, we are able to maintain a stable visual perception despite changes in the visual input. However, previous studies have found that our visual system, from primary visual cortex to higher level visual regions, represents object locations in natively reitnotopic (gaze-centered) rather than spatiotopic (gaze-independent) coordinates. This raised question of how can our brain form a stable spatial representation of objects.
Is spatiotopic information represented elsewhere in the brain, or can it be elicited in visual areas by some other factors (e.g., dynamic saccades, landmarks, etc.)? I am conducting fMRI studies to address these.

Mapping representations between human brains and artificial neural networks

More brain-like artificial neural networks via human brain alignment

Although current artificial neural networks have very high performance in many tasks, they are still very different from the human brain. One way to achieve more brain-like models is to let models learn the internal representations of human brains. I am focusing on applying neural activations of humans brain (fMRI and EEG) to align the representation of artificial neural networks to implement really human brain-like models.

Brain encoding models

The most intuitive and direct way to understand the relationship between visual input from the world and neural computing in our brains is to get through from visual inputs to neural latent space to brain activity and from brain activity to visual latent space to visual inputs.
In my research, I aim to build a image-to-brain encoding (signal generation) model, Img2EEG, to gain deeper insights into visual perception by studying the mapping from visual iputs to neural representations.

Inter-individual neural converters

One of the challenges in cognitive and computational neuroscience is that models trained on one subject do not generalize to other subjects due to individual differences. An ideal solution to this problem is to develop an individual-to-individual neural converter that can generate real neural signals of one subject from those of another one. This would enable us to overcome the challenge of individual differences in cognitive and computational models.
My research focuses on proposing novel inter-individual EEG converters, EEG2EEG, which can realize a flexible and high-performance mapping from invidual to indivual. This work has the potential to provide valuable insights for both neural engineering and cognitive neuroscience.

Reverser engineering to interpret neural mechanisms

Recently, artificial neural networks (ANNs) have shown remarkable progress in achieving human-level performance in object and face recognition tasks. In my research, I am exploring the use of hypothesis-driven reverse engineering to provide a novel approach to understanding the neural mechanisms underlying object and face perception. The approach involves manipulating ANN activations based on different hypotheses and comparing the modified representations to human brain representations. This allows us to make inferences about the neural mechanisms underlying human behaviors and gain deeper insights into the functioning of object and face recognition systems.