Compressed Sensing in Neuroscience

Wired has a nice lay-person write-up of the rapidly developing field of compressed sensing. This is a technique that allows accurate reconstructions of highly undersampled sparse datasets. This field really took off in 2004 when Emmanuel J. Candès discovered that a tomography phantom image could be reconstructed exactly even with data deemed insufficient by the Nyquist-Shannon criterion. It is probably the hottest topic in imaging theory today.

Modified Shepp-Logan phantom with enhanced contrast for visual perception.

According to this review, Compressed Sensing MRI, its successful application requires three conditions to be met :

• Transform Sparsity: The desired image must have a sparse representation in a known transform domain (i.e., it must be compressible by transform coding),
• Incoherence of Undersampling Artifacts: The aliasing artifacts in a linear reconstruction caused by k-space undersampling must be incoherent (noise-like) in the sparsifying transform domain.
• Nonlinear Reconstruction: The image must be reconstructed by a non-linear method which enforces both sparsity of the image representation and consistency of the reconstruction with the acquired samples.

These conditions are well met by MRI imaging.  This decoding technique dramatically shortens the required sampling times in an MRI magnet, which reduces the impact of motion artifacts, the bane of high-resolution MRI.

Unfortunately, I don’t think it is very applicable to situations where signal/noise of the underlying source is poor, like counting action potentials in shot-noise limited in vivo calcium imaging. But it’s use is spreading into other related problems, such as mapping the functional connectivity of neural circuitry.  Tao Hu and Mitya Chklovvskii apply the compressed sensing algorithms in Reconstruction of Sparse Circuits Using Multi-neuronal Excitation (RESCUME) from the latest Advances in Neural Information Processing Systems journal. They measure a post-synaptic neuron’s voltage while stimulating sequentially random subsets of multiple potentially pre-synaptic neurons. The sparseness of connectivity allows them to map connectivity much faster than by a sequential method.

UPDATE: If you want to get a better sense of the breadth and depth of the applications of compressed sensing, check out Igor Carron’s comprehensive site, Compressive Sensing : The Big Picture.

Updated: fMRI Based Visual Stimulus Reconstruction

A simple view of what the brain does is acquire input, process it, then produce output. One strategy for understanding what processing takes place is to record the patterns of brain activity while showing many patterns of input, then see if you can use the information gained to predict a novel input, given the pattern of brain activity. The canonical example of this approach is visual input reconstruction based on recorded spike trains in the visual system of the blowfly.

The blowfly is a relatively simple system (though quite efficient) with a tiny brain. Could a similar approach work in humans?  Although we can’t drop electrodes into the visual cortex (usually), we can put people in fMRI scanners to visualize the pattern of blood oxygenation, which is correlated with neural activity.

In Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders, Miyawaki et al demonstrate visual input prediction using fMRI responses. Using 3mm³ voxels, the group measured the activity level across early visual cortex (V1-V4) for numerous 10×10 binary patterns of visual stimuli. They looked at correlations in 1×1, 1×2, 2×1 and 2×2 bins of voxel activity to hundreds of visual test patterns. The activity represented local image elements. Then they displayed novel visual input and used a linear combination of the local image element responses to predict the visual input from the brain activity alone. It is noteworthy that they only required several hundred training images before visual input prediction was possible.

Predicted visual input from fMRI activity in V1 and V2

Note that a rentinotopic map, where the relative spatial position of visual input is reflected in the activity across the visual cortex, is not strictly required for this technique to work. What is required is that the response of each local element is consistent across similar patterns of input in the element’s receptive field. Furthermore, the spatial scale of pattern representation in early processing regions of human visual cortex is broad enough to be picked up by the fMRI scanner.

It would be interesting to see how much higher visual resolution could be predicted with an fMRI approach. Could this approach be adapted to predict input from the responses of cells with more complex receptive fields in higher cortical areas? Or, are those cells too intermingled with neighbors with vastly different response properties to be separable by fMRI?  Higher areas are vital for our own brains to rapidly perceive the contours of complex images. I’d also like to see how well non-contiguous images are predicted.

Cellular resolution calcium imaging with bulk loaded dyes has been used to map fine-grained detail of receptive fields in lower animals visual and somatosensory cortex. Is input prediction possible from these recordings? Is the input training set too limited? Could more complex input be perceived using a fewer number of complex cells from higher visual areas (V2 and above)?

Preview : fMRI Based Visual Stimulus Reconstruction

I’m going to try a new format for getting brand-new articles up on the site quickly. Often, I want to post something but don’t have the time to read the paper carefully and then create a quality writeup. This provides a lot of posting inhibition. Rather than just sit on the paper, I’ll now post the paper, the link, and the abstract.  Then, if and when I find the time, I’ll post and update and go more in depth.  Here’s the first preview!

Please see the updated post on this paper.

Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders

Yoichi Miyawaki^1,2,6,Hajime Uchida^2,3,6,Okito Yamashita²,Masa-aki Sato²,Yusuke Morito^4,5,Hiroki C. Tanabe^4,5,Norihiro Sadato^4,5andYukiyasu Kamitani^2,3,,

Perceptual experience consists of an enormous number of possible states. Previous fMRI studies have predicted a perceptual state by classifying brain activity into prespecified categories. Constraint-free visual image reconstruction is more challenging, as it is impractical to specify brain activity for all possible images. In this study, we reconstructed visual images by combining local image bases of multiple scales, whose contrasts were independently decoded from fMRI activity by automatically selecting relevant voxels and exploiting their correlated patterns. Binary-contrast, 10 10-patch images (2¹⁰⁰ possible states) were accurately reconstructed without any image prior on a single trial or volume basis by measuring brain activity only for several hundred random images. Reconstruction was also used to identify the presented image among millions of candidates. The results suggest that our approach provides an effective means to read out complex perceptual states from brain activity while discovering information representation in multivoxel patterns.