gistfile1

Fisher and shannon information in finite neural populations.

Stuart Yarrow, Edward Challis, and Peggy Seriès.

The precision of the neural code is commonly investigated using two
families of statistical measures: Shannon mutual information and
derived quantities when investigating very small populations of
neurons and Fisher information when studying large populations. These
statistical tools are no longer the preserve of theorists and are
being applied by experimental research groups in the analysis of
empirical data. Although the relationship between
information-theoretic and Fisher-based measures in the limit of
infinite populations is relatively well understood, how these measures
compare in finite-size populations has not yet been systematically
explored. We aim to close this gap. We are particularly interested in
understanding which stimuli are best encoded by a given neuron within
a population and how this depends on the chosen measure. We use a
novel Monte Carlo approach to compute a stimulus-specific
decomposition of the mutual information (the SSI) for populations of
up to 256 neurons and show that Fisher information can be used to
accurately estimate both mutual information and SSI for populations of
the order of 100 neurons, even in the presence of biologically
realistic variability, noise correlations, and experimentally relevant
integration times. According to both measures, the stimuli that are
best encoded are those falling at the flanks of the neuron's tuning
curve. In populations of fewer than around 50 neurons, however, Fisher
information can be misleading.

Neural Comput, 2012 vol. 24 (7) pp. 1740-1780

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=22428594&retmode=ref&cmd=prlinks

Comment: This paper mainly highlights a different method of computing
stimulus relevant information, using the fisher information, which is
related to the minimum variance achievable for any estimator. To
compute it, one has to postulate a model for estimating the stimulus
from the data, though, and then compute the covariance matrix of that
estimate across all neurons. The reciprocal of the determinant of that
matrix is then an estimate of the fisher information

#####################################################################################

Sparse low-order interaction network underlies a highly correlated and
learnable neural population code.

Elad Ganmor, Ronen Segev, and Elad Schneidman.

Information is carried in the brain by the joint activity patterns of
large groups of neurons. Understanding the structure and function of
population neural codes is challenging because of the exponential
number of possible activity patterns and dependencies among neurons.
We report here that for groups of ~100 retinal neurons responding to
natural stimuli, pairwise-based models, which were highly accurate for
small networks, are no longer sufficient. We show that because of the
sparse nature of the neural code, the higher-order interactions can be
easily learned using a novel model and that a very sparse low-order
interaction network underlies the code of large populations of
neurons. Additionally, we show that the interaction network is
organized in a hierarchical and modular manner, which hints at
scalability. Our results suggest that learnability may be a key
feature of the neural code.

Proceedings of the National Academy of Sciences, 2011 vol. 108 (23)
pp. 9679-9684

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=21602497&retmode=ref&cmd=prlinks

Comment: They use a variant of the maximum entropy model that does not
directly compute all the possible pairwise expectation values, but
rather uses Monte Carlo methods to minimize minimize the distance
between the true and the empirical distribution.

#####################################################################################

Correlations, feature-binding and population coding in primary visual cortex.

Huw D R Golledge, Stefano Panzeri, Fashan Zheng, Gianni Pola, Jack W
Scannell, Dimitrios V Giannikopoulos, Roger J Mason, Martin J Tovée,
and Malcolm P Young.

To test the hypothesis that correlated neuronal activity serves as the
neuronal code for visual feature binding, we applied information
theory techniques to multiunit activity recorded from pairs of V1
recording sites in anaesthetised cats while presenting either single
or separate bar stimuli. We quantified the roles of firing rates of
individual channels and of cross-correlations between recording sites
in encoding of visual information. Between 89 and 96% of the
information was carried by firing rates; correlations contributed
4-11% extra information. The distribution across the population of
either correlation strength or correlation information did not co-vary
systematically with changes in perception predicted by Gestalt
psychology. These results suggest that firing rates, rather than
correlations, are the main element of the population code for feature
binding in primary visual cortex.

Neuroreport, 2003 vol. 14 (7) pp. 1045-1050

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=12802200&retmode=ref&cmd=prlinks

Comment: This is a straight forward computation of information encoded
by pairs of neurons in primary visual cortex. Perhaps more relevant
for the shufflesync paper.

#####################################################################################

Information processing with population codes.

A POUGET, P Dayan, and R Zemel.

Information is encoded in the brain by populations or clusters of
cells, rather than by single cells. This encoding strategy is known as
population coding. Here we review the standard use of population codes
for encoding and decoding information, and consider how population
codes can be used to support neural computations such as noise removal
and nonlinear mapping. More radical ideas about how population codes
may directly represent information about stimulus uncertainty are also
discussed.

Nature Reviews Neuroscience, 2000 vol. 1 (2) pp. 125-132

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=11252775&retmode=ref&cmd=prlinks
Comment: A nice review paper on the use of information theory for
population encoding and decoding.

#####################################################################################

Neuronal population coding of continuous and discrete quantity in the
primate posterior parietal cortex.

Oana Tudusciuc and Andreas Nieder.

Quantitative knowledge guides vital decisions in the life of animals
and humans alike. The posterior parietal cortex in primates has been
implicated in representing abstract quantity, both continuous (extent)
and discrete (number of items), supporting the idea of a putative
generalized magnitude system in this brain area. Whether or not single
neurons encode different types of quantity, or how quantitative
information is represented in the neuronal responses, however, is
unknown. We show that length and numerosity are encoded by
functionally overlapping groups of parietal neurons. Using a
statistical classifier, we found that the activity of populations of
quantity-selective neurons contained accurate information about
continuous and discrete quantity. Unexpectedly, even neurons that were
nonselective according to classical spike-count measures conveyed
robust categorical information that predicted the monkeys' quantity
judgments. Thus, different information-carrying processes of partly
intermingled neuronal networks in the parietal lobe seem to encode
various forms of abstract quantity.

Proc Natl Acad Sci USA, 2007 vol. 104 (36) pp. 14513-14518

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=17724337&retmode=ref&cmd=prlinks
Comment: Uses a statistical decoder to quantify the amount of
information carried by populations of neurons

#####################################################################################

Dynamic population coding of category information in inferior temporal
and prefrontal cortex.

Ethan M Meyers, David J Freedman, Gabriel Kreiman, Earl K Miller, and
Tomaso Poggio.

Most electrophysiology studies analyze the activity of each neuron
separately. While such studies have given much insight into properties
of the visual system, they have also potentially overlooked important
aspects of information coded in changing patterns of activity that are
distributed over larger populations of neurons. In this work, we apply
a population decoding method to better estimate what information is
available in neuronal ensembles and how this information is coded in
dynamic patterns of neural activity in data recorded from inferior
temporal cortex (ITC) and prefrontal cortex (PFC) as macaque monkeys
engaged in a delayed match-to-category task. Analyses of activity
patterns in ITC and PFC revealed that both areas contain "abstract"
category information (i.e., category information that is not directly
correlated with properties of the stimuli); however, in general, PFC
has more task-relevant information, and ITC has more detailed visual
information. Analyses examining how information coded in these areas
show that almost all category information is available in a small
fraction of the neurons in the population. Most remarkably, our
results also show that category information is coded by a
nonstationary pattern of activity that changes over the course of a
trial with individual neurons containing information on much shorter
time scales than the population as a whole.

Journal of Neurophysiology, 2008 vol. 100 (3) pp. 1407-1419

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=18562555&retmode=ref&cmd=prlinks
Comment: Uses a linear classifier to find a lower bound on the amount
of knowledge encoded by populations of neurons.

#####################################################################################

Decoding the activity of neuronal populations in macaque primary visual cortex.

Arnulf B A Graf, Adam Kohn, Mehrdad Jazayeri, and J Anthony Movshon.

Visual function depends on the accuracy of signals carried by visual
cortical neurons. Combining information across neurons should improve
this accuracy because single neuron activity is variable. We examined
the reliability of information inferred
from populations of simultaneously recorded neurons in macaque primary
visual cortex. We considered a decoding framework that computes the
likelihood of visual stimuli from a pattern of population activity by
linearly combining neuronal responses
and tested this framework for orientation estimation and
discrimination. We derived a simple parametric decoder assuming
neuronal independence and a more sophisticated empirical decoder that
learned the structure of the measured neuronal response distributions,
including their correlated variability. The empirical decoder used the
structure of these response distributions to perform better than its
parametric variant, indicating that their structure contains critical
information for sensory decoding.
These results show how neuronal responses can best be used to inform
perceptual decision-making.

Nat. Neurosci., 2011 vol. 14 (2) pp. 239-245

http://libproxy1.nus.edu.sg/login?url=http://www.nature.com/doifinder/10.1038/nn.2733

#####################################################################################

Redundancy in the Population Code of the Retina.

Jason L Puchalla, Elad Schneidman, Robert A Harris, and Michael J Berry.

We have explored the manner in which the population of retinal
ganglion cells collectively represent the vi- sual world. Ganglion
cells in the salamander were re- corded simultaneously with a
multielectrode array during stimulation with both artificial and
natural vi- sual stimuli, and the mutual information that single cells
and pairs of cells conveyed about the stimulus was estimated. We found
significant redundancy be- tween cells spaced as far as 500 􏰣m apart.
When we used standard methods for defining functional types, only
ON-type and OFF-type cells emerged as truly independent information
channels. Although the av- erage redundancy between nearby cell pairs
was moderate, each ganglion cell shared information with many
neighbors, so that visual information was repre- sented 10-fold within
the ganglion cell population. This high degree of retinal redundancy
suggests that design principles beyond coding efficiency may be
important at the population level.

Neuron, 2005 vol. 46 (3) pp. 493-504

http://libproxy1.nus.edu.sg/login?url=http://linkinghub.elsevier.com/retrieve/pii/S0896627305003119
Comment: This paper only does pair-wise information to look at redundancy.

#####################################################################################

Excess synchrony in motor cortical neurons provides redundant
direction information with that from coarse temporal measures.

M W Oram, N G Hatsopoulos, B J Richmond, and J P Donoghue.

Previous studies have shown that measures of fine temporal
correlation, such as synchronous spikes, across responses of motor
cortical neurons carries more directional information than that
predicted from statistically independent neurons. It is also known,
however, that the coarse temporal measures of responses, such as spike
count, are not independent. We therefore examined whether the
information carried by coincident firing was related to that of
coarsely defined spike counts and their correlation. Synchronous
spikes were counted in the responses from 94 pairs of simultaneously
recorded neurons in primary motor cortex (MI) while monkeys performed
arm movement tasks. Direct measurement of the movement-related
information indicated that the coincident spikes (1- to 5-ms
precision) carry approximately 10% of the information carried by a
code of the two spike counts. Inclusion of the numbers of synchronous
spikes did not add information to that available from the spike counts
and their coarse temporal correlation. To assess the significance of
the numbers of coincident spikes, we extended the stochastic spike
count matched (SCM) model to include correlations between spike counts
of the individual neural responses and slow temporal dependencies
within neural responses (approximately 30 Hz bandwidth). The extended
SCM model underestimated the numbers of synchronous spikes. Therefore
as with previous studies, we found that there were more synchronous
spikes in the neural data than could be accounted for by this
stochastic model. However, the SCM model accounts for most (R(2) =
0.93 +/- 0.05, mean +/- SE) of the differences in the observed number
of synchronous spikes to different directions of arm movement,
indicating that synchronous spiking is directly related to spike
counts and their broad correlation. Further, this model supports the
information theoretic analysis that the synchronous spikes do not
provide directional information beyond that available from the firing
rates of the same pool of directionally tuned MI neurons. These
results show that detection of precisely timed spike patterns above
chance levels does not imply that those spike patterns carry
information unavailable from coarser population codes but leaves open
the possibility that excess synchrony carries other forms of
information or serves other roles in cortical information processing
not studied here.

Journal of Neurophysiology, 2001 vol. 86 (4) pp. 1700-1716

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=11600633&retmode=ref&cmd=prlinks

#####################################################################################

Population coding of stimulus location in rat somatosensory cortex.

R S Petersen, S Panzeri, and M E Diamond.

This study explores the nature of population coding in sensory cortex
by applying information theoretic analyses to neuron pairs recorded
simultaneously from rat barrel cortex. We quantified the roles of
individual spikes and spike patterns in encoding whisker stimulus
location. 82%-85% of the total information was contained in the timing
of individual spikes: first spike time was particularly crucial. Spike
patterns within neurons accounted for the remaining 15%-18%. Neuron
pairs located in the same barrel column coded redundantly, whereas
pairs in neighboring barrel columns coded independently. The barrel
cortical population code for stimulus location appears to be the time
of single neurons' first poststimulus spikes-a fast, robust coding
mechanism that does not rely on "synergy" in crossneuronal spike
patterns.

Neuron, 2001 vol. 32 (3) pp. 503-514

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=11709160&retmode=ref&cmd=prlinks

#####################################################################################

The structure of large-scale synchronized firing in primate retina.

Jonathon Shlens, Greg D Field, Jeffrey L Gauthier, Martin Greschner,
Alexander Sher, Alan M Litke, and E J Chichilnisky.

Synchronized firing among neurons has been proposed to constitute an
elementary aspect of the neural code in sensory and motor systems.
However, it remains unclear how synchronized firing affects the
large-scale patterns of activity and redundancy of visual signals in a
complete population of neurons. We recorded simultaneously from
hundreds of retinal ganglion cells in primate retina, and examined
synchronized firing in completely sampled populations of approximately
50-100 ON-parasol cells, which form a major projection to the
magnocellular layers of the lateral geniculate nucleus. Synchronized
firing in pairs of cells was a subset of a much larger pattern of
activity that exhibited local, isotropic spatial properties. However,
a simple model based solely on interactions between adjacent cells
reproduced 99% of the spatial structure and scale of synchronized
firing. No more than 20% of the variability in firing of an individual
cell was predictable from the activity of its neighbors. These results
held both for spontaneous firing and in the presence of independent
visual modulation of the firing of each cell. In sum, large-scale
synchronized firing in the entire population of ON-parasol cells
appears to reflect simple neighbor interactions, rather than a unique
visual signal or a highly redundant coding scheme.

Journal of Neuroscience, 2009 vol. 29 (15) pp. 5022-5031

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=19369571&retmode=ref&cmd=prlinks
comment: also uses a maximum entropy model to characterize the joint
activity of pairs of neurons

#####################################################################################

Spatio-temporal correlations and visual signalling in a complete
neuronal population.

Jonathan W Pillow, Jonathon Shlens, Liam Paninski, Alexander Sher,
Alan M Litke, E J Chichilnisky, and Eero P Simoncelli.

Statistical dependencies in the responses of sensory neurons govern
both the amount of stimulus information conveyed and the means by
which downstream neurons can extract it. Although a variety of
measurements indicate the existence of such dependencies, their origin
and importance for neural coding are poorly understood. Here we
analyse the functional significance of correlated firing in a complete
population of macaque parasol retinal ganglion cells using a model of
multi-neuron spike responses. The model, with parameters fit directly
to physiological data, simultaneously captures both the stimulus
dependence and detailed spatio-temporal correlations in population
responses, and provides two insights into the structure of the neural
code. First, neural encoding at the population level is less noisy
than one would expect from the variability of individual neurons:
spike times are more precise, and can be predicted more accurately
when the spiking of neighbouring neurons is taken into account.
Second, correlations provide additional sensory information: optimal,
model-based decoding that exploits the response correlation structure
extracts 20% more information about the visual scene than decoding
under the assumption of independence, and preserves 40% more visual
information than optimal linear decoding. This model-based approach
reveals the role of correlated activity in the retinal coding of
visual stimuli, and provides a general framework for understanding the
importance of correlated activity in populations of neurons.

Nature, 2008 vol. 454 (7207) pp. 995-999

http://libproxy1.nus.edu.sg/login?url=http://www.nature.com/doifinder/10.1038/nature07140
Comment: uses a generalized linear model to show that decoding using
correlations gives more information than ignoring them.

#####################################################################################

Neural correlations, population coding and computation.

Bruno B Averbeck, Peter E Latham, and Alexandre Pouget.

How the brain encodes information in population activity, and how it
combines and manipulates that activity as it carries out computations,
are questions that lie at the heart of systems neuroscience. During
the past decade, with the advent of multi-electrode recording and
improved theoretical models, these questions have begun to yield
answers. However, a complete understanding of neuronal variability,
and, in particular, how it affects population codes, is missing. This
is because variability in the brain is typically correlated, and
although the exact effects of these correlations are not known, it is
known that they can be large. Here, we review studies that address the
interaction between neuronal noise and population codes, and discuss
their implications for population coding in general.

Nature Reviews Neuroscience, 2006 vol. 7 (5) pp. 358-366

http://libproxy1.nus.edu.sg/login?url=http://www.nature.com/doifinder/10.1038/nrn1888
Comment: Another nice review of population coding

#####################################################################################

Population coding of orientation in the visual cortex of alert
cats--an information theoretic analysis.

Christoph Kayser and Peter König.

We studied the encoding of stimulus orientation in the visual cortex
of alert animals using information theory methods. Based on a
labeled-line code, the encoding of orientation was mostly synergistic
and only few pairs coded redundant. The synergy contributed about 20%
of the information and was strongest for sites with distinct tuning
curves. A recently proposed decomposition of synergy revealed that
redundancy introduced by common tuning preferences is more than just
compensated by noise correlations which mostly contributed
synergistically. Based on a pooled response code, the contribution of
noise correlations diminished resulting in a severe information loss.
Thus, to operate economically, cortical neurons should either employ a
labeled-line code or, if using pooled responses, be highly selective
in choosing afferents.

Neuroreport, 2004 vol. 15 (18) pp. 2761-2764

http://libproxy1.nus.edu.sg/login?url=http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=15597049&retmode=ref&cmd=prlinks
Comments: Another paper computing pair-wise information using firing rates.

#####################################################################################

Information Transmission and Detection Thresholds in the Vestibular
Nuclei: Single Neurons versus Population Encoding.

C Massot, M J Chacron, and K E Cullen.

Under- standing how sensory neurons transmit information about
relevant stimuli remains a major goal in neuroscience. Of particular
relevance are the roles of neural variability and spike timing in
neural coding. Peripheral vestibular afferents display differential
variability that is correlated with the importance of spike timing;
regular afferents display little variability and use a timing code to
transmit information about sensory input. Irregular afferents,
conversely, display greater variability and instead use a rate code.
We studied how central neurons within the vestibular nuclei integrate
information from both afferent classes by recording from a group of
neurons termed vestib- ular only (VO) that are known to make
contributions to vestibulospi- nal reflexes and project to
higher-order centers. We found that, although individual central
neurons had sensitivities that were greater than or equal to those of
individual afferents, they transmitted less information. In addition,
their velocity detection thresholds were significantly greater than
those of individual afferents. This is because VO neurons display
greater variability, which is detrimental to infor- mation
transmission and signal detection. Combining activities from multiple
VO neurons increased information transmission. However, the
information rates were still much lower than those of equivalent
afferent populations. Furthermore, combining responses from multi- ple
VO neurons led to lower velocity detection threshold values
approaching those measured from behavior (􏰙2.5 vs. 0.5–1°/s). Our
results suggest that the detailed time course of vestibular stimuli
encoded by afferents is not transmitted by VO neurons. Instead, they
suggest that higher vestibular pathways must integrate information
from central vestibular neuron populations to give rise to
behaviorally observed detection thresholds.

Journal of Neurophysiology, 2011 vol. 105 pp. 1798-1814

http://libproxy1.nus.edu.sg/login?url=http://jn.physiology.org/cgi/doi/10.1152/jn.00910.2010
Comments: They assume that neurons are independent, and compute the
joint information by summing the information of individual neurons.