Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

I have developed "tennis elbow" from lugging this book around the past four weeks, but it is worth the pain, the effort, and the aspirin. It is also worth the (relatively speaking) bargain price. Including appendixes, this book contains 894 pages of text. The entire panorama of the neural sciences is surveyed and examined, and it is comprehensive in its scope, from genomes to social behaviors. The editors explicitly state that the book is designed as "an introductory text for students of biology, behavior, and medicine," but it is hard to imagine any audience, interested in any fragment of neuroscience at any level of sophistication, that would not enjoy this book. The editors have done a masterful job of weaving together the biologic, the behavioral, and the clinical sciences into a single tapestry in which everyone from the molecular biologist to the practicing psychiatrist can find and appreciate his or

Principles of Neural Science

A free-energy principle has been proposed recently that accounts for action, perception and learning. This Review looks at some key brain theories in the biological (for example, neural Darwinism) and physical (for example, information theory and optimal control theory) sciences from the free-energy perspective. Crucially, one key theme runs through each of these theories — optimization. Furthermore, if we look closely at what is optimized, the same quantity keeps emerging, namely value (expected reward, expected utility) or its complement, surprise (prediction error, expected cost). This is the quantity that is optimized under the free-energy principle, which suggests that several global brain theories might be unified within a free-energy framework.

/pdf/the-free-energy-principle-a-unified-brain-theory-3vz1twg7nv.pdf

The free-energy principle: a unified brain theory?

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. This target article critically examines this "hierarchical prediction machine" approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

/pdf/whatever-next-predictive-brains-situated-agents-and-the-epj56cs868.pdf

Whatever next? Predictive brains, situated agents, and the future of cognitive science

The Reorienting System of the Human Brain: From Environment to Theory of Mind

http://www.gatsby.ucl.ac.uk/~dayan/papers/yud2005.pdf

Uncertainty, neuromodulation, and attention.

Many large and small decisions we make in our daily lives—which ice cream to choose, what research projects to pursue, which partner to marry—require an exploration of alternatives before committing to and exploiting the benefits of a particular choice. Furthermore, many decisions require re-evaluation, and further exploration of alternatives, in the face of changing needs or circumstances. That is, often our decisions depend on a higher level choice: whether to exploit well known but possibly suboptimal alternatives or to explore risky but potentially more profitable ones. How adaptive agents choose between exploitation and exploration remains an important and open question that has received relatively limited attention in the behavioural and brain sciences. The choice could depend on a number of factors, including the familiarity of the environment, how quickly the environment is likely to change and the relative value of exploiting known sources of reward versus the cost of reducing uncertainty through exploration. There is no known generally optimal solution to the exploration versus exploitation problem, and a solution to the general case may indeed not be possible. However, there have been formal analyses of the optimal policy under constrained circumstances. There have also been specific suggestions of how humans and animals may respond to this problem under particular experimental conditions as well as proposals about the brain mechanisms involved. Here, we provide a brief review of this work, discuss how exploration and exploitation may be mediated in the brain and highlight some promising future directions for research.

/pdf/should-i-stay-or-should-i-go-how-the-human-brain-manages-the-3od8yuln5f.pdf

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

Extensive animal studies indicate that the neuromodulator norepinephrine plays an important role in specific aspects of vigilance, attention and learning, putatively serving as a neural interrupt or reset function. The activity of norepinephrine-releasing neurons in the locus coeruleus during attentional tasks is modulated not only by the animal's level of engagement and the sensory inputs, but also by temporally rich aspects of internal decision-making processes. Here, we propose that it is unexpected changes in the world within the context of a task that activate the noradrenergic interrupt signal. We quantify this idea in a Bayesian model of a well-studied visual discrimination task, demonstrating that the model captures a rich repertoire of noradrenergic responses at the sub-second temporal resolution.

/pdf/phasic-norepinephrine-a-neural-interrupt-signal-for-13sl9e93yi.pdf

Phasic norepinephrine: a neural interrupt signal for unexpected events.

It is largely recognized that microRNAs (miRNAs) function to silence gene expression by targeting 3'UTR regions. However, miRNAs have also been implicated to positively-regulate gene expression by targeting promoter elements, a phenomenon known as RNA activation (RNAa). In the present study, we show that expression of mouse Cyclin B1 (Ccnb1) is dependent on key factors involved in miRNA biogenesis and function (i.e. Dicer, Drosha, Ago1 and Ago2). In silico analysis identifies highly-complementary sites for 21 miRNAs in the Ccnb1 promoter. Experimental validation identified three miRNAs (miR-744, miR-1186 and miR-466d-3p) that induce Ccnb1 expression in mouse cell lines. Conversely, knockdown of endogenous miR-744 led to decreased Ccnb1 levels. Chromatin immunoprecipitation (ChIP) analysis revealed that Ago1 was selectively associated with the Ccnb1 promoter and miR-744 increased enrichment of RNA polymerase II (RNAP II) and trimethylation of histone 3 at lysine 4 (H3K4me3) at the Ccnb1 transcription start site. Functionally, short-term overexpression of miR-744 and miR-1186 resulted in enhanced cell proliferation, while prolonged expression caused chromosomal instability and in vivo tumor suppression. Such phenotypes were recapitulated by overexpression of Ccnb1. Our findings reveal an endogenous system by which miRNA functions to activate Ccnb1 expression in mouse cells and manipulate in vivo tumor development/growth.

/pdf/upregulation-of-cyclin-b1-by-mirna-and-its-implications-in-mmclc6zre5.pdf

Upregulation of Cyclin B1 by miRNA and its implications in cancer

In a variety of behavioral tasks, subjects exhibit an automatic and apparently sub-optimal sequential effect: they respond more rapidly and accurately to a stimulus if it reinforces a local pattern in stimulus history, such as a string of repetitions or alternations, compared to when it violates such a pattern. This is often the case even if the local trends arise by chance in the context of a randomized design, such that stimulus history has no real predictive power. In this work, we use a normative Bayesian framework to examine the hypothesis that such idiosyncrasies may reflect the inadvertent engagement of mechanisms critical for adapting to a changing environment. We show that prior belief in non-stationarity can induce experimentally observed sequential effects in an otherwise Bayes-optimal algorithm. The Bayesian algorithm is shown to be well approximated by linear-exponential filtering of past observations, a feature also apparent in the behavioral data. We derive an explicit relationship between the parameters and computations of the exact Bayesian algorithm and those of the approximate linear-exponential filter. Since the latter is equivalent to a leaky-integration process, a commonly used model of neuronal dynamics underlying perceptual decision-making and trial-to-trial dependencies, our model provides a principled account of why such dynamics are useful. We also show that parameter-tuning of the leaky-integration process is possible, using stochastic gradient descent based only on the noisy binary inputs. This is a proof of concept that not only can neurons implement near-optimal prediction based on standard neuronal dynamics, but that they can also learn to tune the processing parameters without explicitly representing probabilities.

/pdf/sequential-effects-superstition-or-rational-behavior-r85qd333j9.pdf

Angela J. Yu

Papers

Uncertainty, neuromodulation, and attention.

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

Phasic norepinephrine: a neural interrupt signal for unexpected events.

Upregulation of Cyclin B1 by miRNA and its implications in cancer

Sequential effects: Superstition or rational behavior?