Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

In this article, we provide guidance for substantive researchers on the use of structural equation modeling in practice for theory testing and development. We present a comprehensive, two-step modeling approach that employs a series of nested models and sequential chi-square difference tests. We discuss the comparative advantages of this approach over a one-step approach. Considerations in specification, assessment of fit, and respecification of measurement models using confirmatory factor analysis are reviewed. As background to the two-step approach, the distinction between exploratory and confirmatory analysis, the distinction between complementary approaches for theory testing versus predictive application, and some developments in estimation methods also are discussed.

/pdf/structural-equation-modeling-in-practice-a-review-and-1ue2omj492.pdf

Structural equation modeling in practice: a review and recommended two-step approach

In the first part of the paper we consider the problem of dynamically apportioning resources among a set of options in a worst-case on-line framework. The model we study can be interpreted as a broad, abstract extension of the well-studied on-line prediction model to a general decision-theoretic setting. We show that the multiplicative weight-update Littlestone?Warmuth rule can be adapted to this model, yielding bounds that are slightly weaker in some cases, but applicable to a considerably more general class of learning problems. We show how the resulting learning algorithm can be applied to a variety of problems, including gambling, multiple-outcome prediction, repeated games, and prediction of points in Rn. In the second part of the paper we apply the multiplicative weight-update technique to derive a new boosting algorithm. This boosting algorithm does not require any prior knowledge about the performance of the weak learning algorithm. We also study generalizations of the new boosting algorithm to the problem of learning functions whose range, rather than being binary, is an arbitrary finite set or a bounded segment of the real line.

/pdf/a-decision-theoretic-generalization-of-on-line-learning-and-3ykfx1u7er.pdf

A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

We develop a new version of prospect theory that employs cumulative rather than separable decision weights and extends the theory in several respects. This version, called cumulative prospect theory, applies to uncertain as well as to risky prospects with any number of outcomes, and it allows different weighting functions for gains and for losses. Two principles, diminishing sensitivity and loss aversion, are invoked to explain the characteristic curvature of the value function and the weighting functions. A review of the experimental evidence and the results of a new experiment confirm a distinctive fourfold pattern of risk attitudes: risk aversion for gains and risk seeking for losses of high probability; risk seeking for gains and risk aversion for losses of low probability. Expected utility theory reigned for several decades as the dominant normative and descriptive model of decision making under uncertainty, but it has come under serious question in recent years. There is now general agreement that the theory does not provide an adequate description of individual choice: a substantial body of evidence shows that decision makers systematically violate its basic tenets. Many alternative models have been proposed in response to this empirical challenge (for reviews, see Camerer, 1989; Fishburn, 1988; Machina, 1987). Some time ago we presented a model of choice, called prospect theory, which explained the major violations of expected utility theory in choices between risky prospects with a small number of outcomes (Kahneman and Tversky, 1979; Tversky and Kahneman, 1986). The key elements of this theory are 1) a value function that is concave for gains, convex for losses, and steeper for losses than for gains,

http://psych.fullerton.edu/mbirnbaum/psych466/articles/Tversky_Kahneman_JRU_92.pdf

Advances in prospect theory: cumulative representation of uncertainty

Publisher Summary Turn taking is used for the ordering of moves in games, for allocating political office, for regulating traffic at intersections, for the servicing of customers at business establishments, and for talking in interviews, meetings, debates, ceremonies, conversations. This chapter discusses the turn-taking system for conversation. On the basis of research using audio recordings of naturally occurring conversations, the chapter highlights the organization of turn taking for conversation and extracts some of the interest that organization has. The turn-taking system for conversation can be described in terms of two components and a set of rules. These two components are turn-constructional component and turn-constructional component. Turn-allocational techniques are distributed into two groups: (1) those in which next turn is allocated by current speaker selecting a next speaker and (2) those in which next turn is allocated by self-selection. The turn-taking rule-set provides for the localization of gap and overlap possibilities at transition-relevance places and their immediate environment, cleansing the rest of a turn's space of systematic bases for their possibility.

/pdf/a-simplest-systematics-for-the-organization-of-turn-taking-536vognu54.pdf

A simplest systematics for the organization of turn-taking for conversation

Games and decisions

Individual Choice Behavior

Individual Choice Behavior: A Theoretical Analysis

1. Representing Response Times as Random Variables PART I: DETECTION PARADIGMS 2. Simple Reaction Times: Basic Data 3. Decomposition into Decision and Residual Latencies 4. Distributions of Simple Decision Latencies 5. Detection of Signals Presented at Irregular Times PART II: IDENTIFICATION PARADIGMS 6. Two-Choice Reaction Times: Basic Ideas and Data 7. Mixture Models 8. Stochastic Accumulation of Information in Discrete Time 9. Stochastic Accumulation of Information in Continuous Time 10. Absolute Identification of More than Two Signals PART III: MATCHING PARADIGMS 11. Memory, Scanning, Visual Search, and Same-Difference Designs 12. Processing Stages and Strategies

R. Duncan Luce

Papers

Games and decisions

Individual Choice Behavior

Individual Choice Behavior: A Theoretical Analysis

Individual Choice Behavior: A Theoretical Analysis

Response Times: Their Role in Inferring Elementary Mental Organization