The online laboratory: conducting experiments in a real labor market
TLDR
In this paper, the authors use an online labor market to replicate three classic experiments and find quantitative agreement between levels of cooperation in a prisoner's dilemma played online and in the physical laboratory.Abstract:
Online labor markets have great potential as platforms for conducting experiments. They provide immediate access to a large and diverse subject pool, and allow researchers to control the experimental context. Online experiments, we show, can be just as valid—both internally and externally—as laboratory and field experiments, while often requiring far less money and time to design and conduct. To demonstrate their value, we use an online labor market to replicate three classic experiments. The first finds quantitative agreement between levels of cooperation in a prisoner’s dilemma played online and in the physical laboratory. The second shows—consistent with behavior in the traditional laboratory—that online subjects respond to priming by altering their choices. The third demonstrates that when an identical decision is framed differently, individuals reverse their choice, thus replicating a famed Tversky-Kahneman result. Then we conduct a field experiment showing that workers have upward-sloping labor supply curves. Finally, we analyze the challenges to online experiments, proposing methods to cope with the unique threats to validity in an online setting, and examining the conceptual issues surrounding the external validity of online results. We conclude by presenting our views on the potential role that online experiments can play within the social sciences, and then recommend software development priorities and best practices.read more
Citations
More filters
Journal ArticleDOI
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
TL;DR: It is shown that respondents recruited in this manner are often more representative of the U.S. population than in-person convenience samples but less representative than subjects in Internet-based panels or national probability samples.
Journal ArticleDOI
Inside the Turk Understanding Mechanical Turk as a Participant Pool
Gabriele Paolacci,Jesse Chandler +1 more
TL;DR: The characteristics of Mechanical Turk as a participant pool for psychology and other social sciences, highlighting the traits of the MTurk samples, why people become Mechanical Turk workers and research participants, and how data quality on Mechanical Turk compares to that from other pools and depends on controllable and uncontrollable factors as mentioned in this paper.
Journal ArticleDOI
Data collection in a flat world: the strengths and weaknesses of mechanical turk samples
TL;DR: The authors compared Mechanical Turk participants with community and student samples on a set of personality dimensions and classic decision-making biases and found that MTurk participants are less extraverted and have lower self-esteem than other participants, presenting challenges for some research domains.
Journal ArticleDOI
Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing
TL;DR: It is concluded that for some behavioral tests, online recruitment and testing can be a valid-and sometimes even superior-partner to in-person data collection.
Journal ArticleDOI
Prolific.ac—A subject pool for online experiments
TL;DR: This article presents www.prolific.ac and lays out its suitability for recruiting subjects for social and economic science experiments, and traces the platform’s historical development, present its features, and contrast them with requirements for different types of social andEconomic experiments.
References
More filters
Book
The Evolution of Cooperation
TL;DR: In this paper, a model based on the concept of an evolutionarily stable strategy in the context of the Prisoner's Dilemma game was developed for cooperation in organisms, and the results of a computer tournament showed how cooperation based on reciprocity can get started in an asocial world, can thrive while interacting with a wide range of other strategies, and can resist invasion once fully established.
Journal ArticleDOI
The Framing of Decisions and the Psychology of Choice
Amos Tversky,Daniel Kahneman +1 more
TL;DR: The psychological principles that govern the perception of decision problems and the evaluation of probabilities and outcomes produce predictable shifts of preference when the same problem is framed in different ways.
Journal ArticleDOI
z-Tree: Zurich toolbox for ready-made economic experiments
TL;DR: Z-Tree as mentioned in this paper is a toolbox for ready-made economic experiments, which allows programming almost any kind of experiments in a short time and is stable and easy to use.
Journal ArticleDOI
A theory of fairness, competition and cooperation
Ernst Fehr,Klaus M. Schmidt +1 more
TL;DR: This paper showed that if some people care about equity, the puzzles can be resolved and that the economic environment determines whether the fair types or the selesh types dominate equilibrium behavior in cooperative games.
Journal ArticleDOI
Estimating causal effects of treatments in randomized and nonrandomized studies.
TL;DR: A discussion of matching, randomization, random sampling, and other methods of controlling extraneous variation is presented in this paper, where the objective is to specify the benefits of randomization in estimating causal effects of treatments.