Journal ArticleDOI
Answering the Call for a Standard Reliability Measure for Coding Data
TLDR
This work proposes Krippendorff's alpha as the standard reliability measure, general in that it can be used regardless of the number of observers, levels of measurement, sample sizes, and presence or absence of missing data.Abstract:
In content analysis and similar methods, data are typically generated by trained human observers who record or transcribe textual, pictorial, or audible matter in terms suitable for analysis. Conclusions from such data can be trusted only after demonstrating their reliability. Unfortunately, the content analysis literature is full of proposals for so-called reliability coefficients, leaving investigators easily confused, not knowing which to choose. After describing the criteria for a good measure of reliability, we propose Krippendorff's alpha as the standard reliability measure. It is general in that it can be used regardless of the number of observers, levels of measurement, sample sizes, and presence or absence of missing data. To facilitate the adoption of this recommendation, we describe a freely available macro written for SPSS and SAS to calculate Krippendorff's alpha and illustrate its use with a simple example.read more
Citations
More filters
Journal ArticleDOI
Drones for parcel and passenger transportation: A literature review
TL;DR: In this paper, the authors present a systematic literature review of 111 interdisciplinary publications (2013 - 03/2019) which systematizes the current socio-technical debate on civil drones for transportation purposes allowing for a (critical) interim assessment.
Journal ArticleDOI
Splitting hair for cortisol? Associations of socio-economic status, ethnicity, hair color, gender and other child characteristics with hair cortisol and cortisone.
Ralph C. A. Rippe,Gerard Noppe,Dafna A. Windhorst,Dafna A. Windhorst,Henning Tiemeier,Elisabeth F.C. van Rossum,Vincent W. V. Jaddoe,Frank C. Verhulst,Marian J. Bakermans-Kranenburg,Marinus H. van IJzendoorn,Marinus H. van IJzendoorn,Erica L T van den Akker +11 more
TL;DR: The authors' results serve as a starting point for choosing covariates and confounders in studies of substantive predictors or outcomes and gender, BMI, income, the number of persons in a household, ethnicity, hair color and recency of hair washing are strongly suggested to take into account.
Journal ArticleDOI
Real-time mutual gaze perception enhances collaborative learning and collaboration quality
Bertrand Schneider,Roy Pea +1 more
TL;DR: In this article, the authors present the results of an eye-tracking study on collaborative problem-solving dyads, where dyads remotely collaborated to learn from contrasting cases involving basic concepts about how the human brain processes visual information.
Journal ArticleDOI
Firm Value Creation Through Major Channel Expansions: Evidence from an Event Study in the United States, Germany, and China.
TL;DR: In this article, the authors investigate whether a firm's announcement of an increase in distribution intensity or the establishment of a new channel influences firm value and test their hypotheses with an event study of 240 announcements of major channel expansions.
Journal ArticleDOI
The impact of a rubric and friendship on peer assessment: Effects on construct validity, performance, and perceptions of fairness and comfort
TL;DR: In this paper, the authors investigated the impact of an assessment rubric and friendship between the assessor and assessee on construct validity of peer assessment (PA) and found that using a rubric resulted in higher quality concept maps for peer and expert ratings.
References
More filters
Journal ArticleDOI
Coefficient alpha and the internal structure of tests.
TL;DR: In this paper, a general formula (α) of which a special case is the Kuder-Richardson coefficient of equivalence is shown to be the mean of all split-half coefficients resulting from different splittings of a test, therefore an estimate of the correlation between two random samples of items from a universe of items like those in the test.
Book
An introduction to the bootstrap
Bradley Efron,Robert Tibshirani +1 more
TL;DR: This article presents bootstrap methods for estimation, using simple arguments, with Minitab macros for implementing these methods, as well as some examples of how these methods could be used for estimation purposes.
Book
Nonparametric statistics for the behavioral sciences
TL;DR: This is the revision of the classic text in the field, adding two new chapters and thoroughly updating all others as discussed by the authors, and the original structure is retained, and the book continues to serve as a combined text/reference.
Journal ArticleDOI
A Coefficient of agreement for nominal Scales
TL;DR: In this article, the authors present a procedure for having two or more judges independently categorize a sample of units and determine the degree, significance, and significance of the units. But they do not discuss the extent to which these judgments are reproducible, i.e., reliable.
Book
Content analysis: an introduction to its methodology
TL;DR: History Conceptual Foundations Uses and Kinds of Inference The Logic of Content Analysis Designs Unitizing Sampling Recording Data Languages Constructs for Inference Analytical Techniques The Use of Computers Reliability Validity A Practical Guide