EDITORIAL BOARD Robert Davison Aviles, Bradley University Harley E. Baker, California State University–Channel Islands Jean-Guy Blais, Universite de Montreal, Canada Catherine Y. Chang, Georgia State University Robert C. Chope, San Francisco State University Kevin O. Cokley, University of Missouri, Columbia Patricia B. Elmore, Southern Illinois University Shawn Fitzgerald, Kent State University John J. Fremer, Educational Testing Service Vicente Ponsoda Gil, Universidad Autonoma de Madrid, Spain Jo-Ida C. Hansen, University of Minnesota Charles C. Healy, University of California at Los Angeles Robin K. Henson, University of North Texas Flaviu Adrian Hodis, Victoria University of Wellington, New Zealand Janet K. Holt, Northern Illinois University David A. Jepsen, The University of Iowa Gregory Arief D. Liem, National Institute of Education, Nanyang Technological University Wei-Cheng J. Mau, Wichita State University Larry Maucieri, Governors State College Patricia Jo McDivitt, Data Recognition Corporation Peter F. Merenda, University of Rhode Island Matthew J. Miller, University of Maryland Ralph O. Mueller, University of Hartford Jane E. Myers, The University of North Carolina at Greensboro Philip D. Parker, University of Western Sydney Ralph L. Piedmont, Loyola College in Maryland Alex L. Pieterse, University at Albany, SUNY Nicholas J. Ruiz, Winona State University James P. Sampson, Jr., Florida State University William D. Schafer, University of Maryland, College Park William E. Sedlacek, University of Maryland, College Park Marie F. Shoffner, University of Virginia Len Sperry, Florida Atlantic University Kevin Stoltz, University of Mississippi Jody L. Swartz-Kulstad, Seton Hall University Bruce Thompson, Texas A&M University Timothy R. Vansickle, Minnesota Department of Education Steve Vensel, Palm Beach Atlantic University Dan Williamson, Lindsey Wilson College F. Robert Wilson, University of Cincinnati

measurement and evaluation in counseling and Development

With genre now viewed as a fundamental element of writing, both second language writing and mainstream composition studies have seen an increased focus on the question of how writers learn genres. The purpose of this paper is to review key findings from 60 empirical studies that have investigated this question. To this point, research has typically studied genre learning as it occurs either through professional or disciplinary practice or through classroom instruction; almost no studies have looked at the same writers as they traverse these multiple domains. I therefore categorize studies as taking place in either ‘practice-based’ or ‘instructional’ settings and identify trends in the research findings from each setting. After examining one study which takes place in multiple settings, I tease out some of the commonalities and distinctions between learning in practice-based and instructional contexts and between first language and second language genre learning. On the basis of this comparative review of research, I suggest future directions for the interdisciplinary study of genre learning.

Language Testing に関する覚書

Language Test Construction and Evaluation

From Testing Research to Educational Policy: Introducing a New Comprehensive Test of Oral Proficiency.

This study examined the number of writing samples needed to obtain a reliable estimate of young struggling writers’ capabilities. It further assessed if performance in one genre was reflective of performance in other genres for these children. Second- and third-grade students (81 boys, 56 girls), who were identified as struggling writers in need of special assistance by their teacher and scored at the 25th percentile or lower on a norm-referenced story-writing test, wrote four compositions: a story, personal narrative, opinion essay, and informative text. Applying generalizability theory (G-theory), students’ scores on three writing measures (total number of words [TNW], vocabulary diversity, and writing quality) for the four compositions were each portioned into variance due to the following sources: students, writing tasks, and the interaction between students and writing tasks. We found that 14, 8, and 11 compositions, respectively, would be needed to obtain a reliable estimate of these students’ writi...

/pdf/assessing-the-writing-achievement-ofyoung-struggling-writers-4gevq4f3k0.pdf

Assessing the Writing Achievement ofYoung Struggling Writers: Application ofGeneralizability Theory

This study addressed a need to examine and improve current assessments of listening comprehension (LC) of university EFL learners. These assessments adopted a traditional approach where test-takers listened to an audio recording of a spoken interaction and then independently responded to a set of questions. This static approach to assessment is at odds with the way teaching listening was carried out in the classroom, where LC tasks often involved some scaffolding. To address this limitation, a dynamic assessment (DA) of a listening test was proposed and investigated. DA involves mediation and meaning negotiation when responding to LC tasks and items. This paper described: (a) the local assessment context, (b) the relevance of DA in this context, and (c) the findings of an empirical study that examined the new and current LC assessments. Sixty Tunisian EFL students responded to a LC test with two parts, static and dynamic. The tests were scored by 11 raters. Both the test-takers and raters were interviewed about their views of the two assessments. Score analyses, using the Multi-Facet Rasch Measurement (MFRM) (FACETS program, version, 3.61.0), indicated that test-taker ability, rater behavior and item difficulty estimates varied across test types. Qualitative data analysis indicated that although the new assessment provided better insights into learners' cognitive and meta-cognitive processes than did the traditional assessment, raters were doubtful about the value of and processes involved in DA mainly because they were unfamiliar with it. The paper discussed the findings and their implications for listening assessment practices in this context and for theory and research on listening assessment.

/pdf/developing-and-evaluating-a-dynamic-assessment-of-listening-1pkmql5mr6.pdf

Developing and evaluating a dynamic assessment of listening comprehension in an EFL context

The overriding objective of this study was to investigate secondary and university teachers’ assessment conceptions in an EFL context using a four-factor ( Student Accountability, School Accountability, Improvement and Irrelevance ) teachers’ conceptions of assessment (TCoA) inventory (Brown, 2006). Data of the study were collected by a questionnaire administered to secondary school ( n =336) and university ( n =206) teachers. Factor analyses (exploratory factor analysis (EFA), parallel analysis (principal component analysis (PCA)), dimension analysis (SPSS R-Menu v.2.0) and confirmatory factor analysis (CFA)) using SPSS v. 22.0 and AMOS v. 21.0 were examined. Results delineated a three-factor model ( Accountability , Improvement and Irrelevant ) with an endorsement of and a significant relationship between Accountability and Improvement . The ecological relationship between factors and indicators denoted teachers’ misconceptions about assessment. Implications for future research on TCoA in similar-related contexts were also discussed.

/pdf/conceptions-of-assessment-investigating-what-assessment-4qh7yae5sl.pdf

Conceptions of assessment: Investigating what assessment means to secondary and university teachers

Revisiting the Assessment of Second Language Abilities: From Theory to Practice

This article is twofold: (a) it explored the impacts of mediation strategies on promoting cognitive modifiability in a dynamic assessment (DA) writing exam among ESP learners, and (b) it co...

Static vs. dynamic assessment of students’ writing exams: a comparison of two assessment modes

Validation is carried out to explore the different facets that come into play to design useful dynamic tests. This chapter underscored a theoretical and practical overview of test specs (specs) validation of a dynamic assessment (DA) of a reading comprehension test for learners of English in an EAP program. A special focus was attended to qualitative and quantitative data analyses of interactions between two mediators and test-takers in three testing phases: Input, interaction and output and a Multi-Faceted RASCH Measurement, FACETS analysis, of test scores. Results showed that test-takers’ performance significantly improved with the support of a mediator; thus resulting in a more relevant output. The FACETS quantitative results also confirmed the results of the qualitative analysis. The study suggested a list of specs for designing dynamic reading tests for learners of English in a similar-related context. Limitations and recommendations were also discussed.

Sahbi Hidri

Papers

Developing and evaluating a dynamic assessment of listening comprehension in an EFL context

Conceptions of assessment: Investigating what assessment means to secondary and university teachers

Revisiting the Assessment of Second Language Abilities: From Theory to Practice

Static vs. dynamic assessment of students’ writing exams: a comparison of two assessment modes

Specs Validation of a Dynamic Reading Comprehension Test for EAP Learners in an EFL Context