CIVIL rights legislation, the attendant actions of compliance agencies, and a few landmark court cases have provided the impetus for the extension of the application of content validity from academic achievement testing to personnel testing in business and industry. Pressed by the legal requirement to demonstrate validity, and constrained by the limited applicability of traditional criterion-related methodologies, practitioners are more and more turning to content validity in search of solutions. Over time, criterion-related validity principles and strategies have evolved so that the term, "commonly accepted professional practice" has meaning. Such is not the case with content validity. The relative newness of the field, the proprietary nature of work done by professionals practicing in industry, to say nothing of the ever present legal overtones, have predictably militated against publication in the journals and formal discussion at professional meetings. There is a paucity of literature on content validity in employment testing, and much of what exists has eminated from civil service commissions. The selectipn of civil servants, with its eligibility lists and "pass-fail" concepts, has always been something of a special case with limited transferability to industry. Given the current lack of consensus in professional practice, practitioners will more and more face each other in adversary roles as expert witnesses for plaintiff and defendant. Until professionals reach some degree of concurrence regarding what constitutes acceptable evidence of content validity, there is a serious risk that the courts and the enforcement agencies will play the major determining role. Hopefully, this paper will modestly contribute to the improvement of this state of affairs (1) by helping sharpen the content ' A paper presented at Content Validity [1, a conference held at Bowling Green

/pdf/a-quantitative-approach-to-content-validity-37betfnl24.pdf

A quantitative approach to content validity

An explanation of the effects of leader behavior on subordinate satisfaction, motivation, and performance is presented. The explanation is derived from a path-goal theory of motivation. Dimensions of leader behavior such as leader initiating structure, consideration, authoritarianism, hierarchical influence, and closeness of supervision are analyzed in terms of path-goal variables such as valence and instrumentality. The theory specifies some of the situational moderators on which the effects of specific leader behaviors are contingent. A set of general propositions are advanced which integrate and explain earlier fragmentary research findings. Several specific predictions are made to illustrate how the general propositions can be operationalized. The usefulness of the theory is demonstrated by showing how several seemingly unrelated prior research findings could have been deduced from its general propositions and by applying it to reconcile what appear to be contradictory findings from prior studies. Results of two empirical studies are reported that provide support for seven of eight hypotheses derived directly from the general propositions of the theory. A third study designed to test three of the original eight hypotheses is also reported. Two of these three hypotheses are successfully replicated. In the light of these results and the integrative power of the theory, it is argued that the theory shows promise and should be further tested with experimentallas well as correlational methods.

A Path-Goal Theory of Leader Effectiveness

The issue of whether IS positivist researchers were validating their instruments sufficiently was initially raised fifteen years ago. Rigor in IS research is still one of the critical scientific issues facing the field. Without solid validation of the instruments that are used to gather data on which findings and interpretations are based, the very scientific basis of the profession is threatened. This study builds on four prior retrospectives of IS research that conclude that IS positivist researchers continue to face major barriers in instrument, statistical, and other forms of validation. It goes beyond these studies by offering analyses of the state-of-the-art of research validities and deriving specific heuristics for research practice in the validities. Some of these heuristics will, no doubt, be controversial. But we believe that it is time for the IS academic profession to bring such issues into the open for community debate. This article is a first step in that direction. Based on our interpretation of the importance of a long list of validities, this paper suggests heuristics for reinvigorating the quest for validation in IS research via content/construct validity, reliability, manipulation validity, and statistical conclusion validity. New guidelines for validation and new research directions are offered.

/pdf/validation-guidelines-for-is-positivist-research-5fusz79fbw.pdf

Validation guidelines for is positivist research

Nurse researchers typically provide evidence of content validity for instruments by computing a content validity index (CVI), based on experts' ratings of item relevance. We compared the CVI to alternative indexes and concluded that the widely-used CVI has advantages with regard to ease of computation, understandability, focus on agreement of relevance rather than agreement per se, focus on consensus rather than consistency, and provision of both item and scale information. One weakness is its failure to adjust for chance agreement. We solved this by translating item-level CVIs (I-CVIs) into values of a modified kappa statistic. Our translation suggests that items with an I-CVI of .78 or higher for three or more experts could be considered evidence of good content validity.

Is the CVI an acceptable indicator of content validity? Appraisal and recommendations

Nurse researchers typically provide evidence of content validity for instruments by computing a content validity index (CVI), based on experts' ratings of item relevance. We compared the CVI to alternative indexes and concluded that the widely-used CVI has advantages with regard to ease of computation, understandability, focus on agreement of relevance rather than agreement per se, focus on consensus rather than consistency, and provision of both item and scale information. One weakness is its failure to adjust for chance agreement. We solved this by translating item-level CVIs (I-CVIs) into values of a modified kappa statistic. Our translation suggests that items with an I-CVI of .78 or higher for three or more experts could be considered evidence of good content validity. 2007 Wiley Periodicals, Inc. Res Nurs Health 30:459-467, 2007

/pdf/focus-on-research-methods-is-the-cvi-an-acceptable-indicator-1c7cnx29u8.pdf

C. H. Lawshe

Papers

A quantitative approach to content validity

Principles of personnel testing

What can industrial psychology do for small business (a symposium) 2. Employee selection.

Foreman Leadership as Perceived By Superiors and Subordinates1

Expectancy Charts II. Their Theoretical Development