Interrater Reliability Services

It is crucial that Test Administrators have a common understanding of how student responses are scored in relation to the LAS Links scoring rubric. LAS Links test level statistics were used to describe form performance and provide evidence for the evaluation of form comparability year and across grades, reasonableness of student performance overall and by subgroups, inter-rater and internal consistency (reliability), and consistency of classification.

Content-related validity for language proficiency tests is evidenced by a correspondence between test content and instructional content. To ensure such correspondence, developers conducted a comprehensive curriculum review and met with educational experts to determine common educational goals and the knowledge and skills emphasized in curricula across the United States.

LAS Links program was developed in accordance with the criteria for test development, administration, and use described in the Standards for Educational and Psychological Testing (1999) adopted by:

  • The American Educational Research Association (AERA),
  • The American Psychological Association (APA), and the National Council of Measurement in Education (NCME)

Interrater Reliability
Check sets papers selected by the scoring supervisor that closely matched the established scoring rubrics and guidelines), were administered to monitor scoring accuracy and to maintain the established rubrics and guidelines. Readers whose scores differed from the check -set were removed and given additional training and, subsequently, dismissed from scoring if they were not able to re-qualify. Exact agreement between reader scores and check-set scores was obtained on approximately 80% to 100% of the check-set papers.

A significant number of papers in each content area were scored by a second reader to establish interrater- reliability statistics for all constructed-response items. Double-blind reads were used to maintain high rates of interrater reliability over time.
Interrater reliability study for all constructed-response items showed uniformly high reliability for all items, indicating good agreement between the first and second readers.

Other (Predictive, Face, or Construct Validity)
LAS Links was designed based on the tenet that second-language acquisition occurs when students have multiple opportunities to process meaning with their fellow students, teachers, and other adults both in and outside of school. Students process information best when that information is systematically introduced, when it is engaging, and when it is accompanied by opportunities to practice the knowledge and skills required to learn the language they will need for academic, social, and testing situations. The tasks and item types in LAS Links are engaging for students and take into account their different learning styles.