The Development of An Item Pool for Assessing Practitioner Knowledge of The Attributes of Recommended Grading and Marking Systems

The purpose of this study was to create a pool of test items to assess knowledge of principles and practices associated with the rational, psychometric, legal, and learning domains of grading and marking systems, for which reliability estimates and content validity characteristics would be established. The items were created and validated by implementing a plan to enhance content validity and assess reliability. Two research objectives were specified to assist in validating the test items. These were: (1) To ascertain the extent of agreement among experts, with content knowledge congruent with the four domains of grading and marking systems, as to what information is essential for practitioners to either know or apply to implement grading and marking systems that are consistent with recommended practices. (2) To discover the extent of agreement among teachers who regularly apply grading and marking systems as to what information is essential to either know or apply to implement recommended grading and marking systems. Material to be included in the test items was chosen by conducting a content analysis of research findings associated with studies of grading and marking systems. The items were validated by collecting and analyzing information from a professional jury and a sample of Nebraska teachers. Data regarding the nature, content, and merit of the test items were collected from 4 school psychologists, 6 educational psychologists, 7 measurement and evaluation specialists, and 2 school attorneys; all had been identified as nationally known experts in their fields. A sample of 167 Nebraska teachers completed tasks to provide information for reliability estimates and item analyses. Although content validity was built into the items and a pool of items representing information that experts agreed was essential to know or apply was developed, the items in the pool possessed little reliability. Suggestions for modifying the items to enhance their reliability were outlined, and a list of recommendations for developing and validating grading and marking systems was delineated.