Systems, Methods, and Software for Enabling Automated, Interactive Assessment

摘要

Methods, system, and software that enables students to create high quality, automatically gradable questions without requiring any manual rating of questions, while at the same time aiding the learning of the students in each of their interactions with the system. The problem of determining the quality of student-submitted questions may be solved by automatically assigning discrimination scores to questions that indicate the extent to which successfully answering a question corresponds to overall learning achievement, e.g., total score on a set of questions. Students may then be rewarded for creating questions with high discrimination scores (as a proxy of high quality of the question) and/or correctly answering questions with high discrimination scores. A question bank of high-quality automatically gradable questions can be created that can be used in the same or future iterations of a course. Both creating the questions and taking the tests are valuable learning experiences for students.

主权项

1. A method of enabling automated, interactive assessment of one or more of a plurality of untrusted individuals distributed across one or more networks with respect to one or more subjects, the method performed by an assessment management system and comprising:
displaying to one or more individuals of a first portion of the plurality of individuals a prompt for an assessment item including a specification related to the one or more subjects and one or more consistent responses and inconsistent responses to the specification; receiving at least two assessment items from one or more individuals of the first portion of the plurality of individuals; displaying to a first individual of a second portion of the plurality of individuals specifications of at least two assessment items received from at least one different individual of the plurality of individuals and prompts for responses to the specifications; receiving responses to the specifications from the first individual; displaying to a second individual of the second portion of the plurality of individuals specifications of at least two assessment items received from at least one different individual of the plurality of individuals and prompts for responses to the specifications, wherein the specifications of at least two assessment items displayed to the second individual are the specifications of at least two assessment items displayed to the first individual; receiving responses to the specifications from the second individual; determining an assessment result for each response received in response to each respective specification as a function of one or more of a consistent response and an inconsistent response to the respective specification received from the one or more individuals of the first portion of the plurality of individuals; determining an assessment item quality for each respective assessment item as a function of a correlation between the assessment result for each response received in response to the specification of the assessment item from the first and second individuals and assessment results for responses received in response to a specification of at least one different respective assessment item from the first and second individuals; and generating and storing an overall assessment of one or more individuals of the plurality of individuals with respect to the one or more subjects as a function of the assessment item quality for at least one assessment item either for which a specification was received from the individual or in response to the specification of which a response was received from the individual.