System and Method for Automated Scoring of a Summary-Writing Task

摘要

In accordance with the teachings described herein, systems and methods are provided for measuring a user's comprehension of subject matter of a text. A summary generated by the user is received, where the summary summarizes the text. The summary is processed to determine a first numerical measure indicative of a similarity between the summary and a reference summary. The summary is processed to determine a second numerical measure indicative of a degree to which a single sentence of the summary summarizes an entirety of the text. The summary is processed to determine a third numerical measure indicative of a degree of copying in the summary of multi-word sequences present in the text. A numerical model is applied to the first numerical measure, the second numerical measure and the third numerical measure to determine a score for the summary indicative of the user's comprehension of the subject matter of the text.

主权项

1. A computer-implemented method of measuring a user's comprehension of subject matter of a text, the method comprising:
receiving a summary generated by the user, the summary being a constructed response that summarizes a text; parsing the summary with a processing system to identify a number of sentences contained in the summary and to identify in the summary a plurality of multi-word sequences; processing the summary and a reference summary with the processing system to determine a first numerical measure indicative of a similarity between the summary and a reference summary, the reference summary having been designated as representative of the subject matter of the text; processing the summary with the processing system to determine a second numerical measure indicative of a degree to which a single sentence of the summary summarizes an entirety of the text; processing the summary and the text with the processing system to determine a third numerical measure indicative of a degree of copying in the summary of multi-word sequences present in the text; and applying a numerical model to the first numerical measure, the second numerical measure and the third numerical measure to determine a score for the summary indicative of the user's comprehension of the subject matter of the text, the numerical model including
a first variable and an associated first weighting factor, the first variable receiving a value of the first numerical measure,a second variable and an associated second weighting factor, the first variable receiving a value of the second numerical measure, anda third variable and an associated third weighting factor, the third variable receiving a value of the third numerical measure.