Scoring the Direct Writing Measure
For the HEIghten® direct writing measure present a writing prompt for which the test taker enters a response. Each submitted essay receives a score from at least one trained human reader, using a holistic scale. In holistic scoring, readers are trained to assign scores on the basis of the overall quality of an essay in response to the assigned task. The essay score is then scored by the e-rater® Scoring Engine, a Natural Language Processing (NLP) system developed by ETS that is capable of identifying essay features related to writing proficiency. If the human and e-rater scores closely agree, the two scores are added together and used as the final score. If they disagree, additional human scores are obtained, and the final score is determined based on the most accurate scores available.
The human reader is an important feature of scoring the Direct Writing Measure. Based upon the inclusion of human readers, the Direct Writing Measure may take up to 30 days for the scoring to be completed, and for the overall score for the HEIghten Written Communications assessment to be finalized.
Scoring Guide
The HEIghten Written Communication direct writing measure essay is scored on a holistic scale according to the criteria below. Each essay receives two scores on a 0 to 6 scale, one score from a human reader and one score from the e-rater.
A single score from 0 to 12 is reported for the Direct Writing Measure.
6 (11 to 12 Total Score) — OUTSTANDING
A rater score of 6 (11 to 12 total score) demonstrates cogent development of a position and conveys meaning skillfully. A typical response in this category:
- articulates a clear and insightful position on the claim in the stimulus
- develops the position fully with compelling reasons and/or persuasive examples
- sustains a well-focused, well-organized discussion, connecting ideas logically
- conveys ideas fluently and precisely, using effective vocabulary and sentence variety
- clearly demonstrates facility with the conventions of standard written English, but may have minor errors
5 (9 to 10 Total Score) — STRONG
A rater score of 5 demonstrates thoughtful development of a position and conveys meaning clearly. A typical response in this category:
- presents a clear and well-considered position on the claim in the stimulus
- develops the position with logically sound reasons and/or well-chosen examples
- is focused and generally well organized, connecting ideas appropriately
- conveys ideas clearly and well, using appropriate vocabulary and sentence variety
- demonstrates facility with the conventions of standard written English, but may have minor errors
4 (7 to 8 Total Score) — ADEQUATE
A rater score of 4 response demonstrates competent development of a position and conveys meaning with acceptable clarity. A typical response in this category:
- presents a clear position on the claim in the stimulus
- develops the position with relevant reasons and/or examples
- is adequately focused and organized
- demonstrates sufficient control of language to convey ideas with acceptable clarity
- generally demonstrates control of the conventions of standard written English, but may have some errors
3 (5 to 6 Total Score) — LIMITED
A rater score of 3 response demonstrates some competence in developing a position and in conveying meaning, but is obviously flawed. A typical response in this category exhibits one or more of the following characteristics:
- is vague or limited in presenting or developing a position on the claim in the stimulus
- is weak in the use of relevant reasons or examples or relies largely on unsupported claims
- is limited in focus and/or organization
- has problems in language and sentence structure that result in a lack of clarity
- contains occasional major errors or frequent minor errors in grammar, usage or mechanics that can interfere with meaning
2 (3 to 4 Total Score) — SERIOUSLY FLAWED
A rater score of 2 response demonstrates serious weaknesses in written communication. A typical response in this category exhibits one or more of the following characteristics:
- is unclear or seriously limited in presenting or developing a position on the claim in the stimulus
- provides few, if any, relevant reasons or examples in support of its claims
- is poorly focused and/or poorly organized
- has serious problems in language and sentence structure that frequently interfere with meaning
- contains serious errors in grammar, usage or mechanics that frequently obscure meaning
1 (1 to 2 Total Score) — FUNDAMENTALLY DEFICIENT
A rater score of 1 response demonstrates fundamental deficiencies in written communication. A typical response in this category exhibits one or more of the following characteristics:
- provides little or no evidence of understanding the claim in the stimulus
- provides little evidence of the ability to develop an organized response (e.g., is disorganized and/or extremely brief)
- has severe problems in language and sentence structure that persistently interfere with meaning
- contains pervasive errors in grammar, usage or mechanics that result in incoherence
0 (0 Total Score)
A rater score of 0 is indicative of off topic (provides no evidence of an attempt to respond to the assigned task), is in a foreign language, merely copies the topic, consists of only keystroke characters or is illegible, nonverbal or blank.