HEIghten scores provide institutions with the tools for making appropriate inferences about their learners’ demonstration of learning, and to make informed changes to educational interventions.
To appropriately use data from HEIghten assessment instruments, stakeholders must understand the various types of scores made available to them, what these scores mean, and appropriate uses/interpretations of these scores.
Types of Scores
Each HEIghten module uses scaled scores to aid in the interpretation of learner performance. A scaled score is a conversion of the numerical raw score achieved on the assessment (e.g., 50 of 80 tasks/questions answered correctly) to a score in the predetermined range used for each HEIghten assessment module (from 150 to 180, or 90 to 150). Each HEIghten module has multiple test forms. The scaled scores on the HEIghten modules are computed in a way that adjusts for the potential difference in difficulty of the questions on each test form. This statistical process allows the performance of learners taking different forms of the same module to be compared, regardless of minor differences in the difficulty of those forms. While scores cannot be compared across the different modules (e.g., comparing Critical Thinking scores to Written Communication scores), scores can be compared across different forms of the same module. Additionally, overall scores on the Civic Competency and Engagement module (including Civic Competency and Civic Attitudes) and Intercultural Competency and Diversity module (including Approach and Analyze & Act) should not be combined to create a total score.
The following are the types of scores that are found on the HEIghten modules:
Total/Overall Score. This score is provided on each HEIghten assessment module as a description of the total or overall performance within a module. It is the estimated statistical representation of a learner’s skill as represented by the HEIghten assessment module content. Higher scores indicate an estimate of higher skill or attitude than lower scores. Total scores are provided for the Critical Thinking, Quantitative Literacy, and Written Communication modules, and overall scores are provided for the different dimensions of the Civic Competency and Engagement and Intercultural Competency and Diversity modules (the Civic Competency and Engagement and Intercultural Competency and Diversity modules do not report a total score). Specifically, overall scores are provided for the Civic Competency and Civic Attitudes dimensions of the Civic Competency and Engagement module, and overall scores are provided for Approach and Analyze & Act dimensions of the Intercultural Competency and Diversity module. It should also be noted that some scores in the Intercultural Competency and Diversity and Civic Competency and Engagement modules may not be accurate reflections of how individuals will respond or perform in real-world situations.
Subscores. These scores represent performance in key aspects of each skill area measured by the HEIghten assessment modules. The number of questions on each assessment module and the breadth of the subscore domains helps to determine if a reliable subscore can be reported. When subscores are available for a HEIghten assessment module, those subscores will only be provided at the group level. Individual learners will not receive subscores.
Percentile Ranks. This statistic illustrates the percentage of scores in a frequency distribution that are equal to or below a particular score. Percentile ranks will be provided for institutions and learners. This statistic ranges from 100% to 0%.
Proficiency Classifications. Also known as Performance Level Descriptions (PLDs), these criterion-referenced scores indicate an achievement of varying levels of standards for each HEIghten assessment module. The criterion score for reaching each proficiency level differs. For each HEIghten assessment the proficiency levels are noted as “Developing,” “Proficient,” and “Advanced.”
Attitude/Approach Classifications. In addition to the PLDs, both the Civic Competency and Engagement, and Intercultural Competency and Diversity modules also have classifications for the Civic Attitude and Approach Levels, respectively. For the Civic Competency and Engagement module, the Civic Attitudes subscale utilizes three Attitude Level Descriptions - “Lower,” “Medium,” and “Higher” - based on the learners’ responses to civic-related scenarios. In the case of the Intercultural Competency and Diversity module, the Approach dimension includes three Approach Level Descriptions: “High,” “Neutral,” and “Low.”
Types of Score Reports
Standard score reports are available to all HEIghten users. Examples of the Learner Score Report and Institutional Score Report are described below. Institutions will receive separate reports for each of the HEIghten modules administered to their learners (i.e., an institutional score report for Critical Thinking and a separate institutional score report for Quantitative Literacy).
Learner Score Report. This report provides an individual learner’s total score on the HEIghten module for Critical Thinking, Written Communication, and Quantitative Literacy. This information is an estimate of the amount of critical knowledge in the domain that has been demonstrated by the learner. Higher scores indicate the demonstration of more critical knowledge. Individual learner performance relative to the performance of learners across all participating institutions is indicated by the percentile rank and a comparison to the average score of all other learners. Learners and faculty members can use this report to compare this learner’s critical knowledge to the average score of other learners who were assessed during a specific period of time (e.g., 2018-2023).
The individual score reports for Civic Competency and Engagement and Intercultural Competency and Diversity do not contain a total score. Rather, these modules contain overall scores for the individual for each domain (i.e., Civic Competency and Civic Attitudes for the Civic Competency and Engagement module, and Approach and Analyze & Act for the Intercultural Competency and Diversity module). In the case of the Civic Participation domain for Civic Competency and Engagement, individuals are provided with item-level response percentages for each response.
This report also includes each of the three criterion-referenced proficiency levels (“Developing,” “Proficient,” and “Advanced”) to further describe the learner’s performance and the average performance of all learners who were assessed during a specific period of time for the Critical Thinking, Written Communication, and Quantitative Literacy total scores, and for the Civic Competency and Analyze & Act overall scores within the Civic Competency and Engagement, and Intercultural Competency and Diversity modules, respectively. The Civic Attitudes domain in the Civic Competency and Engagement module, uses “Lower,” “Medium,” and “Higher” to indicate the level of agreement to realistic situations or scenarios. In addition, the Approach domain in the Intercultural Competency and Diversity module uses “Low,” “Neutral,” and “High” level descriptors to reflect the test takers’ views of themselves.
Institutional Score Report. The institutional score report contains information about the distribution of learners’ scaled scores within the Reporting group and Comparison group. The Reporting group includes learners from your institution that completed at least 75% of the assessment. The Comparison group includes learners from other institutions who also took the assessment. This distribution of scores allows the institution to view the range, modality, and skewedness of scores. Characteristics of the Reporting and Comparison groups are described at the top of the institutional score report. Institutional stakeholders can use this information to compare their learners’ average performance to the performance of learners included in the Comparison group. Institutional stakeholders can also use this information to examine the percentage of learners achieving each possible total or overall scaled score on the module (ranging from 150–180 for Critical Thinking, Written Communication, Quantitative Literacy, Civic Competency for Civic Competency and Engagement, and Analyze & Act for Intercultural Competency and Diversity; and ranging from 90-150 for Civic Attitudes for Civic Competency and Engagement, and Approach for Intercultural Competency and Diversity). The institutional report also includes the distribution of learners within the Reporting group and the Comparison Group who achieved each performance/attitude/ approach level.
The institutional report also includes the percentage of learners at each of the three criterion-referenced proficiency levels or Performance Level Descriptions, or PLDs (“Developing,” “Proficient,” and “Advanced”), from the Reporting group and the Comparison group. There are also graphs indicating the percentage of learners at each of the three attitude or approach levels (“Lower,” “Medium,” and “Higher”; “Low,” “Neutral,”, and “High”), from the Reporting group and the Comparison group. Institutional stakeholders can use this information to determine what percentage of their learners performed at each of the three performance/attitude/approach levels. This information is provided for learners in the Comparison group as well.
Lastly, the institution report provides the average total/overall scaled scores and subscores for assessed learners for Critical Thinking, Written Communication, and Quantitative Literacy. Overall scaled scores are only provided from the domain scores for Civic Competency and Engagement and Intercultural Competency and Diversity modules.
Data Download Report. This report is a data file that provides all administration and learner performance information. This data file is provided to allow institutions the flexibility of conducting their own analyses on learner data. For added flexibility, this report is available in csv format for use with Microsoft® Excel® or other spreadsheet software. Institutions can add or modify learner information from their databases such as course tracks, standardized test scores and GPA. Faculty members can use this report to conduct further analyses of learner data. This report provides faculty with the opportunity to manage their own data files and is often used by programs that conduct research on learner learning within their discipline.
Basic Statistical Information for HEIghten Scores
|
Module |
Scores |
Subscores |
|
Critical Thinking |
Scale of 150-180 PLD*: Developing, Proficient, Advanced |
Scale of 1 -10 · Analytic ·Synthetic |
|
Quantitative Literacy Assessment |
Scale of 150-180 PLD: Developing, Proficient, Advanced |
Scale of 1-10 ·Numbers & Operations ·Algebra ·Geometry & Measurement ·Probability & Statistics |
|
Written Communication |
Scale of 150-180 PLD: Developing, Proficient, Advanced |
Scale of 1-10 ·Knowledge of Social & Rhetorical Situations ·Knowledge of Conceptual Strategies ·Knowledge of Language Use & Conventions Direct Writing Measure range 0 to 12 |
|
Intercultural Competency & Diversity |
Approach** - Scale of 90-150 ALD^: Low, Neutral, High |
Scale of 9-15 ·Positive Cultural Orientation ·Cultural Self-Efficacy |
|
Analyze & Act - Scale of 150-180 PLD: Developing, Proficient, Advanced |
Scale of 1-10 ·Self-Awareness ·Cultural Knowledge Application ·Suspending Judgment/Perspective Taking ·Social Monitoring ·Emotion Regulation ·Behavior Regulation |
|
|
Civic Competency & Engagement |
Civic Competency - Scale of 150-180 PLD: Developing, Proficient, Advanced |
Scale of 1-10 ·Civic Knowledge ·Civic Skills |
|
Civic Attitudes** - Scale of 90-150 ALD~: Lower, Medium, Higher |
|
|
|
Civic Participation |
Civic Participation by Location, Volunteering, and Student Government Frequencies of Civic-Related Activities |
*PLD=Proficiency Level Descriptions
**There may be results where a Civic Attitudes score cannot be reported because the learner did not answer a sufficient number of questions. In these instances, the learner score report will display ‘Not Reportable’ in place of the Civic Attitudes score, and the score field will be blank on the institution data download report.
^ALD = Approach Level Descriptors
~ATLD = Attitude Level Descriptors
Proficiency Level Descriptions (PLDs) by Assessment
For Critical Thinking and Quantitative Literacy, the proficiency level achieved is determined by the assessed learner’s overall score. For Written Communication, both the assessed learner’s overall score and direct writing measure score contribute to the proficiency level achieved. The selected-response and constructed-response items measure two distinct aspects of writing, with the former assessing the learner’s knowledge of writing concepts and the second requiring that knowledge be put into practice. Therefore, assessed learners with essay scores of 5 or below are classified as Developing regardless of their overall score. Similarly, assessed learners with an overall score at or below 160 are also always classified as Developing, regardless of their essay score. Assessed learners with overall scores between 161 and 171 and essay scores of 6 or above are classified as Proficient. Finally, learners with total scores of 172 or above and essay scores of 6 or above are classified as Advanced.
|
Assessment Module |
Developing |
Proficient |
Advanced |
|
Critical Thinking |
150–161 |
162–172 |
173–180 |
|
Written Communication |
150–160 |
161–171 |
172–180 |
|
Quantitative Literacy |
150–156 |
157–170 |
171–180 |
|
Civic Competency & Engagement (Civic Competency) |
150-156 |
157-172 |
173-180 |
|
Intercultural Competency & Diversity (Analyze & Act) |
150-157 |
158-174 |
175-180 |
Attitude/Approach Level Descriptors
For the Civic Attitude domain of the Civic Competency and Engagement module, and the Approach section of the Intercultural Competency and Diversity module, a theoretical and statistically-driven method was used to create the categories for the Attitude and Approach Levels.
|
Assessment Module |
Lower/ |
Medium/ Neutral |
Higher/ |
|
Civic Competency & Engagement (Civic Attitudes) |
90-111 |
112-129 |
130-150 |
|
Intercultural Competency & Diversity (Approach) |
90-104 |
105-131 |
132-150 |
Note: Lower, Medium, and Higher are the level descriptors for Civic Attitudes; Low, Neutral, and High are the level descriptors for Approach.