Uncertainty Ranges
Student ratings uncertainty ranges are calculated using the composite score of the responses to the five questions under the Instructor Effectiveness section—questions that were derived from literature which tells us about the instructional behaviors that are most predictive of student learning.
To the right of the composite number you will find a range set off in parentheses (e.g. 3.6-4.6). This is an “uncertainty” range calculated by taking into account sampling and measurement error. For example, if the “Section” number for the course I taught last semester is 4.1 and the range noted in the parenthesis is (3.6-4.6) I will conclude that the true measure of student perceptions of their experience in my class is not an absolute 4.1, but that it falls somewhere on a range of 3.6-4.6.
If the ranges overlap, you cannot conclude that the section performed higher or lower than the department average. Only those ranges which do not overlap (the amount of overlap does not matter) can be defined as performing above or below the department.
Interpretation of the Uncertainty Range
Remember...
- The student ratings tool was not designed to rank faculty. It was designed to group faculty in categories such as average, below average, and above average. Most faculty (statistically speaking) will be average.
- Sometimes, especially in large departments, the department range may not appear to be a range (e.g. 4.5-4.5). This is a range but is due to the large number of students in courses in that department and the fact that we round to the tenths.
- The ratings tool alerts us to the uncertainty in the ranges and reminds us of the need to use multiple measures of instructor effectiveness in addition to student ratings.
- Currently available modeling software does not allow us to produce uncertainty ranges for more levels of aggregation beyond the two we provide for the section and department levels (two more levels are used in the model calculations).
Q Which set of scores above indicates that the instructor's performance was below average? A Example 1, because the uncertainty ranges do not overlap and the section scores are below the department scores. |
Q Which set of scores above indicates that the instructor's performance was average? A Examples 2 and 3, because the uncertainty ranges overlap.The amount of overlap does not matter. In these examples we cannot conclude that the section is above the department average. |
Q Which set of scores above indicates that the instructor's performance was above average? A Example 4, because the uncertainty ranges do not overlap and the section scores are above the department scores. |
Uncertainty Range Calculation
- Section and department composite scores on the student ratings forms are reported as uncertainty ranges. We are 95% sure that the true score lies somewhere between those numbers. The formula used to calculate the uncertainty ranges is as follows:
- We use the standard error at the various levels of aggregation rather than the typical standard deviation method. Our method includes other forms of error including measurement error and sampling error.
- Refer to this document for more information about how we calculate the standard error of the mean at different levels.
