Abstract
This study aimed to demonstrate how one university worked to overcome some of the measurement problems associated with legacy student rating instruments through the creation and investigation of a new student rating instrument based on the most current scholarship on teaching and learning. Measurement problems with legacy instruments include asking about consumer satisfaction (including the use of global ratings) rather than directly assessing the quality of teaching, asking students for self-reports of learning, and asking students to make judgments about the internal state of their instructors. A new instrument was created to intentionally reduce these problems. The new instrument and its predecessor were both administered by 54 instructors in 81 classes and completed by 2,013 students. The following semester, the new instrument was administered university-wide including 1,450 instructors, 3,669 classes, and 58,320 students. The findings indicate that the new instrument created by this process is both reliable and valid but does not reflect multidimensionality. There is also no compelling evidence of bias according to gender or race. This work illustrates the process by which new and better instruments might be created and tested in order to replace flawed legacy instruments in higher education.