Assessment for Effective Intervention, Ahead of Print.
Rubric-based observations of pre- and inservice teachers are common practice in schools. Popular observation tools often result in minimal variation in ratings between teachers, require extensive training and time demands for raters, and provide minimal feedback for professional development. Alternatively, direct observation methods are evidenced to effectively measure instructional behaviors. Applying direct observation to audio recordings would produce quantitative scores and provide valuable feedback to teachers about their instruction. As such, the purpose of the present pilot study was to examine the reliability and efficiency of using audio recordings to measure practices related to explicit instruction. Fleiss’s kappa was modeled to determine the reliability of multiple raters. Regression and correlation examined the strength and direction of the relationship between the full length of a teacher’s lesson and the first 20 min of the lesson. Results indicate that using audio recordings is reliable with kappas ranging from .45 to .80. Based on regression analyses, the first 20 min of a teacher’s lesson is predictive of the rates of behaviors observed in a full lesson. Correlations suggest large, positive relationships between rates of behaviors in the first 20 min and the full lesson. Recommendations for future studies of audio-recorded observations and progress monitoring teacher behavior are discussed.