Another Lilly Conference session by Emily Faulconer and John Griffith (and Bobby McMasters) from Embry Riddle Aeronautical University presented results from their study comparing grades in courses taught by instructors rated average or high quality. Their introduction presented by Faulconer reviewed previous studies addressing similar questions. For their study, they analyzed 9-week terms offered in four modalities at a university with non-traditional students (50% military, average age 34, most employed). Faculty studied had to complete a course and yearly observations by the department head. Faculty quality was determined before the study, which was pre-COVID in 2019. Faulconer presented the hypotheses they sought to test, and John Griffith discussed the statistical analyses. The care taken to design the study and statistical analyses were impressive! They determined that learning mode (classroom, video synchronous classroom, video synchronous home, online) did not affect course grade. Grade distribution and pass rates were also not related (statistically) to the distance course modality. After controlling for modality, they analyzed course grades for instructors rated “excellent” or “average”. In their study, course grade and grade distribution were not related to instructor quality. The data also indicated that instructor quality did not influence pass rate. The presenters discussed possible limitations of the study including course structure (9-week term, standardized syllabus/assignments), non traditional students, and quality ratings were not based on term for data collection. Nevertheless, this study is though-provoking and made me think about the importance of careful design and statistical analyses! The presenters also talked about what to do next, including using instructor presence and feedback to measure instructor quality and analyses of different marginalized student groups. The use of some measure of instructor presence would be intriguing and useful. I wonder what instructor behaviors related to presence positively affect grades in this setting?
