Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For
  Scoring and Providing Actionable Insights on Classroom Instruction
Rose E. Wang Dorottya Demszky
Coaching, which involves classroom observation and expert feedback, is a
widespread and fundamental part of teacher training. However, the majority of
teachers do not have access to consistent, high quality coaching due to limited
resources and access to expertise. We explore whether generative AI could
become a cost-effective complement to expert feedback by serving as an
automated teacher coach. In doing so, we propose three teacher coaching tasks
for generative AI: (A) scoring transcript segments based on classroom
observation instruments, (B) identifying highlights and missed opportunities
for good instructional strategies, and (C) providing actionable suggestions for
eliciting more student reasoning. We recruit expert math teachers to evaluate
the zero-shot performance of ChatGPT on each of these tasks for elementary math
classroom transcripts. Our results reveal that ChatGPT generates responses that
are relevant to improving instruction, but they are often not novel or
insightful. For example, 82% of the model's suggestions point to places in the
transcript where the teacher is already implementing that suggestion. Our work
highlights the challenges of producing insightful, novel and truthful feedback
for teachers while paving the way for future research to address these
obstacles and improve the capacity of generative AI to coach teachers.