Generating Language Corrections for Teaching Physical Control Tasks
Megha Srivastava Noah Goodman Dorsa Sadigh
AI assistance continues to help advance applications in education, from
language learning to intelligent tutoring systems, yet current methods for
providing students feedback are still quite limited. Most automatic feedback
systems either provide binary correctness feedback, which may not help a
student understand how to improve, or require hand-coding feedback templates,
which may not generalize to new domains. This can be particularly challenging
for physical control tasks, where the rich diversity in student behavior and
specialized domains make it challenging to leverage general-purpose assistive
tools for providing feedback. We design and build CORGI, a model trained to
generate language corrections for physical control tasks, such as learning to
ride a bike. CORGI takes in as input a pair of student and expert trajectories,
and then generates natural language corrections to help the student improve. We
collect and train CORGI over data from three diverse physical control tasks
(drawing, steering, and joint movement). Through both automatic and human
evaluations, we show that CORGI can (i) generate valid feedback for novel
student trajectories, (ii) outperform baselines on domains with novel control
dynamics, and (iii) improve student learning in an interactive drawing task.