Learning to Guide Human Experts via Personalized Large Language Models
Debodeep Banerjee Stefano Teso Andrea Passerini
In learning to defer, a predictor identifies risky decisions and defers them
to a human expert. One key issue with this setup is that the expert may end up
over-relying on the machine's decisions, due to anchoring bias. At the same
time, whenever the machine chooses the deferral option the expert has to take
decisions entirely unassisted. As a remedy, we propose learning to guide (LTG),
an alternative framework in which -- rather than suggesting ready-made
decisions -- the machine provides guidance useful to guide decision-making, and
the human is entirely responsible for coming up with a decision. We also
introduce SLOG, an LTG implementation that leverages (a small amount of) human
supervision to convert a generic large language model into a module capable of
generating textual guidance, and present preliminary but promising results on a
medical diagnosis task.