Robust Active Measuring under Model Uncertainty
Merlijn Krale Thiago D. Simão Jana Tumova Nils Jansen
Partial observability and uncertainty are common problems in sequential
decision-making that particularly impede the use of formal models such as
Markov decision processes (MDPs). However, in practice, agents may be able to
employ costly sensors to measure their environment and resolve partial
observability by gathering information. Moreover, imprecise transition
functions can capture model uncertainty. We combine these concepts and extend
MDPs to robust active-measuring MDPs (RAM-MDPs). We present an active-measure
heuristic to solve RAM-MDPs efficiently and show that model uncertainty can,
counterintuitively, let agents take fewer measurements. We propose a method to
counteract this behavior while only incurring a bounded additional cost. We
empirically compare our methods to several baselines and show their superior
scalability and performance.