Allowing humans to interactively train artificial agents to understand language instructions is desirable for both practical and scientific reasons.   Though, given  the lack of sample efficiency in current learning methods, reaching this goal may require substantial research efforts. We introduce the BabyAI research platform, with the goal of supporting investigations towards including humans in the loop for grounded language learning. The BabyAI platform comprises an extensible suite of 19 levels of increasing difficulty. Each level gradually leads the agent towards acquiring a combinatorially rich synthetic language, which is a proper subset of English. The platform also provides a hand-crafted bot agent, which simulates a human teacher.   We report estimated amount of supervision required for training neural reinforcement and behavioral-cloning agents on some BabyAI levels. We put forward strong evidence that current deep learning methods are not yet sufficiently sample-efficient in the context of learning a language with compositional properties. How can a human train an intelligent agent to understand natural language instructions? We believe that this research question is important from both technological and scientific perspectives. No matter how advanced AI technology becomes, human users will likely want to customize their intelligent helpers to better understand their desires and needs. On the other hand, developmental psychology, cognitive science and linguistics study similar questions but applied to human children, and a synergy is possible between research in grounded language learning by computers and research in human language acquisition.In this work, we present the BabyAI research platform, whose purpose is to facilitate research on grounded language learning. In our platform we substitute a simulated human expert for a real human; yet our aspiration is that BabyAI-based studies enable substantial progress towards putting an actual human in the loop. The current domain of BabyAI is a 2D gridworld in which synthetic natural-looking instructions (e.g. "put the red ball next to the box on your left") require the agent to navigate the world (including unlocking doors) and move objects to specified locations. BabyAI improves upon similar prior setups BID16 BID7 BID41 by supporting simulation of certain essential aspects of the future human in the loop agent training: curriculum learning and interactive teaching. The usefulness of curriculum learning for training machine learning models has been demonstrated numerous times in the literature BID6 BID23 BID42 BID15 , and we believe that gradually increasing the difficulty of the task will likely be essential for achieving efficient humanmachine teaching, as in the case of human-human teaching. To facilitate curriculum learning studies, BabyAI currently features 19 levels in which the difficulty of the environment configuration and the complexity of the instruction language are gradually increased. Interactive teaching, i.e. teaching differently based on what the learner can currently achieve, is another key capability of human teachers. Many advanced agent training methods, including DAGGER BID28 , TAMER BID35 and learning from human preferences BID38 BID11 , assume that interaction between the learner and the teacher is possible. To support interactive experiments, BabyAI provides a bot agent that can be used to generate new demonstrations on the fly and advise the learner on how to continue acting.Arguably, the main obstacle to language learning with a human in the loop is the amount of data (and thus human-machine interactions) that would be required. Deep learning methods that are used in the context of imitation learning or reinforcement learning paradigms have been shown to be very effective in both simulated language learning settings BID25 BID16 and applications BID32 BID3 BID39 . These methods, however, require enormous amounts of data, either in terms of millions of reward function queries or hundreds of thousands of demonstrations. To show how our BabyAI platform can be used for sample efficiency research, we perform several case studies. In particular, we estimate the number of demonstrations/episodes required to solve several levels with imitation and reinforcement learning baselines. As a first step towards improving sample efficiency, we additionally investigate to which extent pretraining and interactive imitation learning can improve sample efficiency.The concrete contributions of this paper are two-fold. First, we contribute the BabyAI research platform for learning to perform language instructions with a simulated human in the loop. The platform already contains 19 levels and can easily be extended. Second, we establish baseline results for all levels and report sample efficiency results for a number of learning approaches. The platform and pretrained models are available online. We hope that BabyAI will spur further research towards improving sample efficiency of grounded language learning, ultimately allowing human-in-the-loop training. We present the BabyAI research platform to study language learning with a human in the loop. The platform includes 19 levels of increasing difficulty, based on a decomposition of tasks into a set of basic competencies. Solving the levels requires understanding the Baby Language, a subset of English with a formally defined grammar which exhibits compositional properties. The language is minimalistic and the levels seem simple, but empirically we have found them quite challenging to solve. The platform is open source and extensible, meaning new levels and language concepts can be integrated easily.The results in Section 4 suggest that current imitation learning and reinforcement learning methods scale and generalize poorly when it comes to learning tasks with a compositional structure. Hundreds of thousands of demonstrations are needed to learn tasks which seem trivial by human standards. Methods such as curriculum learning and interactive learning can provide measurable improvements in terms of sample efficiency, but, in order for learning with an actual human in the loop to become realistic, an improvement of at least three orders of magnitude is required.An obvious direction of future research to find strategies to improve sample efficiency of language learning. Tackling this challenge will likely require new models and new teaching methods. Approaches that involve an explicit notion of modularity and subroutines, such as Neural Module Networks BID1 or Neural Programmer-Interpreters BID27 , seem like a promising direction. It is our hope that the BabyAI platform can serve as a challenge and a benchmark for the sample efficiency of language learning for years to come.
We present the BabyAI platform for studying data efficiency of language learning with a human in the loop Presents a research platform with a bot in the loop for learning to execute language instructions in which language has compositional structures Introduces a platform for grounded language learning that replaces any human in the loop with a heuristic teacher and uses a synthetic language mapped to a 2D grid world