NeuralThink: Algorithm Synthesis that Extrapolates in General Tasks
Bernardo Esteves Miguel Vasco Francisco S. Melo
While machine learning methods excel at pattern recognition, they struggle
with complex reasoning tasks in a scalable, algorithmic manner. Recent Deep
Thinking methods show promise in learning algorithms that extrapolate: learning
in smaller environments and executing the learned algorithm in larger
environments. However, these works are limited to symmetrical tasks, where the
input and output dimensionalities are the same. To address this gap, we propose
NeuralThink, a new recurrent architecture that can consistently extrapolate to
both symmetrical and asymmetrical tasks, where the dimensionality of the input
and output are different. We contribute with a novel benchmark of asymmetrical
tasks for extrapolation. We show that NeuralThink consistently outperforms the
prior state-of-the-art Deep Thinking architectures, in regards to stable
extrapolation to large observations from smaller training sizes.