ATHENA: Mathematical Reasoning with Thought Expansion
JB. Kim Hazel Kim Joonghyuk Hahn Yo-Sub Han
Solving math word problems depends on how to articulate the problems, the
lens through which models view human linguistic expressions. Real-world
settings count on such a method even more due to the diverse practices of the
same mathematical operations. Earlier works constrain available thinking
processes by limited prediction strategies without considering their
significance in acquiring mathematical knowledge. We introduce Attention-based
THought Expansion Network Architecture (ATHENA) to tackle the challenges of
real-world practices by mimicking human thought expansion mechanisms in the
form of neural network propagation. A thought expansion recurrently generates
the candidates carrying the thoughts of possible math expressions driven from
the previous step and yields reasonable thoughts by selecting the valid
pathways to the goal. Our experiments show that ATHENA achieves a new
state-of-the-art stage toward the ideal model that is compelling in variant
questions even when the informativeness in training examples is restricted.