Neural Network Models of Becoming a Cardinal Principle Knower
Vima Gupta Sashank Varma
As children enter elementary school, their understanding of the ordinal
structure of numbers transitions from a memorized count list of the first
50-100 numbers to knowing the successor function and understanding the
countably infinite. We investigate this developmental change in two neural
network models that learn the successor function on the pairs (N, N+1) for N in
(0, 98). The first uses a one-hot encoding of the input and output values and
corresponds to children memorizing a count list, while the second model uses a
place-value encoding and corresponds to children learning the language rules
for naming numbers. The place-value model showed a predicted drop in
representational similarity across tens boundaries. Counting across a tens
boundary can be understood as a vector operation in 2D space, where the numbers
with the same tens place are organized in a linearly separable manner, whereas
those with the same ones place are grouped together. A curriculum learning
simulation shows that, in the expanding numerical environment of the developing
child, representations of smaller numbers continue to be sharpened even as
larger numbers begin to be learned. These models set the stage for future work
using recurrent architectures to move beyond learning the successor function to
simulating the counting process more generally, and point towards a deeper
understanding of what it means to understand the countably infinite.