Symbolic and Language Agnostic Large Language Models
Walid S. Saba
We argue that the relative success of large language models (LLMs) is not a
reflection on the symbolic vs. subsymbolic debate but a reflection on employing
an appropriate strategy of bottom-up reverse engineering of language at scale.
However, due to the subsymbolic nature of these models whatever knowledge these
systems acquire about language will always be buried in millions of
microfeatures (weights) none of which is meaningful on its own. Moreover, and
due to their stochastic nature, these models will often fail in capturing
various inferential aspects that are prevalent in natural language. What we
suggest here is employing the successful bottom-up strategy in a symbolic
setting, producing symbolic, language agnostic and ontologically grounded large
language models.