Evaluating Brain-Inspired Modular Training in Automated Circuit
  Discovery for Mechanistic Interpretability
Jatin Nainani
Large Language Models (LLMs) have experienced a rapid rise in AI, changing a
wide range of applications with their advanced capabilities. As these models
become increasingly integral to decision-making, the need for thorough
interpretability has never been more critical. Mechanistic Interpretability
offers a pathway to this understanding by identifying and analyzing specific
sub-networks or 'circuits' within these complex systems. A crucial aspect of
this approach is Automated Circuit Discovery, which facilitates the study of
large models like GPT4 or LLAMA in a feasible manner. In this context, our
research evaluates a recent method, Brain-Inspired Modular Training (BIMT),
designed to enhance the interpretability of neural networks. We demonstrate how
BIMT significantly improves the efficiency and quality of Automated Circuit
Discovery, overcoming the limitations of manual methods. Our comparative
analysis further reveals that BIMT outperforms existing models in terms of
circuit quality, discovery time, and sparsity. Additionally, we provide a
comprehensive computational analysis of BIMT, including aspects such as
training duration, memory allocation requirements, and inference speed. This
study advances the larger objective of creating trustworthy and transparent AI
systems in addition to demonstrating how well BIMT works to make neural
networks easier to understand.