Applicability of scaling laws to vision encoding models
Takuya Matsuyama Kota S Sasaki Shinji Nishimoto
In this paper, we investigated how to build a high-performance vision
encoding model to predict brain activity as part of our participation in the
Algonauts Project 2023 Challenge. The challenge provided brain activity
recorded by functional MRI (fMRI) while participants viewed images. Several
vision models with parameter sizes ranging from 86M to 4.3B were used to build
predictive models. To build highly accurate models, we focused our analysis on
two main aspects: (1) How does the sample size of the fMRI training set change
the prediction accuracy? (2) How does the prediction accuracy across the visual
cortex vary with the parameter size of the vision models? The results show that
as the sample size used during training increases, the prediction accuracy
improves according to the scaling law. Similarly, we found that as the
parameter size of the vision models increases, the prediction accuracy improves
according to the scaling law. These results suggest that increasing the sample
size of the fMRI training set and the parameter size of visual models may
contribute to more accurate visual models of the brain and lead to a better
understanding of visual neuroscience.