Construction of a Japanese Financial Benchmark for Large Language Models
Masanori Hirano
With the recent development of large language models (LLMs), models that
focus on certain domains and languages have been discussed for their necessity.
There is also a growing need for benchmarks to evaluate the performance of
current LLMs in each domain. Therefore, in this study, we constructed a
benchmark comprising multiple tasks specific to the Japanese and financial
domains and performed benchmark measurements on some models. Consequently, we
confirmed that GPT-4 is currently outstanding, and that the constructed
benchmarks function effectively. According to our analysis, our benchmark can
differentiate benchmark scores among models in all performance ranges by
combining tasks with different difficulties.