Inductive-bias Learning: Generating Code Models with Large Language
  Model
Toma Tanaka Naofumi Emoto Tsukasa Yumibayashi
Large Language Models(LLMs) have been attracting attention due to a ability
called in-context learning(ICL). ICL, without updating the parameters of a LLM,
it is possible to achieve highly accurate inference based on rules ``in the
context'' by merely inputting a training data into the prompt. Although ICL is
a developing field with many unanswered questions, LLMs themselves serves as a
inference model, seemingly realizing inference without explicitly indicate
``inductive bias''. On the other hand, a code generation is also a highlighted
application of LLMs. The accuracy of code generation has dramatically improved,
enabling even non-engineers to generate code to perform the desired tasks by
crafting appropriate prompts. In this paper, we propose a novel ``learning''
method called an ``Inductive-Bias Learning (IBL)'', which combines the
techniques of ICL and code generation. An idea of IBL is straightforward. Like
ICL, IBL inputs a training data into the prompt and outputs a code with a
necessary structure for inference (we referred to as ``Code Model'') from a
``contextual understanding''. Despite being a seemingly simple approach, IBL
encompasses both a ``property of inference without explicit inductive bias''
inherent in ICL and a ``readability and explainability'' of the code
generation. Surprisingly, generated Code Models have been found to achieve
predictive accuracy comparable to, and in some cases surpassing, ICL and
representative machine learning models. Our IBL code is open source:
https://github.com/fuyu-quant/IBLM