Towards Autonomous Hypothesis Verification via Language Models with
  Minimal Guidance
Shiro Takagi Ryutaro Yamauchi Wataru Kumagai
Research automation efforts usually employ AI as a tool to automate specific
tasks within the research process. To create an AI that truly conduct research
themselves, it must independently generate hypotheses, design verification
plans, and execute verification. Therefore, we investigated if an AI itself
could autonomously generate and verify hypothesis for a toy machine learning
research problem. We prompted GPT-4 to generate hypotheses and Python code for
hypothesis verification with limited methodological guidance. Our findings
suggest that, in some instances, GPT-4 can autonomously generate and validate
hypotheses without detailed guidance. While this is a promising result, we also
found that none of the verifications were flawless, and there remain
significant challenges in achieving autonomous, human-level research using only
generic instructions. These findings underscore the need for continued
exploration to develop a general and autonomous AI researcher.