AI capabilities can be significantly improved without expensive
  retraining
Tom Davidson Jean-Stanislas Denain Pablo Villalobos Guillem Bas
State-of-the-art AI systems can be significantly improved without expensive
retraining via "post-training enhancements"-techniques applied after initial
training like fine-tuning the system to use a web browser. We review recent
post-training enhancements, categorizing them into five types: tool-use,
prompting methods, scaffolding, solution selection, and data generation.
Different enhancements improve performance on different tasks, making it hard
to compare their significance. So we translate improvements from different
enhancements into a common currency, the compute-equivalent gain: how much
additional training compute would be needed to improve performance by the same
amount as the enhancement. Our non-experimental work shows that post-training
enhancements have significant benefits: most surveyed enhancements improve
benchmark performance by more than a 5x increase in training compute, some by
more than 20x. Post-training enhancements are relatively cheap to develop:
fine-tuning costs are typically <1% of the original training cost. Governing
the development of capable post-training enhancements may be challenging
because frontier models could be enhanced by a wide range of actors.