PHOENIX: Open-Source Language Adaption for Direct Preference
  Optimization
Matthias Uhlig Sigurd Schacht Sudarshan Kamath Barkur
Large language models have gained immense importance in recent years and have
demonstrated outstanding results in solving various tasks. However, despite
these achievements, many questions remain unanswered in the context of large
language models. Besides the optimal use of the models for inference and the
alignment of the results to the desired specifications, the transfer of models
to other languages is still an underdeveloped area of research. The recent
publication of models such as Llama-2 and Zephyr has provided new insights into
architectural improvements and the use of human feedback. However, insights
into adapting these techniques to other languages remain scarce. In this paper,
we build on latest improvements and apply the Direct Preference
Optimization(DPO) approach to the German language. The model is available at
https://huggingface.co/DRXD1000/Phoenix.