Create README.md

README.md

@@ -0,0 +1,23 @@
+---
+tags:
+- llama
+---
+Five different instruction-tuned models (which I'm sure are intuitively obvious from the name) merged using the methodology described in [Resolving Interference When Merging Models](https://arxiv.org/abs/2306.01708).
+
+In theory this should retain more of the capabilites of the constituent models than a straight linear merge would. In my testing, it feels quite capable.
+
+Base model used for the merge: [TheBloke/Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16)
+
+Models merged in:
+* [OpenOrca-Platypus2](https://huggingface.co/Open-Orca/OpenOrca-Platypus2)
+* [limarp-13b-merged](https://huggingface.co/Oniichat/limarp-13b-merged)
+* [Nous-Hermes-Llama2-13b](https://huggingface.co/NousResearch/Nous-Hermes-Llama2-13b)
+* [chronos-13b-v2](https://huggingface.co/elinas/chronos-13b-v2)
+* [airoboros-l2-13b-gpt4-1.4.1](https://huggingface.co/jondurbin/airoboros-l2-13b-gpt4-1.4.1)
+
+The script I used to perform the merge is available [here](https://github.com/cg123/ties-merge).
+
+The command that produced this model:
+```
+python ties_merge.py TheBloke/Llama-2-13B-fp16 ./Chronorctypus-Limarobormes-13b --merge elinas/chronos-13b-v2 --merge Open-Orca/OpenOrca-Platypus2-13B --merge Oniichat/limarp-13b-merged --merge jondurbin/airoboros-l2-13b-gpt4-1.4.1 --merge NousResearch/Nous-Hermes-Llama2-13b --cuda
+```