---
base_model:
- microsoft/Phi-3.5-mini-instruct
- meditsolutions/MedIT-Mesh-3B-Instruct
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
---
### TriomphantV2-ECE-78B-SLERP-V1
 
This model has been produced by:
- **ROBERGE Marial**, engineering student at French Engineering School ECE
- **ESCRIVA Mathis**, engineering student at French Engineering School ECE
- **LALAIN Youri**, engineering student at French Engineering School ECE
- **RAGE LILIAN**, engineering student at French Engineering School ECE
- **HUVELLE Baptiste**, engineering student at French Engineering School ECE
 
Under the supervision of:
- **Andre-Louis Rochet**, Lecturer at ECE & Co-Founder of TW3 Partners
- **Paul Lemaistre**, CTO of TW3 Partners
 
With the contribution of:
- **ECE engineering school** as sponsor and financial contributor
- **François STEPHAN** as director of ECE
- **Gérard REUS** as acting director of iLAB
- **Matthieu JOLLARD** ECE Alumni
- **Louis GARCIA** ECE Alumni

### Models Merged

The following models were included in the merge:
* [microsoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)
* [meditsolutions/MedIT-Mesh-3B-Instruct](https://huggingface.co/meditsolutions/MedIT-Mesh-3B-Instruct)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
slices:
  - sources:
      - model: meditsolutions/MedIT-Mesh-3B-Instruct
        layer_range: [0, 32]
      - model: microsoft/Phi-3.5-mini-instruct
        layer_range: [0, 32]    
merge_method: slerp
base_model: meditsolutions/MedIT-Mesh-3B-Instruct
parameters:
  t:
    - filter: self_attn
      value: [0, 0.25, 0.5, 0.75, 1]
    - filter: mlp
      value: [1, 0.75, 0.5, 0.25, 0]
    - value: 0.5
dtype: bfloat16

```