CTranslate2 int8 version of L3-8B-Stheno-v3.1

This is a int8_bfloat16 quantization of L3-8B-Stheno-v3.1
See more on CTranslate2: Docs | Github

This model was converted to ct2 format using the following commnd:

ct2-transformers-converter --model Sao10K/L3-8B-Stheno-v3.1 --output_dir L3-8B-Stheno-v3.1-ct2 --quantization int8_bfloat16 --low_cpu_mem_usage

no converstion needed using the model from this repository as it is already in ct2 format.

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The HF Inference API does not support text-generation models for CTranslate2 library.

Model tree for Anthonyg5005/L3-8B-Stheno-v3.1-int8-ct2

Base model

Quantized

(6)

this model