EXL2 70B for 24GB VRAM - a DeusImperator Collection

DeusImperator 's Collections

EXL2 70B for 24GB VRAM

Small models 8bpw EXL2 quants

Mid-range models exl2 quants

Exl2 8bpw MAX quants

EXL2 70B for 24GB VRAM

updated Jul 14, 2024

70B LLMs that fit in 24GB VRAM with over 16k context (with exl2 Q4 cache)

DeusImperator/Midnight-Miqu-70B-v1.5_exl2_2.4bpw_rpcal

Text Generation • Updated May 19, 2024 • 27 • 1
DeusImperator/Midnight-Miqu-70B-v1.5_exl2_2.4bpw_rpcal_mk2

Text Generation • Updated May 19, 2024 • 5
DeusImperator/Midnight-Miqu-70B-v1.5_exl2_2.4bpw

Text Generation • Updated May 19, 2024 • 8
DeusImperator/Dark-Miqu-70B_exl2_2.4bpw

Text Generation • Updated May 24, 2024 • 7
DeusImperator/Dark-Miqu-70B_exl2_2.4bpw_rpcal_mk2

Text Generation • Updated May 25, 2024 • 11
DeusImperator/Dark-Miqu-70B_exl2_2.4bpw_rpcal_long

Updated Jun 2, 2024 • 3
DeusImperator/Midnight-Miqu-70B-v1.5_exl2_2.4bpw_rpcal_long

Text Generation • Updated Jun 6, 2024 • 13
DeusImperator/Nimbus-Miqu-v0.1-70B_exl2_2.4bpw

Text Generation • Updated Jul 13, 2024 • 5
DeusImperator/Nimbus-Miqu-v0.1-70B_exl2_2.4bpw_rpcal_mk2

Text Generation • Updated Jul 14, 2024 • 5
DeusImperator/sunfall-midnight-miqu-v0.2-v1.5-70B_exl2_2.4bpw

Text Generation • Updated Jun 29, 2024 • 5
DeusImperator/sunfall-midnight-miqu-v0.2-v1.5-70B_exl2_2.4bpw_rpcal_mk2

Text Generation • Updated Jul 1, 2024 • 5