perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	6937	10.7622	11.10	54.42	0.15
IQ1_M	7564	9.2039	12.10	63.64	0.13
IQ2_XXS	8610	7.5917	13.78	77.15	0.11
IQ2_XS	9496	7.1233	15.19	82.23	0.10
IQ2_S	9906	7.0558	15.85	83.01	0.10
IQ2_M	10742	6.7367	17.19	86.95	0.09
Q2_K_S	10955	6.9707	17.53	84.03	0.10
Q2_K	11742	6.7073	18.79	87.33	0.09
IQ3_XXS	12244	6.3673	19.59	91.99	0.09
IQ3_XS	13070	6.2584	20.91	93.59	0.08
Q3_K_S	13725	6.2372	21.96	93.91	0.08
IQ3_S	13768	6.1756	22.03	94.85	0.08
IQ3_M	14124	6.1781	22.60	94.81	0.08
Q3_K_M	15196	6.0711	24.31	96.48	0.08
Q3_K_L	16448	6.0467	26.32	96.87	0.08
IQ4_XS	16873	5.9536	27.00	98.38	0.08
IQ4_NL	17816	5.9556	28.51	98.35	0.08
Q4_0	17844	5.9980	28.55	97.65	0.08
Q4_K_S	17914	5.9378	28.66	98.64	0.08
Q4_K_M	18931	5.9273	30.29	98.82	0.08
Q4_1	19683	5.9349	31.49	98.69	0.08
Q5_K_S	21589	5.8895	34.54	99.45	0.08
Q5_0	21657	5.9057	34.65	99.18	0.08
Q5_K_M	22184	5.8872	35.49	99.49	0.08
Q5_1	23495	5.8896	37.59	99.45	0.08
Q6_K	25640	5.8667	41.02	99.84	0.08
Q8_0	33207	5.8646	53.13	99.88	0.08
F16	62499	5.8574	100	100	0.08