Model evaluation for dynamic quantized models

#52
by shkashya - opened

Are there any benchmarks (specifically - AIME 2024, MATH-500, GPQA Diamond, LiveCodeBench, CodeForces etc.) for the quantized models (UD-IQ1_S, UD-IQ1_M, UD-IQ2_XXS, UD-Q2_K_XL) comparing it to the base DeepSeek R1 BF16 model? It would be great to see these added!
!

Sign up or log in to comment