Compressed Real Numbers for AI: a case-study using a RISC-V CPU
Federico Rossi Marco Cococcioni Roger Ferrer Ibàñez Jesùs Labarta Filippo Mantovani Marc Casas Emanuele Ruffaldi Sergio Saponara
As recently demonstrated, Deep Neural Networks (DNN), usually trained using
single precision IEEE 754 floating point numbers (binary32), can also work
using lower precision. Therefore, 16-bit and 8-bit compressed format have
attracted considerable attention. In this paper, we focused on two families of
formats that have already achieved interesting results in compressing binary32
numbers in machine learning applications, without sensible degradation of the
accuracy: bfloat and posit. Even if 16-bit and 8-bit bfloat/posit are routinely
used for reducing the storage of the weights/biases of trained DNNs, the
inference still often happens on the 32-bit FPU of the CPU (especially if GPUs
are not available). In this paper we propose a way to decompress a tensor of
bfloat/posits just before computations, i.e., after the compressed operands
have been loaded within the vector registers of a vector capable CPU, in order
to save bandwidth usage and increase cache efficiency. Finally, we show the
architectural parameters and considerations under which this solution is
advantageous with respect to the uncompressed one.