I have asked this question on Stack Overflow, I am asking it here in the hope to get more traction.

The relative rounding error for a floating point number x is defined as

$e_r = |\frac{(round(x) - x)}{x}| = |\frac{round(x)}{x} - 1|$ (1)

Assuming that the rounding to nearest mode is used for $round(x)$, the absolute rounding error $|round(x) - x|$ is going to be less than 0.5 ulp(E(x)), where the ulp are units in the last place

$ulp(E) = 2^E \cdot \epsilon$

and $E(x)$ is the exponent used for $x$, and $\epsilon$ is the machine epsilon $\epsilon=2^{-(p-1)}$, $p$ is precision (24 for the single precision and 53 for the double precision IEEE formats).

Using this, the relative error can be expressed for any real number $x$

$e_r = |\frac{(round(x) - x)}{x}| = \frac{|(round(x) - x)|}{|x|} < |0.5 \cdot 2^E \cdot 2^{-(p-1)}| / |2^E| < 0.5 \epsilon$

For denormalized numbers $0 < x < 2^Em \epsilon$, where Em is the minimal exponent (-126 for single precision, -1022 for double):

$0 < x \le 0.5 \cdot \epsilon \cdot 2^{Em}$

the rounding always goes to $0$!

If the round(x) is 0, then by (1)

$e_r =|\frac{(0 - 1)}{1}| = |1|$ !

How is the relative error computed for such numbers? Should the relative error be even used for the numbers that are rounded to 0?


If x = 0 and is rounded to 0 then the error is 0, and the relative error is whatever you want it to be.

If x ≠ 0 and is rounded to 0, then the absolute error is x, and the relative error is 1 (or 100% if that is more intuitive).

If x = 0 and is round to y ≠ 0 then the absolute error is y, and the relative error is infinite or undefined, whatever you prefer.

  • $\begingroup$ '0' is represented exactly by the IEEE floating point standard, so no rounding will take place. For $x\ne0$, if the relative error is really $1$ when $x<$ of the minimal denormalized number, then it is a special case that has to be handled in my program. $\endgroup$
    – tmaric
    Oct 23 '18 at 16:44
  • $\begingroup$ @tmaric That's for a single operation. A sequence of operations can have an exact result of zero and a rounded result != 0. $\endgroup$
    – gnasher729
    Sep 13 '20 at 8:54

Why answer would be preferable to 1?

Let's try abstracting a little: Say you have any small value $c$ for which 0 is the closest value that can be reliably represented. If rounding occurs, how much error occurs as a fraction of $c$? All of it. 100% of $c$. So the rounding error is 100% = 1.0.


Your Answer

