Click here to Skip to main content
Click here to Skip to main content
Go to top

Floating Point in .NET part 1: Concepts and Formats

, 30 May 2005
Rate this:
Please Sign up or sign in to vote.
Introducing the basic concepts of floating-point arithmetic: number formats, accuracy and precision, and round-off error. Includes a discussion of the .NET floating-point types.


Floating-point arithmetic is generally considered a rather occult topic. Floating-point numbers are somewhat fuzzy things whose exact values are clouded in ever growing mystery with every significant digit that is added. This attitude is somewhat surprising given the wide range of every-day applications that don't simply use floating-point arithmetic, but depend on it.

Our aim in this three part series is to remove some of the mystery surrounding floating-point math, to show why it is important for most programmers to know about it, and to show how you can use it effectively when programming for the .NET platform. In this first part, we will cover some basic concepts of numerical computing: number formats, accuracy and precision, and round-off error. We will also cover the .NET floating-point types in some depth. The second part will list some common numerical pitfalls and we'll show you how to avoid them. In the third and final part, we will see how Microsoft handled the subject in the Common Language Runtime and the .NET Base Class Library.



Here's a quick quiz. What is printed when the following piece of code runs? We calculate one divided by 103 in both single and double precision. We then multiply by 103 again, and compare the result to the value we started out with:

Console.WriteLine("((double)(1/103.0))*103 < 1 is {0}.", ((double)(1/103.0))*103 < 1);
Console.WriteLine("((float)(1/103.0F))*103 > 1 is {0}.", ((float)(1/103.0F))*103 > 1);

In exact arithmetic, the left-hand sides of the comparison are equal to 1, and so the answer would be false in both cases. In actual fact, true is printed twice. Not only that, we get results that don't match what we would expect mathematically. Two alternative ways of performing the exact same calculation give totally contradictory results!


float price = 4.99F;
int quantity = 17;
float total = price * quantity;


int fullPrice = 100;
float discount = 0.1F;
Int32 finalPrice = (int)(fullPrice * (1-discount));


There are other variations on the same theme. Mathematical equalities don't seem to hold. Calculations don't seem to conform to what we learnt in grade three. It all looks fuzzy and confusing. You can be assured, however, that underneath it all are solid and exact mathematical computations. The aim of this article is to expose the underlying math, so you can once again go out and multiply, add, and divide with full confidence.

Some terminology

Before we do anything, we should define the words that are commonly used in numerical computing.

Number Formats

A computer program is a model of something in the real world. Many things in the real world are represented by numbers. Those numbers need a representation in our computer program. This is where number formats come in.

From a programmer's point of view, a number format is a collection of numbers. 99.9% of the time, the binary or internal representation is not important. It may be important when we represent non-numeric data, as with bit fields, but that doesn't concern us here. What counts is only that it can represent the numbers from the real world objects we are modeling. Some number formats include certain special values to indicate invalid values or values that are outside the range of the number format.


Most numbers are integers, which are easy to represent. Almost any integer you'll encounter will fit into a "32-bit signed integer", which is a number in the range -2,147,483,648 to 2,147,483,647. For some applications, like counting the number of people in the world, you need the next wider format: 64 bit integers. Its range is wide enough to count every 10th of a microsecond over many millennia. (This is how a DateTime value is represented internally.)

Many other numbers, like measurements, prices and percentages, are real numbers with digits after the decimal point. There are essentially two ways to represent real numbers: fixed point and floating point.

Fixed-point formats


Fixed point works well for many applications. For financial calculations, it has the added benefit that numbers such as 0.1 and 0.01 can be represented exactly with a suitable choice of multiplier. However, it is not suited for many other applications where a greater range is needed. Particle physicists commonly use numbers smaller than 10-20, while cosmologists estimate the number of particles in the observable universe at around 1085. It would be impractical to represent numbers in this range in fixed point format. To cover the whole range, a single number would take up at least 50 bytes!

Floating-point formats

This problem is solved with a floating point format. Floating-point numbers have a variable scale factor, which is specified as the exponent of a power of a small number called the base, which is usually 2 or 10. The .NET framework defines three floating-point types: Single, Double and Decimal. That's right: the Decimal type does not use the fixed point format of the Currency or Money type. It uses a decimal floating-point format.


Before we go into the details of real life formats, we need to define some more terms.

Range, Precision and Accuracy

The range of a number format is the interval from the smallest number in the format to the largest. The range of 16-bit signed integers is -32768 to 32767. The range of double-precision floating-point numbers is (roughly) -1e+308 to 1e+308. Numbers outside a format's range cannot be represented directly. Numbers within the range may not exist in the number format - infinitely many don't. But at least there is always a number in the format that is fairly close to our number.



Precision can be absolute or relative. Integer types have an absolute precision of 1. Every integer within the type's range is represented. Fixed point types, like the Currency type in earlier versions of Visual Basic, also have an absolute precision. For the Currency type, it is 0.0001, which means that every multiple of 0.0001 within the type's range is represented.


Precision is also a property of a calculation. Here, it refers to the number of digits used in the calculation, and in particular also the precision used for intermediate results. As an example, we calculate a simple expression with one and two digit precision:

Using one digit precision:
= 0.2 + 0.2 Round to 1 digit
= 0.4 Final result
Using two digit precision:
= 0.24 + 0.24 Keep the 2 digits
= 0.48 Calculate sum
= 0.5 Round to 1 digit

Comparing to the exact result (0.48), we see that using 1 digit precision gives a result that is off by 0.08, while using two digit precision gives a result that is off by only 0.02. One lesson learnt from this example is that it is useful to use extra precision for intermediate calculations if that option is available.




Byte   n0 = 0x03;
Int16  n1 = 0x0003;
Int32  n2 = 0x00000003;
Single n3 = 3.000000f;
Double n4 = 3.000000000000000;



Round-off error




Standards for Floating-Point Arithmetic

Back in the '60s and '70s, computers were still very new and expensive. Every manufacturer had its own processor technology, with its own numerical formats, its own rules for handling overflows and divide-by-zeros, its own rounding rules. The situation can best be described as total anarchy.


The IEEE-754 standard later became known as "IEC 60559:1989, Binary floating-point arithmetic for microprocessors". This is the official reference standard used by Microsoft in all its current specifications.

The IEC 60559 standard does more than define number formats. It sets guidelines for many aspects of floating-point arithmetic. Many of these guidelines are mandatory. Some are optional. We will see in part three of this series that Microsoft's implementation of the standard in the Common Language Runtime is incomplete. For now, we will focus on the two number formats supported by the CLR: single and double precision floating-point numbers. We will also touch on the 'extended' format, for which the IEC 60559 standard defines minimum specifications, and which is used by the floating-point unit on Intel processors, and is also used internally by the CLR.

Anatomy of the single-precision floating-point format

We will now look at the details of the single and double precision formats. Although you can get by without knowing the internals, this information can help to understand some of the nuances of working with floating-point numbers. In order to keep the discussion clear, we will at first only consider the single precision format. The double precision format is similar, and will be summarized later.

Normalized numbers


part sign exponent fraction
bit # 31 23-30 0-22

The sign s is stored in the most significant bit. A value of 0 indicates a positive value, while 1 indicates a negative value.

The exponent field is an 8 bit unsigned integer called the biased exponent. It is equal to the exponent e plus a constant called the bias which has a value of 127 for single-precision numbers. This means that, for example, an exponent of -44 is stored as -44+127= 83 or 01010011. There are two reserved exponent values: 0 and 255. The reason for this will be explained shortly. As a result, the smallest actual exponent is -126, and the largest is +127.

The number format appears to be ambiguous: You can multiply m by 2 and subtract 1 from e and get the same number. This ambiguity is resolved by minimizing the exponent and maximizing the size of the significand. This process is called normalization. As a result, the significand m always has 24 bits, with the leading bit always equal to 1. Since we know it is always equal to 1, we don't have to store this bit, and so we end up with the significand taking up only 23 bits instead of 24.


Zero and subnormal numbers




When both the biased exponent and the significand are zero, the resulting value is equal to 0. Changing the sign of zero does not change its value, so we have two possible representations of zero: one with a positive sign, and one with a negative sign. As it turns out, it is meaningful to have a 'negative zero' value. Although its value equals the value of normal 'positive zero', it behaves differently in some situations, which we will get into shortly.

Infinities and Not-a-Number


If the biased exponent is all 1's (i.e. equal to 255) and the significand is all 0's, then the number represents infinity. The sign bit indicates whether we're dealing with positive or negative infinity. These numbers are returned for operations that either do not have a finite value (e.g. 1/0) or are too large to be represented by a normalized number (e.g. 21,000,000,000).





The significand bits of a NaN can be set to an arbitrary value, sometimes called the payload. The IEC 60559 standard specifies that the payload should propagate through calculations. For example, when a NaN is added to a normal number, say 5.3, then the result is a NaN with the same payload as the first operand. When both operands are NaNs, then the resulting NaN carries the payload of either one of the operands. This leaves the possibility to pass on potentially useful information in NaN values. Unfortunately, this feature is hardly ever used.

Some examples

Let's look at some numbers and their corresponding bit patterns.

Number Sign Exponent Fraction
0 0 00000000 00000000000000000000000
-0 1 00000000 00000000000000000000000
1 0 01111111 00000000000000000000000
+Infinity 0 11111111 00000000000000000000000
NaN 1 11111111 10000000000000000000000
3.141593 0 10000000 10010010000111111011100
-3.141593 1 10000000 10010010000111111011100
100000 0 10001111 10000110101000000000000
0.000001 0 01101110 01001111100010110101100
1/3 0 01111101 01010101010101010101011
4/3 0 01111111 01010101010101010101011
2-144 0 00000000 00000000000000000100000

Notice the exponent field for 1 and 4/3. Both these numbers are between 1 and 2, and so their unbiased exponent is zero. The biased exponent is therefore equal to the bias, which is 127, or 1111111 in decimal. Numbers larger than 2 have biased exponents greater than 127. Numbers smaller than 1 have biased exponents smaller than 127.

The last number in the table (2-144) is denormalized. The biased exponent is zero, and since 2-144 = 32*2-149 the fraction is 32 = 25.

Double and extended precision formats

Double-precision floating-point numbers are stored in a way that is completely analogous to the single-precision format. Some of the constants are different. The sign still takes up 1 bit - no surprise there. The biased exponent takes up 11 bits, with a bias value of 1023. The significand takes up 52 bits with the 53rd bit implicitly set to 1 for normalized numbers.



Format Single Double Extended
Length (bits) 32 64 80
Exponent bits 8 11 15
Exponent bias 127 1023 16383
Smallest exponent -126 -1022 -16382
Largest exponent +127 +1023 +16383
Precision 24 53 64
Smallest positive value 1.4012985e-45 2.4703282292062327e-324 1.82259976594123730126e-4951
Largest positive value 3.4028235e+38 1.7976931348623157e+308 1.18973149535723176502e+4932

What about the decimal format?



Other parts of the standard



  1. towards the nearest number (round up or down, whichever produces the smaller error)
  2. towards zero (round down for positive numbers, and up for negative numbers)
  3. towards +infinity (always round up)
  4. towards -infinity (always round down)



Exception Situation Return value


Exceptions are not fatal errors. They act similar to integer overflows in the CLR. By default, no action is taken in case of overflow. However in a checked context, an exception is thrown when integer overflow occurs. Similarly, the IEEE-754/IEC 60559 defines a trap mechanism that passes control over to a trap handler when an exception occurs.

Finally, some real code



Function Description
Logb(x) Returns the unbiased exponent of x.


Converting to and from binary representations


For double values, the BitConverter class contains two useful methods: DoubleToInt64Bits and Int64BitsToDouble. As the names suggest, these methods convert a double to and from a 64-bit integer. There is no equivalent for Single values. Fortunately, one line of unsafe code will do the trick.

Finding the next representable neighbor

Finding the next representable neighbor of a floating-point number, which is the purpose of the NextAfter method, appears to be a rather complicated operation. We have to deal with positive and negative, normalized and denormalized numbers, exponents and significands, as well as zeros, infinities, and NaNs!

Fortunately, a special property of the floating-point formats comes to the rescue: the values are ordered like sign-magnitude integers. What this means is that, setting aside the sign bit for the moment, the order of the floating-point numbers and their binary representation is the same. So, all we have to do to find the next neighbor is to increment or decrement the binary representation. There are a few special cases, but all the handling of exponents is taken care of.


In this first article in a three-part series, we introduced the basic concepts of numerical computing: number formats, accuracy, precision, range, and round-off error. We described the most common number formats (single, double and extended precision) and the standard that defines them. Finally, we wrote some code to implement some floating-point functions.

The next part in this series will be of much more direct practical value. We will look more in depth at the dangers that come with doing calculations with floating-point numbers, and we'll show you how you can avoid them.




A list of licenses authors might use can be found here


About the Author

Jeffrey Sax
Founder Extreme Optimization
Canada Canada
Jeffrey has been writing numerical software for many years. He is founder and president of Extreme Optimization, a Toronto based provider of numerical component libraries for the .NET framework. He loves challenges, especially when it comes to making code run fast, and finding simplicity and elegance in what looks like complicated chaos.

Comments and Discussions

GeneralInspecting layout of floating point types PinmemberAndrew Phillips27-May-08 3:53 
QuestionParts II and III Pinmembermadval30-Apr-07 7:55 
GeneralHexa format Pinmemberthanhntcdit13-Jul-06 4:19 
AnswerRe: Hexa format PinmemberJeffrey Sax13-Jul-06 13:07 
GeneralHigh-frequency updating Pinmemberjsiegendorf31-May-06 11:13 
This was very informative article, but I still have some open questions.
I am storing financial data (stock prices, profit and loss figures, etc..) in a SQL 2005 database. The database will be updated with a high frequency. I am debating whether to use decimal or double/float to store these values and I am concerned that using decimal will slow down the database. Several questions:
If I am updating the db with prices at 1000 times/second, will there be a detrimental performance difference if I use decimal to store the prices?
If I am performing calculations on another table with the P&L data using a stored procedure at 1 time/ second, will there be a detrimental performance difference if I use decimal to store the P&L data?
If I went with double over decimal, can I round after the calculation? Will that be faster than just using decimal?
Thank you,
GeneralRe: High-frequency updating PinmemberJeffrey Sax1-Jun-06 16:47 
Generaluniversity and collage PinmemberTrance Junkie14-Mar-06 23:51 
QuestionHelp please Pinmembermmcguirk119-Dec-05 10:42 
AnswerRe: Help please PinmemberJeffrey Sax20-Dec-05 20:27 
GeneralGreat article PinmemberJohn Cardinal9-Dec-05 14:44 
QuestionPart 2 of this Article Pinmemberrajesh2k9-Oct-05 2:56 
AnswerRe: Part 2 of this Article PinmemberQwertie4-Oct-10 12:18 
QuestionHow to set the rounding mode ? PinsussPatrick de Boer22-Jul-05 0:56 
AnswerRe: How to set the rounding mode ? PinmemberJeffrey Sax23-Jul-05 7:24 
GeneralExponent format Pinmemberasifiqbalk21-Jul-05 1:26 
GeneralRe: Exponent format PinmemberJeffrey Sax23-Jul-05 7:35 
GeneralRe: Exponent format PinsussAnonymous9-Aug-05 8:36 
GeneralFixed point numbers PinsussJacobJames4-Jul-05 0:12 
GeneralRe: Fixed point numbers PinmemberJeffrey Sax12-Jul-05 6:25 
Generalsmallest postive value PinmemberTedCB9-Jun-05 9:38 
GeneralRe: smallest postive value PinmemberJeffrey Sax12-Jul-05 6:14 
GeneralSql Data types PinmemberChris Lennon7-Jun-05 11:54 
GeneralRe: Sql Data types PinmemberJeffrey Sax7-Jun-05 14:48 
GeneralRe: Sql Data types PinmemberChris Lennon7-Jun-05 15:15 
GeneralRe: Sql Data types PinmemberJeffrey Sax7-Jun-05 18:52 



| Advertise | Privacy | Mobile
Web04 | 2.8.140916.1 | Last Updated 30 May 2005
Article Copyright 2005 by Jeffrey Sax
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid