Anatomy of a floating point

A floating-point number has three components:

• a sign bit
• a biased exponent
• a mantissa

Bias () is set to where is the number of exponent bits ( in this example).

The function that maps these bits to a decimal value is:

Fully working example

View the bit string as:

• S =
• E = = 9
• M =
• B =

---

Normal and subnormal modes

• Normal: the exponent has at least one non-zero bit
• Subnormal: all exponent bits are

Note: Evaluating a decimal under subnormal mode is slow and expensive.

---

Handling 0

The formula above never evaluates to . A zero is represented by setting all the mantissa and exponent bits to . This actually gives two zeros, one for each value of the sign bit: and .

• Positive 0
• Negative 0

---

Handling inf and NaN

Infinity is represented by setting all exponent bits to and all mantissa bits to . The sign bit then chooses between inf and -inf. If the exponent is all s but the mantissa has at least one non-zero bit, the value is . Any bit pattern in the mantissa works, which is why there are many distinct NaN encodings.

• Positive infinity
• Negative infinity
• NaN

---

Min and max values

Since a FP with an all-s exponent is either inf or NaN, the largest usable exponent is .

• Max positive value
  • all s, which equals
  • .
• Min positive normal value
  • all s, which equals
• Max positive subnormal value
  • all s
  • all s, which equals
• Min positive subnormal value
  • all s

The negative counterparts are obtained by flipping the sign bit.

Subnormals enable gradual underflow and lexicographic ordering

For non-negative floats, the bit-pattern order matches the numeric order: incrementing the bits by 1 gives the next representable value. This holds because the exponent field sits to the left of the mantissa - bumping the mantissa past all s carries into the exponent, exactly like bumping a decimal past carries into the next digit.

Subnormals are what keep this property alive near zero. Without them, the smallest positive normal is , and the next bit pattern below it (all s in the exponent) would have to mean : a jump from straight down to , breaking the "decrement bits by 1, get the next-smaller value" rule.

By interpreting the all-s exponent as , the interval is filled evenly with values, and lexicographic order keeps matching numeric order all the way down to .

Note the gradual jump from minimum normal to maximum subnormal in the above example.

---

Not all representations have inf and NaN

Format	Exponent	Mantissa	±∞	NaN
FP32	8	23	Yes	Yes
BF16	8	7	Yes	Yes
FP16	5	10	Yes	Yes
FP8 E4M3	4	3	No	Yes
FP8 E5M2	5	2	Yes	Yes
FP4 E1M2	1	2	No	No
NVFP4 E2M1	2	1	No	No

---

Interactive E2M1 bits

Modify the four bits of E2M1 yourself. Here bias is .