Ansicht
Dokumentation

Floating-pt arithmetic
Tip-No.: 2798 ( INFO2798 )

Short text

Floating-pt arithmetic
Tip-No.: 2798

Symptom

Floating-pt arithmetic is liable to rounding errors. Even when convert-
ing a number in external (decimal) representation into the internal
representation (to base 2 and 16), rounding is generally applied. This
can have suprising effects: On some of our hardware platforms, for
example, 2,55/0,1 is output as 25,500...0; the number is however already
rounded - the number is in fact 25,499...96... If this number is then
rounded to the nearest whole number, you get 25 (instead of the expected
26), as the number is in fact internally less than 25,5.

Cause

Rounding errors in floating-point arithmetic are basically unavoidable.

Solution

Use type I or P when high degree of accuracy is required. Type I is only
however intended for whole numbers in the range:
-2.147.483.647 to +2.147.483.647.