×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

What Every Programmer Should Know About Floating-Point Arithmetic

Soulskill posted more than 3 years ago | from the gaining-understanding-bit-by-bit dept.

Programming 359

-brazil- writes "Every programmer forum gets a steady stream of novice questions about numbers not 'adding up.' Apart from repetitive explanations, SOP is to link to a paper by David Goldberg which, while very thorough, is not very accessible for novices. To alleviate this, I wrote The Floating-Point Guide, as a floating-point equivalent to Joel Spolsky's excellent introduction to Unicode. In doing so, I learned quite a few things about the intricacies of the IEEE 754 standard, and just how difficult it is to compare floating-point numbers using an epsilon. If you find any errors or omissions, you can suggest corrections."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

359 comments

Simple, effective and useful (1)

Jorl17 (1716772) | more than 3 years ago | (#32064158)

Excellent. The article may not be *that* specific or lengthy, but it helps out most people that have a simple problem. People with more than simple problems should know the standard word-per-word already ;)

Re:Simple, effective and useful (-1, Troll)

Anonymous Coward | more than 3 years ago | (#32064560)

If you do not mind, I would suggest the opposite: the author should return to square one and do his homework instead: this "new" topic has been around for decades - with the internet full of references, links etc. - maybe he never heard of search engines ?

Re:Simple, effective and useful (2, Informative)

Dumnezeu (1673634) | more than 3 years ago | (#32065014)

And wrong. I don't know how to use Github and if he won't bother to post an email address, I won't bother to learn about Github just for this.
The comparison [floating-point-gui.de] page is wrong. Take nearlyEqual(0.0000001, 0) for example. As the author said, using Epsilon can be bad if you don't know what you are doing. The correct form of the function is:

epsilon = 0.00001;
function nearlyEqual(a,b)
{
        return (Math.abs(b) < epsilon) ? (Math.abs(a) < epsilon) : (Math.abs((a-b)/b) < epsilon);
}

Also, parentheses don't hurt you and they help the reader.

#1 Floating Point Rule (-1)

Anonymous Coward | more than 3 years ago | (#32064176)

#1. Don't use Java, it's not IEEE-754 compliant for floating point arithmetic.

Re:#1 Floating Point Rule (3, Informative)

lenmaster (598077) | more than 3 years ago | (#32064334)

If you think that every language except Java implements IEEE-754 to the letter, you are sadly mistakenly. That fact is Java can be used just fine for floating point work in most applications.

Re:#1 Floating Point Rule (3, Insightful)

kestasjk (933987) | more than 3 years ago | (#32064498)

I'm not sure whether that is factually true, but IEEE-754 isn't exactly perfect or without alternatives so I wouldn't base my language choice on it..

That'd be like not using Java because it doesn't represent ints using ones complement; if your code relies on the specific internal implementation of data primitives you're probably doing something wrong.
(Before I get replies: Of course sometimes these things really do matter, but not often enough to dismiss a multi-purpose langauge.)

Re:#1 Floating Point Rule (5, Informative)

sdiz (224607) | more than 3 years ago | (#32064554)

Java have a strictfp keyword for strict IEEE-754 arithmetic.

Analog Computers (2, Informative)

Tisha_AH (600987) | more than 3 years ago | (#32064180)

I seems to me that this problem would pop up any time you worked with an irrational number.

Back in the early days the analog computer was used for things like ballistic calculations. I would think that they would be less prone to this type of problem.

Linearity may still be an issue (analog systems have their own set of problems).

Re:Analog Computers (3, Insightful)

maxume (22995) | more than 3 years ago | (#32064278)

Precision isn't that big a deal (we aren't so good at making physical things that 7 decimal digits become problematic, even on something the scale of an aircraft carrier, 6 digits is enough to place things within ~ 1 millimeter).

The bigger issue is how the errors combine when doing calculations, especially iterative calculations.

Re:Analog Computers (2, Insightful)

Chris Mattern (191822) | more than 3 years ago | (#32064818)

The problem is, if you're doing a long string of calculations--say a loop that repeats calculations thousands of times with the outcome of the last calculation becoming the input for the next (approximating integrals often does this) then the rounding errors can accumulate if you're not paying attention to how the floating point works.

Re:Analog Computers (5, Informative)

Anonymous Coward | more than 3 years ago | (#32064350)

No, irrationality has nothing to do with it. It's a matter of numeric systems, i.e. binary vs. decimal. For example, 0.2 is a rational number. Express it in binary floating point and you'll see the problem: 2/10 is 1/5 is 1/101 in binary. Let's calculate the mantissa: 1/101=110011001100... (long division: 1/5->2/5->4/5->8/5=1,r3->6/5=1,r1->2/5->4/5->8/5...)

All numeric systems have this problem. It keeps tripping up programmers because of the conversion between them. Nobody would expect someone to write down 1/3 as a decimal number, but because people keep forgetting that computers use binary floating point numbers, they do expect them not to make rounding errors with numbers like 0.2.

Re:Analog Computers (1)

-brazil- (111867) | more than 3 years ago | (#32064472)

True, but irrational numbers are those that cannot be written down exactly in *any* base - not even if you use recurring digits.

Re:Analog Computers (0)

Anonymous Coward | more than 3 years ago | (#32064538)

That's why they never cause problems for novice programmers: Irrational numbers can't be written down in source code, so everybody knows there's going to be some rounding error. But when people enter an innocent looking decimal number, do some trivial math and get the "wrong" result back, then the pitchforks come out and programmer heads start rolling.

Re:Analog Computers (2, Informative)

moonbender (547943) | more than 3 years ago | (#32064796)

True, but irrational numbers are those that cannot be written down exactly in *any* base

... except the irrational number's own base. ;)

Re:Analog Computers (0)

Anonymous Coward | more than 3 years ago | (#32064622)

2E-1
Oh my god! How did I do that?

Re:Analog Computers (2, Interesting)

cupantae (1304123) | more than 3 years ago | (#32064678)

Irrational numbers are not such a problem as rational numbers which can't be represented in the base used.

Lets say our computer has 6-digit-decimal precision. If you add two irrational numbers, say pi and e, you'll get 5.85987. It's imprecise, but imprecision is necessary, since it can't be represented in any base.

But if you add 3/7 and 2/3 you get 1.90524 which is imprecise even though a precise answer does exist.

.9999999984 Post (5, Funny)

tonywestonuk (261622) | more than 3 years ago | (#32064210)

Damn...Missed it! lol

Re:.9999999984 Post (0, Funny)

Anonymous Coward | more than 3 years ago | (#32064558)

This is what happens when you use intel cpus, if you want a ifsrt spto use an athlon and nothing can go rwnog.

Only scratching the surface (4, Interesting)

ameline (771895) | more than 3 years ago | (#32064214)

You really need to talk about associativity (and the lack of it). ie a+b+c != c+b+a, and the problems this can cause when vectorizing or otherwise parallelizing code with fp.

And any talk about fp is incomplete without touching on catastrophic cancellation.

Re:Only scratching the surface (1, Insightful)

Anonymous Coward | more than 3 years ago | (#32064264)

If you're interested in that, you're better of reading the article by David Goldberg (linked in TFS and the first paragraph of TFA). The whole point of -brazil-'s page is that it sums up some essential issues for novice programmers that find those in-depth descriptions complicated and daunting. Better to keep his page simple than duplicate an already-existing article and become just as inaccessible to newbies.

Re:Only scratching the surface (1, Informative)

Anonymous Coward | more than 3 years ago | (#32064342)

You really need to talk about associativity (and the lack of it). ie a+b+c != c+b+a

This is actually non-commutativity. Non-associativity means (a+b)+c != a+(b+c).

Re:Only scratching the surface (0)

Anonymous Coward | more than 3 years ago | (#32064608)

Its difficult to say when he is adding 3, not 2 arguments...but yes, what he was trying to express was non-commutativity.

Re:Only scratching the surface (0)

Anonymous Coward | more than 3 years ago | (#32064614)

Floating point arithmetic is both non-commutative & non-associative.

0.5 * 5000000000.78987 * 2000000000000000000 is dependant on the order you do the mutiplication (these particular numbers chosen from thin air, but just demonstate the pattern that'll get you in trouble).

Re:Only scratching the surface (1)

Dylan16807 (1539195) | more than 3 years ago | (#32064836)

Actually, since .5 only affects the exponent, not the mantissa, your example is probably fine.

Re:Only scratching the surface (0)

Anonymous Coward | more than 3 years ago | (#32064648)

Wrong. FP arithmetic is commutative, so his example can't be non-commutativity. It's actually a hidden case of non-associativity:
a+b+c = (a+b)+c = (b+a)+c = c+(b+a) != (c+b)+a = c+b+a.

Re:Only scratching the surface (0)

Anonymous Coward | more than 3 years ago | (#32064384)

a+b+c != c+b+a is about commutativity - I think you mean (a+b)+c != a+(b+c) - that is associativity (or rather, lack of it).

Cool down, please (-1, Flamebait)

Anonymous Coward | more than 3 years ago | (#32064528)

You need a cock in your ass today.

Re:Only scratching the surface (1)

TheRaven64 (641858) | more than 3 years ago | (#32064534)

The lack of associativity is a bigger problem than you might think, because the compiler can rearrange things. If you're using x87 FPU code, you get 80-bit precision in registers, but only 64-bit or 32-bit precision when you spill to the stack. Depending on the optimisations that are run, this spill happens at different times, meaning that you can get different results depending on how good your register allocator is. Even upgrading the compiler can change the results.

If you are using floating point values sensibly, then you are aware that they only guarantee a certain minimum precision and that you shouldn't rely on the low bits too heavily.

Re:Only scratching the surface (0)

Anonymous Coward | more than 3 years ago | (#32065022)

I don't know why you'd expect a vectorized port of a SISD code to produce identical results, nor why you would make this a particular goal. All floating point arithmetic is inaccurate -- who is to say the original result was correct and the new one was wrong? Maybe the original was wrong. Maybe your baseline was computed on a Pentium with the FDIV bug. Who knows?

If you rely on floating point calculations to come out just exactly a certain way, I wonder whether you are trying to use float results in the logical flow of algorithms. That's really the biggest no-no as far as I'm concerned. Floats are for computing more floats, not controlling flow of execution. You never have to worry about whether two floats are "close enough to be equal" if you never test them for equality in the first place. Testing equality means doing logic with floats. Purge this concept from your mind. You don't do logic with floats.

equivalent to Joel Spolsky's excellent introductio (-1, Troll)

Anonymous Coward | more than 3 years ago | (#32064246)

equivalent to Joel Spolsky's excellent introduction to Unicode.

Sorry, but his introduction to unicode is shit. Nobody anywhere should read or follow the advice therein.

If you want accuracy... (2, Interesting)

Nutria (679911) | more than 3 years ago | (#32064276)

use BCD math. With h/w support it's fast enough...

Why don't any languages except COBOL and PL/I use it?

Re:If you want accuracy... (1)

larry bagina (561269) | more than 3 years ago | (#32064302)

how do you express 1/3 in BCD?

Re:If you want accuracy... (1)

vlm (69642) | more than 3 years ago | (#32064370)

$x_numerator = 1;
$x_denominator = 3;

Algorithms do exist for fractional number arithmetic. If the denominator gets unwieldy, who cares, its a computer and its fast and memory is "free".

Re:If you want accuracy... (1)

Nutria (679911) | more than 3 years ago | (#32064426)

how do you express 1/3 in BCD?

Just as in "paper" decimal arithmetic, you must truncate it somewhere.

Since I've long forgotten how to "punch" the sign, this is what it would look like in 8(2) imaginary "unsigned" BCD, in hex: 00000033.

Re:If you want accuracy... (4, Insightful)

JamesP (688957) | more than 3 years ago | (#32064430)

Maybe because BCD is the worse possible way to do 'proper' decimal arithmetic, also it would absolutely be very slow.

BCD = 2 decimal digits per 8 bits (4 bits per dd). Working 'inside' the byte sucks

Instead you can put 20 decimal digits in 64bits (3.2 bits per db) and do math much more faster

Why don't any languages except COBOL and PL/I use it?

Exactly

Re:If you want accuracy... (3, Interesting)

TheRaven64 (641858) | more than 3 years ago | (#32064544)

also it would absolutely be very slow

Depends on the architecture. IBM's most recent POWER and System-Z chips have hardware for BCD arithmetics.

Re:If you want accuracy... (1)

Nutria (679911) | more than 3 years ago | (#32064584)

Maybe because BCD is the worse possible way to do 'proper' decimal arithmetic,

"0.1 + 0.2 = 0.30000000000000004" doesn't really seem all that proper to me.

But BCD *does* do "0.1 + 0.2 = 0.3".

also it would absolutely be very slow.

Without h/w support.

Instead you can put 20 decimal digits in 64bits (3.2 bits per db) and do math much more faster

I want accurate math, not estimates.

Exactly

Do you pride yourself a Rational Man, or a low down dirty bigot?

Re:If you want accuracy... (1)

cgenman (325138) | more than 3 years ago | (#32064810)

How is that Hardware Support going? Just curious.

Re:If you want accuracy... (1)

Nutria (679911) | more than 3 years ago | (#32065044)

How is that Hardware Support going?

Very well, on machines designed for business (i.e., mainframes and VAXen).

Re:If you want accuracy... (2, Insightful)

amorsen (7485) | more than 3 years ago | (#32065056)

Instead you can put 20 decimal digits in 64bits (3.2 bits per db) and do math much more faster

I want accurate math, not estimates.

Math with 20 decimal digits in 64 bits is proper decimal arithmetic. It acts exactly like BCD does, it just doesn't waste tons of space and CPU power.

Re:If you want accuracy... (1)

TerranFury (726743) | more than 3 years ago | (#32064752)

A rational number class seems like a better solution, though there are some issues with representing every number as a ratio of integers... For instance, you need extremely large numerators as your denominators get large. For another, you need to keep computing gcfs to reduce your fractions. Still, this is the approach used in some calculator programs.

I wonder if a continued fraction [wikipedia.org] representation would have advantages -- and if it has ever been used as a number representation in practice? It seems like it'd be very expensive, but I'd need to give it some thought...

a bit dup'y (1)

smoothnorman (1670542) | more than 3 years ago | (#32064304)

Somewhat a dup from seven years ago ... Posted by Cliff on Thursday June 26 2003, @04:42PM http://developers.slashdot.org/story/03/06/26/0044223/Floating-Point-Programming-Today [slashdot.org] ...ah well

Re:a bit dup'y (0)

Anonymous Coward | more than 3 years ago | (#32064332)

Just because it's about a related subject doesn't mean it's a dupe. This article is specifically about the new web site, so it can't possibly be a dupe of an article from years before the web site was created.

Another potential solution is Interval arithmetic (3, Insightful)

renoX (11677) | more than 3 years ago | (#32064310)

Maybe in your list of solutions you could mention interval arithmetic [wikipedia.org] , it's not very much used, but it gives "exact" solution.

Re:Another potential solution is Interval arithmet (1)

-brazil- (111867) | more than 3 years ago | (#32064518)

Why don't you write it up yourself and give me a github pull request? :)

Re:Another potential solution is Interval arithmet (1)

renoX (11677) | more than 3 years ago | (#32064736)

Because I don't know how to use github and it looks to me as a really, really complicated way to make a wiki..

Re:Another potential solution is Interval arithmet (1)

-brazil- (111867) | more than 3 years ago | (#32064846)

Well, I'll do it when I get around to it. Doing it as a wiki would mean that I'd have to deal with vandalism and spam, and it's really intended more as small self-contained site than a community thing.

Re:Another potential solution is Interval arithmet (0)

Anonymous Coward | more than 3 years ago | (#32064938)

Nice attitude. "Hey, I think this would be a good idea". "Shut up and do it yourself instead". Typical response from an open source developer; aggressive, smug and defensive.

What Every Programmer Should Know (1)

trust_jmh (651322) | more than 3 years ago | (#32064320)

Computers are about cheating.
Do it right and you won't let anyone input enough to get caught out by the bug in your codes hidden methods.

#1 Rule, Don't use Java (-1, Troll)

Anonymous Coward | more than 3 years ago | (#32064330)

and the #1 floating point rule, if you need any sort of accuracy, or people's lives depend [heaven forbid] on your code, Don't use java, since it's not IEEE-754 compliant.

Might as well run on an FDIV Pentium, since the java philosophy is everything needs to be just "good enough".

Re:#1 Rule, Don't use Java (2, Informative)

abigor (540274) | more than 3 years ago | (#32064356)

Actually, the linked article says exactly the opposite, and up above I posted a link to the JVM definition that verifies it. So you are 100% incorrect.

What school are these programmers going to? (1)

TSRX (1129939) | more than 3 years ago | (#32064358)

What kind of terrible CS program doesn't teach floating-point arithmetic as part of a required course?

Re:What school are these programmers going to? (1)

-brazil- (111867) | more than 3 years ago | (#32064536)

These days, not everyone who writes programs has some sort of formal CS education. And those who do may have forgotten about it by the time they run into this kind of problem.

Re:What school are these programmers going to? (0, Flamebait)

larry bagina (561269) | more than 3 years ago | (#32064542)

At one previous job, of about 15 programmers, I think fewer than 5 had a CS degree. At another, of 3 programmers, not one had a CS degree. There are more copy-paste programmers than ever (especially with low barriers to entry -- php, python, javascript, vb/vba, etc).

I'd just avoid it (4, Interesting)

Chemisor (97276) | more than 3 years ago | (#32064396)

Given the great complexity of dealing with floating point numbers properly, my first instinct, and my advice to anybody not already an expert on the subject, is to avoid them at all cost. Many algorithms can be redone in integers, similarly to Bresenham, and work without rounding errors at all. It's true that with SSE, floating point can sometimes be faster, but anyone who doesn't know what he's doing is vastly better off without it. At the very least, find a more experienced coworker and have him explain it to you before you shoot your foot off.

Re:I'd just avoid it (4, Informative)

-brazil- (111867) | more than 3 years ago | (#32064576)

The non-trivial problems with floating-point really only turn up in the kind of calculations where *any* format would have the same or worse problems (most scientific computing simply *cannot* be done in integers, as they overflow too easily).

Floating-point is an excellent tool, you just have to know what it can and cannot do.

Re:I'd just avoid it (2, Insightful)

Rogerborg (306625) | more than 3 years ago | (#32064920)

Depends how many "integers" you use. If you need accuracy - and "scientific" computing certainly does - then don't use floats. Decide how much accuracy you need, and implement that with as many bytes of data as it takes.

Floats are for games and 3D rendering, not "science".

Re:I'd just avoid it (1)

smallfries (601545) | more than 3 years ago | (#32064872)

And once you've finished writing your algorithm in manually coded fixed point to avoid the "complexities" of float-point you can sit down and tuck into a tasty plate of quiche.

float are over (0)

Anonymous Coward | more than 3 years ago | (#32064398)

float are over, we should go back to fixed point. With 64bits interger we already have more precision then 32bit float. Let use that 64bit adventage to drop the float madness and leave the 64bit float for a few niche applications.

Re:float are over (4, Funny)

sco08y (615665) | more than 3 years ago | (#32064564)

Really, the best answer is to store all numbers on the cloud, and just use a 256-bit GUID to look them up when needed.

Three one-thirds is 1, riiiiight? (0)

Anonymous Coward | more than 3 years ago | (#32064418)

I'm completely baffled by this (in Python):
>>> print (1/3)*3
0

I expected 1, and my FPU is from AMD, not Intel ;)

Re:Three one-thirds is 1, riiiiight? (2, Informative)

jjohnson (62583) | more than 3 years ago | (#32064474)

>>> (1.0/3)*3
1.0

Re:Three one-thirds is 1, riiiiight? (0)

Anonymous Coward | more than 3 years ago | (#32064964)

In Perl, this outputs "0.333333333333333" either way. But it gets "0.1 + 0.3" right.

Re:Three one-thirds is 1, riiiiight? (1)

TheRaven64 (641858) | more than 3 years ago | (#32064588)

I try to avoid Python, but this result makes sense if you are doing integer division. 1/3 is 0 remainder 2. The result of the integer division will be 0, which remains 0 when multiplied by 3. You'd get the same result in C, but if you do (1.0/3)*3 then you should get something approximately equal to 1.

Re:Three one-thirds is 1, riiiiight? (1)

MobileTatsu-NJG (946591) | more than 3 years ago | (#32064784)

I'm completely baffled by this (in Python):
>>> print (1/3)*3
0

I expected 1, and my FPU is from AMD, not Intel ;)

You didn't use the FPU. Try this: print (1.0 / 3) * 3;

An intuitive description of floating-point (1)

noidentity (188756) | more than 3 years ago | (#32064434)

Here's a one-page intuitive description of floating-point [fly.net] , to give a feel for why multiplication and division are fairly benign, but addition and subtraction aren't. None of the other descriptions have made it as clear as this.

Re:An intuitive description of floating-point (0)

Anonymous Coward | more than 3 years ago | (#32064730)

I wouldn't say fairly benign. I'd say less likely to have as big problems e.g. adding a bunch of very small numbers to a large number can cause a bigger error compared to multiplying those bunch of numbers, but there could still be an increasing error with the latter.

Re:An intuitive description of floating-point (1)

noidentity (188756) | more than 3 years ago | (#32064864)

As described in the linked text, when you multiply the relative error stays roughly the same. Obviously some error does occur, and it's cumulative in many cases, but this is the case for any imprecise operation. I still think the error is fairly benign.

No, base 10 arithmetic isn't "more accurate". (3, Interesting)

Animats (122034) | more than 3 years ago | (#32064454)

The article gives the impression that base 10 arithmetic is somehow "more accurate". It's not. You still get errors for, say, 1/3 + 1/3 + 1/3. It's just that the errors are different.

Rational arithmetic, where you carry along a numerator and denominator, is accurate for addition, subtraction, multiplication, and division. But the numerator and denominator tend to get very large, even if you use GCD to remove common factors from both.

It's worth noting that, while IEEE floating point has an 80-bit format, PowerPCs, IBM mainframes, Cell processors, and VAXen do not. All machines compliant with the IEEE floating point standard should get the same answers. The others won't. This is a big enough issue that, when the Macintosh went from Motorola 68xxx CPUs to PowerPC CPUs, most of the engineering applications were not converted. Getting a different answer from the old version was unacceptable.

Re:No, base 10 arithmetic isn't "more accurate". (1)

-brazil- (111867) | more than 3 years ago | (#32064600)

Actually, I tried to make it very clear in several places that base 10 has just the same problems. I am open to any suggestions for improvement, though.

Re:No, base 10 arithmetic isn't "more accurate". (1)

MobileTatsu-NJG (946591) | more than 3 years ago | (#32064812)

The article gives the impression that base 10 arithmetic is somehow "more accurate". It's not. You still get errors for, say, 1/3 + 1/3 + 1/3. It's just that the errors are different.

What kind of errors are you referring to?

Stop with the educational articles (4, Funny)

sunderland56 (621843) | more than 3 years ago | (#32064458)

Look, times are tough for programmers already. Knowing how to do things correctly - like proper floating point math - is one of the ways to separate the true CS professional from the wannabe new graduates. Articles like this just make everyone smarter, and make finding a job that much harder.

Not sure it belongs in an intro explanation, but (4, Informative)

dr2chase (653338) | more than 3 years ago | (#32064466)

Other issues that might be worth mentioning:
  • Catastrophic cancellation in complex arithmetic.
  • Single vs double rounding, in fused vs cascaded multiply-add operations.
  • Range reduction in trig functions (Intel hardware only uses a 68-bit PI, this causes problems sometimes).
  • Double-rounding when converting FP precision (e.g., 64-bit mantissa to 53, or 53 with extended exponent to 53 with regular exponent when the mantissa goes denorm).
  • Conversion to/from string representation.
  • Issues with "constructive reals" (a=b? is not necessarily an answerable question -- you might need to look at "all" the digits in order to answer "yes").
  • Distributive law DOES NOT HOLD -- a * (b+c) != a*b + a*c

Re:Not sure it belongs in an intro explanation, bu (1, Informative)

Anonymous Coward | more than 3 years ago | (#32064492)

PLEASE stop this thread - interval arithmetic and all that has been around as a research topic for decades now.

If you did not know: there are search engines providing lots of links...

Consistency across platforms (1)

slart42 (694765) | more than 3 years ago | (#32064566)

Another thing I think an article labelled "What every Programmer should know..." should mention is that, while "Nearly all hardware and programming languages use floating-point numbers in the same binary formats", implementations can actually be very inconsistent, in the results of computations.

For example, mac and windows programs will have different rounding behaviors, and come up with different computation results in floating point math (at least with the default compiler settings - does anyone know if it is possible to change this?). This is constantly causing us trouble, when trying to write Unit tests only to realize the expected output values are slightly different on different machines to run the tests on.

I also think that, as somebody posted above, the non-associativity of floating point operations is important and should be mentioned. But, otherwise, a well written and helpful article!

Re:Consistency across platforms (0)

Anonymous Coward | more than 3 years ago | (#32064780)

Uh, floating point results can even change due to using a different compiler.
It's clearly said that you can't just do a == on floating point numbers, why would you think this does not apply to your unit tests just as much?
(Not to mention that your unit tests are obviously not testing what you want, because if they did a failing test means: you _can't_ use floating point for your problem and you have to rewrite your code to not use it).
Only way to get reliable results with floating point is assembler code (and making sure to set the floating point state in that assembler code, if applicable to your architecture).

Every software development job interview... (1)

fragMasterFlash (989911) | more than 3 years ago | (#32064596)

...should include a sample of some code that attempts to perform logical operations based on the outcome of floating point operations. Anyone who attempts to check the result of a floating point operation for equality without normalizing precision should be shown to the door and never called back.

CS stuff (1)

mseeger (40923) | more than 3 years ago | (#32064612)

This sounds a lot like the stuff i heard in CS/Mathematics classes more than 20 years ago (Hackbusch, Praktische Analysis, Summer term 1987/88). That course was mandatory then for any CS, Mathematics and Physics student. Has that changed yet? It's about the differences between a number at it's binary representation (and examples about consequences).

I completely agree, that every programmer should know about that. But this is nothing new, it was already important 40 years ago. I'm pretty sure some space probe/missile already got lost due to this problems.

CU, Martin

P.S. It's not only important for programmers (dealing with C, C++, Java, etc.). Several times i was called to debug some "Excel Error" when it turned out to be a floating point restriction.... It would be interesting if you can create a macro that checks for such conditions.

Scientist Programmers (0)

Anonymous Coward | more than 3 years ago | (#32064660)

I used to do algorithm ports for science grade code to operational grade for satellite data processing. The hardest part of the job was trying to convince the phds that considered themselves 'expert programmers' that we hadn't screwed something up in the port when the answers didn't exactly match. I'm not sure if its because they didn't think we had anything of value to contribute to the process, or if they were freaked out by the fact that all of the accuracy they had been touting was only perceived instead of actual. To their credit it is pretty difficult to convince someone that everything is working just fine when the algorithm takes a single sensor value representing an 800 sq m swath of land and spits out 'desert' on their system and 'tundra' on ours.

Comparison code seems wrong (0)

Anonymous Coward | more than 3 years ago | (#32064748)

On your page about comparison, you give the following code:

function nearlyEqual(a,b)
        {
                return a==b || Math.abs((a-b)/b) 0.00001;
        }

if(nearlyEqual(a,b))

I think it will fail with a division by zero if e.g. a==2.0 and b==0.0.

What almost every programmer should know about FP: (1)

Ant P. (974313) | more than 3 years ago | (#32064814)

Nothing at all.

Someone else has already written a library for the hard part of what you're doing, and their code *works*. Use it.

Joel Spolsky's Unicode explanation was a joke... (0)

Anonymous Coward | more than 3 years ago | (#32064892)

I remember when I first read it: it was all wrong, utterly confusing several notions. I flamed that donkey to death and he updated/changed his Unicode explanation. It was really a "ah ah, I just understood why Unicode is good, let's blog about it"-type of article, full of errors.

That guy is not a good programmer at all. Somehow commented here on /. a while ago that Joel Spolsky was approximately saying something interesting as often as sunlight shines in one's arse. Honestly the only thing that Spolsky said that I found interesting was when he said that he created stackoverflow.com because he was tired of "looking for good speakers" and finding endless "meta" discussion about the speakers-forum ("why is my comment in bold?") instead of finding discussion about speakers itself.

stackoverflow.com kinda works (it's a hive full of retards but it kinda works once you ask "advanced", like help for some elisp piece of code) and for that kudos to M. Spolsky...

But his articles are typically "ah ah moments" full of errors. He's not a good programmer and should not be quoted nor used as a reference.

Please look here (4, Informative)

ctrl-alt-canc (977108) | more than 3 years ago | (#32064916)

People interested into floating point math will find some very interesting materials and horror stories in the documents collected at the home page [berkeley.edu] of professor William Kahan, the man behind IEEE754 standard.
According to my personal experience the paper by David Goldberg cited in the post isn't that difficult after all. Plenty of interesting materials can also be found in the Oppenheim & Shafer [amazon.com] textbook about digital signal processing.

Hard to debug floating point when it goes wrong! (4, Interesting)

Cliff Stoll (242915) | more than 3 years ago | (#32064930)

Over at Evans Hall at UC/Berkeley, stroll down the 8th floor hallway. On the wall, you'll find an envelope filled with flyers titled, "Why is Floating-Point Computation so Hard to Debug whe it Goes Wrong?"

It's Prof. Kahan's challenge to the passerby - figure out what's wrong with a trivial program. His program is just 8 lines long, has no adds, subtracts, or divisions. There's no cancellation or giant intermediate results.

But Kahan's malignant code computes the absolute value of a number incorrectly on almost every computer with less than 39 significant digits.

Between seminars, I picked up a copy, and had a fascinating time working through his example. (Hint: Watch for radioactive roundoff errors near singularities!)

Moral: When things go wrong with floating point computation, it's surprisingly difficult to figure out what happened. And assigning error-bars and roundoff estimates is really challenging!

Try it yourself at:
http://www.cs.berkeley.edu/~wkahan/WrongR.pdf [berkeley.edu]

Thank goodness for IEEE 754 (1)

FranTaylor (164577) | more than 3 years ago | (#32065012)

Most of you are too young to have dealt with architectures like VAX and Gould PowerNode with their awful floating point implementations.

IEEE 754 has problems but it's a big improvement over what came before.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...