Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Firefox Analyzed for Bugs by Software

CowboyNeal posted more than 8 years ago | from the killing-bugs-dead dept.

226

eldavojohn writes "In a brief article on CNet, a company named Coverity announced that Firefox is using software to detect flaws in Firefox's source code. Even more interesting is the DHS initiative for Coverity to use this same bug detection software on 40 open source projects." An interesting tidbit from the article: "Most of the 40 programs tested averaged less than one defect per thousand lines of code. The cleanest program was XMMS, a Unix-based multimedia application. It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code. The buggiest program is the Advanced Maryland Automatic Network Disk Archiver, or AMANDA, a Linux backup application first developed at the University of Maryland. Coverity found 108 bugs in its 88,950 lines of code, or about 1.214 bugs per thousand lines of code." We've covered this before, only now Firefox is actually licensing the Coverity software and using it directly.

cancel ×

226 comments

Sorry! There are no comments related to the filter you selected.

GNAA (-1, Offtopic)

6N44 (994472) | more than 8 years ago | (#15894403)

.________________________________________________. | ______________________________________._a,____ | | _______a_._______a_______aj#0s_____aWY!400.___ | | __ad#7!!*P____a.d#0a____#!-_#0i___.#!__W#0#___ | | _j#'_.00#,___4#dP_"#,__j#,__0#Wi___*00P!_"#L,_ | | _"#ga#9!01___"#01__40,_"4Lj#!_4#g_________"01_ | | ________"#,___*@`__-N#____`___-!^_____________ | | _________#1__________?________________________ | | _________j1___________________________________ | | ____a,___jk_GAY_NIGGER_ASSOCIATION_OF_AMERICA_ | | ____!4yaa#l___________________________________ | | ______-"!^____________________________________ | ` _______________________________________________'

Re:GNAA (-1, Offtopic)

Anonymous Coward | more than 8 years ago | (#15894513)

Meanwhile, GNAA's average bug count rises to 999 per thousand lines?

You guys are losing your touch. Pity.

Re:GNAA (4, Funny)

biscon (942763) | more than 8 years ago | (#15894571)

Looks like somebody failed troll academy ;)

Re:GNAA (1)

wwiiol_toofless (991717) | more than 8 years ago | (#15894608)

Wait! Is that the source code?!

Math (5, Informative)

Anonymous Coward | more than 8 years ago | (#15894412)

That's .051 bugs per thousand lines of code for XMMS, an order of magnitude better.

Re:Math (0)

Anonymous Coward | more than 8 years ago | (#15894521)

Good catch. It made no sense that quality varied so little (from 0.51 to 1.214) for 40 different programs. Now the numbers make more sense.

6/116.899 = .051
108/88.950 = 1.214

Re:Thank God (1, Funny)

Anonymous Coward | more than 8 years ago | (#15894611)

And thank god for that score. I don't mind my AMANDA backups being corrupted, but if my mp3 of "Hit Me Baby, One More Time" ever skipped a beat, I wouldn't know what to do...

Re:Thank God (0)

Anonymous Coward | more than 8 years ago | (#15894817)

I agree... can't get enough of the Children of Bodom!

Re:Math (4, Informative)

It'sYerMam (762418) | more than 8 years ago | (#15894981)

Quite possibly because XMMS is practically stagnant. I don't even use it, but Amarok is by far the best audio app I've tried for Linux, quite possibly because the people developing it have some idea of which decade they're in.

not just any software? (0)

amrust (686727) | more than 8 years ago | (#15894416)

"Firefox is using software to detect flaws in Firefox's source code."

Didn't they always use software of some sort, Bugzilla, etc? I assume they specifically mean "Firefox is using Coverity software to detect flaws in Firefox's source code."

Re:not just any software? (3, Informative)

decadre (980513) | more than 8 years ago | (#15894437)

Err?... I always thought Bugzilla was just where you reported bugs in Mozilla suite products?...

How does this bug detection software work anyway?

Re:not just any software? (3, Informative)

Anonymous Coward | more than 8 years ago | (#15894447)

Didn't they always use software of some sort, Bugzilla, etc?

Bugzilla is a issue tracking software; it's useful only after you've already found a bug. The only other bug-related tool they use is the FullCircle crash reporter thingy, again, after-the-fact thing. This is different - this tool finds flaws from the source code automatically.

Firefox is a browser (2, Insightful)

vain gloria (831093) | more than 8 years ago | (#15894634)

I assume they specifically mean "Firefox is using Coverity software to detect flaws in Firefox's source code."

And I'm assuming that they mean "Mozilla is using Coverity..." or "Firefox developers are using Coverity...". After all you don't hear about what Internet Explorer is doing, but rather what MS are doing with it.

Wouldn't it be great if the summary was clearer and neither of us had to make mental amendments? :(

Re:not just any software? (1, Funny)

Anonymous Coward | more than 8 years ago | (#15894803)

Didn't they always use software of some sort, Bugzilla, etc?
Unfricking believable! You obviously know nothing about this subject, so please, please keep your trap shut and let the grown-ups talk.

If this is the same (3, Interesting)

Anonymous Coward | more than 8 years ago | (#15894425)

If this is the same as most automated testing software I've seen, it detects many things which aren't truly bugs as bugs. Accuracy on automated testing tools I've been exposed to is around 40%.

Re:If this is the same (1)

toochoos (991616) | more than 8 years ago | (#15894842)

Indeed, the story looks more like a sladvertisment than anything else. These tools can't neither find all bug in your code nor swear that those it finds are real bugs. At best, it helps to follow "good practices" and be more precautious. I have been confronted to these kind of tools before and what I disliked most, and the bad understanding of project managers. They definitely are the only ones who get excited about it!

Re:If this is the same (1, Informative)

Anonymous Coward | more than 8 years ago | (#15894858)

Coverity claims a false positive rate that is less than 20%.

That would tend to reccomend it to me (2, Insightful)

jnelson4765 (845296) | more than 8 years ago | (#15894430)

I will definitely take another look at Coverity's products, if the Firefox team is finding value in it.

Re:That would tend to reccomend it to me (0, Troll)

Timesprout (579035) | more than 8 years ago | (#15894454)

Would you also start sucking down laxatives if the Firefox team became incontinent?

Re:That would tend to reccomend it to me (0)

Whiney Mac Fanboy (963289) | more than 8 years ago | (#15894507)

Would you also start sucking down laxatives if the Firefox team became incontinent?

Or maybe he follows development advice from one of the largest software projects around? Tard.

Re:That would tend to reccomend it to me (0, Troll)

Timesprout (579035) | more than 8 years ago | (#15894574)

If he is not already using tooling like this then he probably should not be developing, this is basic QA.

Re:That would tend to reccomend it to me (1)

Vlad_the_Inhaler (32958) | more than 8 years ago | (#15894605)

If he is not already using tooling like this then he probably should not be developing, this is basic QA.
The point of the article was that this particular tool - not some other generic tool like it - is being used by some major projects. You are in a hole. Stop digging.

Re:That would tend to reccomend it to me (0)

Anonymous Coward | more than 8 years ago | (#15894692)

Or maybe he should be digg ing, nyuk nyuk nyuk!

Re:That would tend to reccomend it to me (1)

Whiney Mac Fanboy (963289) | more than 8 years ago | (#15894613)

If he is not already using tooling like this then he probably should not be developing, this is basic QA.

What utter nonsense. Coverity goes far beyond basic QA.

Re:That would tend to reccomend it to me (0)

Anonymous Coward | more than 8 years ago | (#15894671)

If he is not already using tooling like this then he probably should not be developing, this is basic QA.

Considering that this statement is nearly the opposite of your previous one, I'm guessing that you just can never stand to appear to be wrong about anything, and so you defensively thrash about, tossing out any wild claim you can to avoid looking dumb.

Amusingly, it has the opposite effect.

Re:That would tend to reccomend it to me (0)

onemorechip (816444) | more than 8 years ago | (#15894885)

Would you also start sucking down laxatives if the Firefox team became incontinent?

Worst...analogy...ever!

Re:That would tend to reccomend it to me (1)

mobby_6kl (668092) | more than 8 years ago | (#15895046)

>Worst...analogy...ever!

I'm afraid that distinction is reserved for this user:
BadAnalogyGuy [slashdot.org]

Re:That would tend to reccomend it to me (1)

charlieman (972526) | more than 8 years ago | (#15894919)

Aaaaand that's exactly what they want you to do!!

That's the whole point of this article!

is there -1 paranoic?

Re:That would tend to reccomend it to me (0)

Anonymous Coward | more than 8 years ago | (#15894932)

Exactly. This is part of Coverity's marketing campaign. Evaluate it intelligently and look at the other options. Don't just pick it blindly because Firefox is using it.

Errr... (0, Redundant)

The MAZZTer (911996) | more than 8 years ago | (#15894435)

I hope these Coverity guys aren't pompous enough to think that their tool can find ALL bugs in a program with... magic...

Hmm, they should run their tool on its own source code, that would be fun.

Re:Errr... (2, Insightful)

portmapper (991533) | more than 8 years ago | (#15894497)

> I hope these Coverity guys aren't pompous enough to think that their tool can find ALL bugs in a program with... magic...

I am sure that they know their tools limitations, but I am pretty sure that others will interpret
no outstanding bugs as if the application is secure or bugfree. Ethereal (now known as wireshark) has
a very low bug count, but I will not use it due to numerous past remote exploits coupled with
little interest in fixing bugs contra adding new features.

> Hmm, they should run their tool on its own source code, that would be fun.

I would be very surprised if they did not.

Re:Errr... (5, Interesting)

twiddlingbits (707452) | more than 8 years ago | (#15894500)

Finding all POSSIBLE bugs in a software program means traversing all possible paths in the code with all possible inputs. That's a HUGE problem. You can "model" the code using Logic Equations and that helps some but any errors in the conversion from code to logic equations invalidate results. The DoD and NASA have spent many millions on solving this problem over the last 10-12 yrs. When I was at NASA we used several different tools (CodeSurfer, Purify, Lint, Polyspace as I recall) as each tool was better at one thing (i.e memory leaks vs null pointer dereferences). A The complete process took a couple of days to weeks and then human eyes and expertise were still needed to remove false positives. A good site for all the tools out there, old & new is http://spinroot.com/static/ [spinroot.com] . Looks like Coverty might be a good one to look into, as the best I had seen was CodeSurfer. All the good tools I have seen are commercial (NOT open Source) and EXPENSIVE!! I'd love to see a decent open source tool to run as a first pass before applying the other tools. Another point is that these tools are STATIC analysis. Run-Time Analysis is a whole 'nother animal but that area is improving with tools like DTRACE in Solaris.

Re:Errr... (1)

portmapper (991533) | more than 8 years ago | (#15894640)

> Finding all POSSIBLE bugs in a software program means traversing all possible paths in the code with all possible inputs. That's a HUGE problem.

That is provable impossible for applications in general using the software tools as of today (in general).
So tools concentrate on common problems, or low-hanging-fruits, so to speak.

> You can "model" the code using Logic Equations and that helps some but any errors in the conversion from code to logic equations invalidate results.

There are several logic models where everything expressed in those model is provable true or false. But
using these models demands a higher level of mathematics and tolerance to "slow" progress that just about
any business or open source project will tolerate. Of course, you need a programming language where this
is practical, and C/C++ does not cut it.

> I'd love to see a decent open source tool to run as a first pass before applying the other tools.

OpenBSD has recently put quite a bit effort into making the in-tree lint much more useful.

Re:Errr... (4, Interesting)

twiddlingbits (707452) | more than 8 years ago | (#15894799)

I had some extensive conversations with the team at CodeSurfer and they think they the problem is NOT impossible, maybe more like Polynomial time. The DOD was funding them (this was about 3 yrs) ago to try to develop a solution that worked for C/C++ and Ada. NASA wanted to tag along on the research but we were told it was "classified" and DOD only. It's rare when someone turns down research money so they must be on to something.

Re:Errr... (0)

Anonymous Coward | more than 8 years ago | (#15894703)

The only open-source tool I know of is called FindBugs developed by the University of Maryland and available at http://findbugs.sourceforge.net/ [sourceforge.net] .

However, it only performs static analysis on Java programs.

Re:Errr... (1)

Wulfstan (180404) | more than 8 years ago | (#15894750)

Hate to tell you, but Purify is most definitely a dynamic analysis tool. Basically it works by substituting malloc, new and friends and then showing you on the fly your program behaviour (good at catching things like memory leaks). Static analysis tools like lint do not involve executing the programs, just analyzing it's structure to see which code paths are followed; dynamic analysis tools hook the actual execution.

Re:Errr... (1)

twiddlingbits (707452) | more than 8 years ago | (#15894783)

Lots more to run time analysis than what Purify does. I can check for the same faults w/o executing the code in many other tools.

Re:Errr... (1)

Wulfstan (180404) | more than 8 years ago | (#15894788)

Yeah, but my point is that the parent and the site he referenced claimed that Purify was a static analysis tool; it isn't.

Re:Errr... (2, Insightful)

astralbat (828541) | more than 8 years ago | (#15894808)

As an object oriented programmer, I always follow the general rule of having a function always give the same output for the same inputs. That is, you then don't have to worry about the 'state' of an object and you as a result have fewer paths to test and fix. This is why, IMO, global variables aren't such a good thing unless they are constant/rarely change.
This should be common knowledge to a good object oriented programmer, but I wonder how often it's employed in the 'C' discipline.

Re:Errr... (4, Insightful)

John Nowak (872479) | more than 8 years ago | (#15894841)

A function that always returns the same value given its inputs is part of functional programming, not object-oriented programming. Most OO code is littered with side-effects and state-dependent behaviour. If you like to program in such a way, you may find yourself much more comfortable with a functional programming language. Languages like Haskell even enforce this.

Re:Errr... (1)

Chandon Seldon (43083) | more than 8 years ago | (#15894905)

That works great until you want to do something really off the wall... like input.

Re:Errr... (2, Interesting)

twiddlingbits (707452) | more than 8 years ago | (#15894954)

Good programming practice says ANY function should give the same outputs ALL the time for the same inputs (i.e if you put in a 2 today you get out a 4 and the same thing tomorrow). What you seem to be talking about are "side effects" where a global variable or input parameter is modified within the context of a function. Some programming languages DO allow you to change the value of a parameter within the function and that result is passed back to the caller. In fact thats easy to do in C with pointers. Harder to do in other languages. Either way IMHO, it's a horrible programming practice. The hardest thing I ever saw was a bunch of C programmers trying to learn how to code in Ada. All the "shortcuts" they used to use were removed by strong typing and strict rules. Testing of OO code where you are changing the internal state of an object via one of it's methods or via another method (such as in C++) makes things a LOT harder to develop good tests for and I would suspect good code analysis tools.

Re:Errr... (1)

kemo_by_the_kilo (971543) | more than 8 years ago | (#15894510)

If they gave it, its own source wouldnt that be like use playing with DNA and stem cells? -Kemo

Re:Errr... (1)

It'sYerMam (762418) | more than 8 years ago | (#15895004)

No. Because the program is not alive

What are they teaching kids nowadays?

Re:Errr... (1, Interesting)

Anonymous Coward | more than 8 years ago | (#15894570)

I hope these Coverity guys aren't pompous enough to think that their tool can find ALL bugs in a program with... magic...

I should hope not, as that is demonstrably false. For example, at one point the KDE project with its I-don't-know-how-many-millions of lines of code had a coverity rating of 0 open bugs, but I'm sure no one is silly enough to think that such a large and complex project has no bugs at all!

Most static analysers look for very simple, easily machine-detectable, low-level imperfections which could conceivably lead to hard-to-spot bugs - not initialising a variable before it is used is probably the classic example of the kind of "bug" that would be detected by an analyser such as Coverity. I imagine Coverity is quite a lot more sophisticated than that, though :)

Re:Errr... (1)

vertinox (846076) | more than 8 years ago | (#15894678)

Hmm, they should run their tool on its own source code, that would be fun.

And if they figure out how to get the tool to modify and improve its own code we'll have Strong AI.

Re:Errr... (1)

Millenniumman (924859) | more than 8 years ago | (#15894933)

Tools like this aren't for perfecting software. They will often find a lot of bugs so that QA doesn't have to. The software will still need to be tested by a person, but some of the work will already have been done. The best way to find bugs is to have a person see if they can break the software.

this slashdot news is already outdated (5, Informative)

msh104 (620136) | more than 8 years ago | (#15894444)

if you look at the coverity site ( http://scan.coverity.com/ [coverity.com] ) you will see that there are already multiple projects who have brought there bugs down to zero. samba being on of the earliest.

Re:this slashdot news is already outdated (5, Insightful)

StrawberryFrog (67065) | more than 8 years ago | (#15894477)

there are already multiple projects who have brought there bugs down to zero.

You mean "who have brought down the count of their bugs that this tool can detect down to zero." I'm sure they will have other bugs in code and design.

How does this tool compare to tools that do analysis by introspection on bytecode from languages like C# and Java. I use FxCop [gotdotnet.com] on C# code, and while it is very cool, using it is not newsworthy at all. Does this tool do more? Is is the news that it's used in a high-profile C++ program?

Integrating tools like this into your build process may be cutting-edge best-practice at present, but give it a while.

Re:this slashdot news is already outdated (1)

Monkelectric (546685) | more than 8 years ago | (#15894538)

Yes. Automated testing/code inspection is handy, but a signifigant source of bugs is actually missing logic paths, or combanatorics errors which can't be detected by tools generally.

Re:this slashdot news is already outdated (4, Interesting)

bcrowell (177657) | more than 8 years ago | (#15894540)

You mean "who have brought down the count of their bugs that this tool can detect down to zero." I'm sure they will have other bugs in code and design.
Yeah, if they could make a program that would detect all bugs in a program, it would violate Turing's proof that the halting probelm is undecidable. [wikipedia.org]

From the articles, it sounds like they're basically looking for mistakes that could lead to security flaws, e.g., buffer overflows. If AMANDA is particularly buggy by their metric (detectable bugs per thousand lines of code), it's probably because AMANDA doesn't interface to the web, so the people coding it knew that certain classes of buffer overflow "bugs" wouldn't be a problem, because they wouldn't be exploited through an internet-facing interface. If you went back and ran this program on Unix apps written in C from the 1980's, you'd probably find zillions of bugs, but it wouldn't indicate low quality, it would just mean that the programs weren't written for an internet-facing environment in the year 2006, when the internet has become a battle zone for evil spammers, botnets, etc. If the only way such a bug can show up is for the user to supply carefully tailored input, and the result is simply that the program dumps core, then that's not a bug for a program that isn't facing the modern internet.

Computer != Turing machine (2, Informative)

tepples (727027) | more than 8 years ago | (#15894584)

if they could make a program that would detect all bugs in a program, it would violate Turing's proof

Unless the program's domain is restricted to context-sensitive languages. In fact, it is impossible for a computer to try to decide anything more general than a context-sensitive language because anything bigger requires the resources of a Turing machine, which has infinite memory. Computers implementable in a finite amount of matter are equivalent to linear bounded automata [wikipedia.org] , not Turing machines.

So? (1)

MarkusQ (450076) | more than 8 years ago | (#15894775)

You say this as if it invalidates his point. Since (as you would obviously agree) no computer is more powerful than a Turing machine, if something is impossible for a Turing machine it is necessarily impossible for a computer as well. If anything, your quibble makes his argument stronger.

--MarkusQ

Re:So? (1)

Profane MuthaFucka (574406) | more than 8 years ago | (#15894974)

But it DOES invalidate the point made. It is possible to write a computer program that can tell me if ANY given program on any specific computer will have a bug or not. Every single one, perfectly. Do you disagree? Because you'd be completely wrong.

Re:this slashdot news is already outdated (1)

camcorder (759720) | more than 8 years ago | (#15895017)

Have you ever heard of local exploits? You don't need to have internet to be vulnerable. Applications should prevent from buffer overflows because there can be local attacts as well. You would not want a local user to crash your public workstation or get root on it do you?

Re:this slashdot news is already outdated (2, Interesting)

Vlad_the_Inhaler (32958) | more than 8 years ago | (#15894586)

No major piece of software is ever bug free.

I follow the news:linux.samba [linux.samba] Newsgroup a bit. Various Samba features have been shipped broken in various recent releases.

CIFSfs? (it is replacing smbfs and some Linux distributions have taken to disabling smbfs in the kernel to force people to switch) Cifsfs was broken in the newest major release. An intermediate release fixed that.
'Valid Users' used with 'smbpasswd': that was broken in the intermediate release. The next intermediate release will cover that.

No major piece of software is ever bug free, at least the Samba guys are very responsive to error reports.

Re:this slashdot news is already outdated (2, Informative)

Almost-Retired (637760) | more than 8 years ago | (#15894774)

Thank you for that link, it rather nicely confirms that at the time amanda was inspected last, it was clean.

--
Cheers, Gene

Interesting... (2, Interesting)

porkThreeWays (895269) | more than 8 years ago | (#15894451)

I find the AMANDA results interesting because AFAIK it hasn't recieved a code rewrite since the early 90's. I think an interesting study would be the to compare older projects with ones that have been rewritten from the ground up. Comparing the rate of new bugs introduced as opposed to those hidden in legacy code.

AMANDA _is_ in active development (2, Informative)

Noksagt (69097) | more than 8 years ago | (#15894643)

zmanda [zmanda.com] and other amanda hackers [yahoo.com] have been actively developing AMANDA. While the comparison of bugs in new code and legacy code might be interesting, one wouldn't really see this by just counting projects.

Re:Interesting... (1)

Almost-Retired (637760) | more than 8 years ago | (#15894820)

Not in the least bit interesting. Amanda is under constant development, and to say that it hasn't been touched since the 90's is an outright lie. You work for Veritas or Arkiea?

The current stable snapshot is amanda-2.5.0-20060424..tar.gz. And there are, sitting on my hard drive, about 12 versions of the next generation that will lead to a 2.5.1 release in a month or so.

Please put brain in gear and go check your so-called facts before spouting off in front of nearly a million /. readers and making yourself look like somebody with a commercial axe to grind, bad-mouthing the open source efforts because it might put another sale of outragiously priced but dis-functional backup software in your bottom line. Or worse yet, an idiot...

--
Cheers, Gene

Re:Interesting... (0, Flamebait)

porkThreeWays (895269) | more than 8 years ago | (#15894912)

Jebus, I usually don't respond to idiots like you, but jeez you are a fuckin idiot. I said a code rewrite. As in they scrap the current code base and rewrite huge portions of it. Sendmail has done this at least once in the past few years and so did the mach kernel project. Never once did I say amanda wasn't currently being developed.

Bug/Lines of Code (5, Funny)

X43B (577258) | more than 8 years ago | (#15894460)

"It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code."

Sounds like someone needs to run this debugger on their calculator.

Re:Bug/Lines of Code (1)

Alsee (515537) | more than 8 years ago | (#15895001)

Oh, their calculator software is fine. They were running it on a Pentium.

-

Once detected can they also fix the flaws? (3, Funny)

Browzer (17971) | more than 8 years ago | (#15894476)

Or that job is left for the monkeys banging on the keyboards.

Check the checker (0, Redundant)

Mini-Geek (915324) | more than 8 years ago | (#15894496)

But what checks Coverity for bugs? Coverity? What if it has a bug-checking bug causing it to not see its own bug.

Re:Check the checker (1)

mad.frog (525085) | more than 8 years ago | (#15894636)

The real question is, what happens if they run it on itself and it reports that it DOES have bugs? Suddenly we're in "this statement is false" territory...

AMANDA is cross-platform (1, Interesting)

Anonymous Coward | more than 8 years ago | (#15894511)

Amanda works on many unix and unixoid operating systems, it's not a "linux" backup system. It's used primarily for driving remote backups to big tape libraries, most /. reading linux users would never have systems large enough to justify its use. :-)

Amanda IS, however being very actively developed right now, lots of new features -> lots of new bugs. Other issue is that it's a componenty, plugin architecture, made of a few processes communicating over pipes and sockets. A failure in one component won't necessarily be a security risk or take the whole system down, it's extremely robust in normal operation in my experience, despite this "high bug count". Unlike XMMS, various contributed plugins (e.g. tape changer robot drivers) are redistributed in the source tarball but only used by very small numbers of people with outlandish hardware.
I suspect if you included various XMMS plugins in the XMMS count, things would be different...

None of that *really* excuses a high bug count - but what really pisses me off is coverity's "we've found X bugs, but we're not going to tell you what they are or substantiate our claims (some of amanda is quite old code, has a lot of strcpys, I know that some automated security checkers will treat a strcpy as a "bug" even if it's safe), just FUD your project in various public fora...

Re:AMANDA is cross-platform (1, Insightful)

Anonymous Coward | more than 8 years ago | (#15894556)

Uh. Coverity TOLD the amanda folk, bugs were fixed, now amanda's fixed all coverity bugs found ?! Score -1 FUD...

Which type of bugs? (3, Informative)

Erixxxxx (920617) | more than 8 years ago | (#15894514)

One has to wonder if these are coding/language bugs or logical bugs. Finding coding bugs is of course a valuable time saver, but the challenging and usually most costly bugs are of the logical sort, and invariably app specific.

agreed (0)

Anonymous Coward | more than 8 years ago | (#15894558)

static analysis only gets one so far. generally it's the higher order logic that gets reviewed in code reviews, I would hope, though.

"Meh. So much for the 'many eyes' theory" (4, Insightful)

rjamestaylor (117847) | more than 8 years ago | (#15894531)

Even more interesting is the DHS initiative for Coverity to use this same bug detection software on 40 open source projects.
Before the F/OSS nay sayers toss out the obligatory (and to be expected) "Meh. So much for the 'many eyes' theory" let's point out that having the ability to run a code checker on source code is only possible to the holders of said source code. So, while absolutely true that a proprietary vendor can run the code checker on their code as well as an open source project, there is a huge difference when it comes to the customer/user of said software: with Open Source the user has the freedom to run such a tool over the source code themselves.


In this age of SarbOx and risk management there is a real competitive advantage to F/OSS over proprietary code to large companies: audit-ability. In previous roles I've had to attest under HIPPA::Security that proprietary code was "secure" -- how? All I could do was obtain a vendor statement that was as non-commital and burden-shifting as possible. Yet, with a true ability to audit the code my pharmaceutical company depended on it would tilt the balance between similar-featured Closed vs Open source solutions. Especially today.

Ok, maybe nobody really cares about the 'many eyes' theory anymore. Regardless, the "open the hood" theory still applies, perhaps more than ever.

Automatic Exploitation (1)

kripkenstein (913150) | more than 8 years ago | (#15894975)

So, while absolutely true that a proprietary vendor can run the code checker on their code as well as an open source project, there is a huge difference when it comes to the customer/user of said software: with Open Source the user has the freedom to run such a tool over the source code themselves.

Actually, I would argue that it isn't just a freedom, it's a necessity. Having the source open means that wrongdoers can use bug-seeking programs to find exploits (presumably they have already been doing so for a while). So just to even things out, the Open Source community should scan them as well. This issue shouldn't be ignored.

Of course, closed-source programs are also being scanned by exploit-seeking software (it's not too hard to e.g. search for all calls to strcpy in a binary). So this isn't a 'new' problem with open-source projects. But, with the advantages of Open Source come a few risks, which we should deal with, as mentioned in the previous paragraph.

Meanwhile... (3, Funny)

Skiron (735617) | more than 8 years ago | (#15894537)

Coverity segfaulted whilst auditing MS Vista.

Re:Meanwhile... (3, Funny)

HRogge (973545) | more than 8 years ago | (#15894555)

Run it against Windows ME and you will get an interger overflow.

Re:Meanwhile... (2)

kevlarman (983297) | more than 8 years ago | (#15894976)

Run it against Windows ME and you will get an interger overflow.
twice!

For those who are interested in Firefox' results (5, Interesting)

RebelWebmaster (628941) | more than 8 years ago | (#15894539)

Here are some links to show the bugs in the Bugzilla database which were turned up by Coverity.
Open Coverity Bugs [mozilla.org]
All Coverity Bugs [mozilla.org]

Re:For those who are interested in Firefox' result (1)

Bazouel (105242) | more than 8 years ago | (#15894666)

I took a look at the bugs and funny enough, almost all of these would be immediately catched by the C# compiler or would be non issue (memory leaks). The remaining others (and much more) would be detected by FxCop. And then, there is still Spec# ! Like someone said before, the real hard bugs are logical. So really, one question : how is this newsworthy ?

The only thing that this teach me is that a language/platform that allow better typing, memory management and static analysis is far far more robust and productive in the end.

Re:For those who are interested in Firefox' result (0)

Anonymous Coward | more than 8 years ago | (#15894726)

The only thing that this teach me is that a language/platform that allow better typing, memory management and static analysis is far far more robust and productive in the end.

Haskell.

Is this the best way? (0)

Anonymous Coward | more than 8 years ago | (#15894565)

I have to admit I am not upto speed on the best C and C++ testing suites, but is Coverity's product the best for this kind of thing?
I know that Parasoft has a solution in their product Insure++, which gives runtime analysis, not the static code analysis that Coverity appears to give from my short look at their website.

Any one else with better knowledge on these types of products wish to comment further?

Why AMANDA is buggy (2, Insightful)

swordgeek (112599) | more than 8 years ago | (#15894589)

AMANDA could easily be the buggiest OSS program in existence, and it would still be OK. The reason? It just has to be less buggy than Netbackup, and more usable than Legato. Luckily for the AMANDA developers, this are very very difficult criteria to miss.

AMANDA (and others) have fixed the defects (1)

Noksagt (69097) | more than 8 years ago | (#15894620)

Coverity's own site [coverity.com] shows how many defects each product has fixed. the number of outstanding defects on AMANDA is now zero. zdnet reported the fixes back in April. [zdnet.com]

Those that follow amanda-hackers will know that there was less than a week [yahoo.com] between when coverity released the report on March 6th and it was announced that all bugs were fixed in AMANDA on March 12th.

I dislike the idea of Coverity (1, Interesting)

Myria (562655) | more than 8 years ago | (#15894628)

Coverity sounds like a scam. It is not possible for a program to analyze another program and find all the bugs; see halting problem [wikipedia.org] .

I would find heuristic analysis annoying. I'd get quite annoyed if the program says "fix this buffer overflow" 1000 times because I use "strcpy" somewhere - even though I'm very careful and only use it when I know it can't overflow.

I should write a program that searches for odd perfect numbers [wikipedia.org] and terminates if it finds one. I wonder whether Coverity would say it is an infinite loop.

Coverity sounds like scare tactics to make money by claiming to do the impossible. They won't even disclose what their algorithm is. I would never trust them, especially on closed-source programs. Firefox doesn't have that risk, but they are wasting money.

Microsoft's PREfast is simpler but seems like a much more realistic solution: mark up your code to say how things are supposed to be used and the compiler can decidably sense problems. I'd just get tired of typing 2 underscores a million times.

Melissa

Re:I dislike the idea of Coverity (-1, Offtopic)

Anonymous Coward | more than 8 years ago | (#15894727)

Coverity sounds like a scam. It is not possible for a program to analyze another program and find all the bugs; see halting problem.


Listen Myria, just run your project through this Department of Homeland Security sponsored program and it will "fix" all of the "bugs". Don't ask questions, or THE BAD GUYS WIN!!!

Re:I dislike the idea of Coverity (4, Insightful)

Animats (122034) | more than 8 years ago | (#15894737)

It is not possible for a program to analyze another program and find all the bugs; see halting problem .

Wrong. It is quite possible to analyze a program and find all the bugs that violate the language constraints (null pointers, buffer overflows, etc.). That's what program verification is for. For some programs, you can't tell whether a bug condition will occur, so you treat that as a bug.

Automated program verification is a good idea that went away because C and C++ have such ambiguous semantics. It's hopeless for those languages. The "pointer equals array" concept alone makes it very tough, because the language has no idea how big an array is. Worst idea in the language, and the root cause of buffer overflows.

Good verifiers were written for Pascal (I headed one of those projects [animats.com] ), a good one was written for Java [dec.com] (at DEC, just before DEC went under), and Microsoft is working on one for C#. [microsoft.com]

Re:I dislike the idea of Coverity (0)

Anonymous Coward | more than 8 years ago | (#15894958)

It is not possible for a program to analyze another program and find all the bugs; see halting problem .
Wrong. It is quite possible to analyze a program and find all the bugs that violate the language constraints (null pointers, buffer overflows, etc.).
He's not wrong. You disagree with his assertion and then build a straw man by changing "bugs" to "bugs that violate the language constraints." Stop being such an asshole.

The halting problem is not an issue (5, Informative)

Animats (122034) | more than 8 years ago | (#15895047)

The halting problem is not an issue for program verification. This claim is raised repeatedly by the clueless, and it just isn't an issue.

Yes, you can construct a program that's formally undecideable. It's a hard way to write a bad program. It takes some work, and the resulting program is unlikely to be useful.

Most crash-type and security-hole problems in programs are entirely decidable. This is because almost all subscript calculations are composed from addition, multiplication by constants, and logic operations. Those are totally decideable, and there are good decision algorithms for that problem. Only when multiplication of two variables (both non-constant) is introduced can formal undecidability appear. See Presburger arithmetic [wikipedia.org] .

In fact, halting is decidable for all deterministic machines with finite memory. Either you repeat a previous state, or halt within a finite number of cycles. The decision process may be made arbitrarily hard, but that's not undecidability. True undecidability in the Turing sense requires infinite memory.

Most of the practical problems with program verification come from dealing with interactions between various parts of the program. Containing those interactions well enough that you can localize problems is constraining on the programmer. "Design by contract" languages like Eiffel try to do that, but they're not popular. Retrofitting design by contract into C and C++ has been discussed, but the proposed schemes all have holes you could drive a truck through. A big truck.

Although software work seldom uses proof of correctness techniques, there's a whole industry doing it for hardware. There was a machine-generated formal proof of correctness for the FPU in AMD's K7 processor. [onr.com] AMD thus avoided the "Pentium division bug".

Re:I dislike the idea of Coverity (1)

buswolley (591500) | more than 8 years ago | (#15894804)

It doesn't claim o do the impossible. It doesn't claim to find ALL the bugs. That would be ridiculous. However, finding bugs is not impossible.

Re:I dislike the idea of Coverity (4, Informative)

anpe (217106) | more than 8 years ago | (#15894850)

If you'd followed the lkml, you could have seen actual patches fixing real bugs, found by Coverity. Just run this search on google: "by coverity" patch site:lkml.org to convince yourself.
The fact that it is impossible to solve the whole problem of program correctness and that false positives will come up doesn't mean that the problem Coverity is adressing isn't usefull.

Regards,

Static code analysis does help (0)

Anonymous Coward | more than 8 years ago | (#15894868)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Melissa wrote:
> Coverity sounds like a scam.

I used Coverity to scan for bugs in the development release of imap-uw
and in nfs userspace components a couple of months back. For instance:

http://sourceforge.net/mailarchive/me ssage.php?msg_id=20590002

Most turned out to be legitimate bugs that would likely remain unfixed
today were it not for my use of the static code analysis tool.

Mike

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iQEVAwUBRN4N3NtAhTFtyodpAQPL4gf9Gpp XRN2Cf5Pp0KOrhNb6GHN9kgEBMMc4
ShVXSYsAQk2kcPnz97I JPQgkEmhT4RBXslkSAjMfYZ1Xg6DEuo9jwKt//RcScL5U
da2 Aueo2TiRlXoo0bHRujATgzP+U29nLIXIM4EdtEJo99vH2eg1HZ ka82hNtpN7u
2AF791miIEpRY5vaKxnT09juoXKrgXoXx0+71 8bBkQP6sLEAMrHz/XGKkuqRR+dG
SFRnuEUcZr16frHR4L7C9 Fn8UvI55iDW07pQVy2fC+YDJ7ZQnIwpfANUEdnWbRuV
eHyPn xPrOwm2CMaRZsNZq8odK1pTZCxHsK+zjYL3wkmUvLCyr+Z8eA= =
=fZMZ
-----END PGP SIGNATURE-----

(drop the space in ``message'' to follow link and verify sig)

That's silly (1, Insightful)

TheLink (130905) | more than 8 years ago | (#15894944)

"Coverity sounds like a scam. It is not possible for a program to analyze another program and find all the bugs"

What a silly reason! How about gzip etc then?

"gzip sounds like a scam. It is not possible for a program to analyze any data and always compress it successfully"[1].

I could go on: "life sounds like a scam..."

But I suggest you wake up to the harsh imperfect real world some time and leave that sort of thinking to the run-of-the-mill "academics".

How you deciding whether Coverity is good or not should be like how you decide whether gzip is good or not. If Coverity doesn't find bugs better than even gcc then it probably useless to most people.

[1] On a related note, in my opinion programming can be viewed as a type of compression.

Wasn't Coverty basically stolen from the community (0)

Anonymous Coward | more than 8 years ago | (#15894717)

They built their tool on gcc, and never gave any source back to the community. "But they didn't distribute it", you say. Really?

All this ''helping open source'' is just their giant guilt trip.

Follow the leader (0)

Anonymous Coward | more than 8 years ago | (#15894723)

Yawn. Microsoft has been doing this for years already.

Maybe if Firefox were written properly the first time around, blah blah blah.

No rsync? (2, Interesting)

ortholattice (175065) | more than 8 years ago | (#15894756)

Funny selection of programs; I don't see rsync on the list. From the article: DHS wants to reinforce the quality of open-source programs supporting the U.S. infrastructure. So, XMMS (an MP3 player) is more important to the U.S. infrastructure than rsync?

Re:No rsync? (0)

Anonymous Coward | more than 8 years ago | (#15894809)

well, not more important, but definately used more often. :/

In defense of amanda (2, Informative)

Almost-Retired (637760) | more than 8 years ago | (#15894757)

I'm somewhat surprised to see amanda being badmouthed here by this tool. It was mentioned on the amanda-users list a few months back that the amanda tree had been checked by coverity, and the 2 bugs coverity found were promptly fixed.

Thats not to say that as new features are added, new bugs haven't been too, but to actually call amanda a truely buggy application does stretch this users belief a wee bit. I'm currently running a 20060424 dated snapshot of the 2.5.0 tree, with no hiccups at all.

--
Cheers, Gene

So then ... (1)

iknowcss (937215) | more than 8 years ago | (#15894776)

can they run these programs on the source of the program itself to look for bugs? Or would that be like the human brain being able to completely understand itself inside and out (aka. not possible)

Wake up Mozilla (0)

Anonymous Coward | more than 8 years ago | (#15894825)

I just care for one thing that mozilla has been ignoring since FF 1.0.x...
Will this make Firefox any faster?

WTG XMMS (0)

Anonymous Coward | more than 8 years ago | (#15894878)

props! No wonder I keep always using it, after trying all the other audio media players out there. It "just works".

VMware Uses It Too (1, Informative)

Anonymous Coward | more than 8 years ago | (#15895042)

We were using it since it was the Meta Compiler. I believe we had some interns from the project. They used our codebase to research their algorithms and we got free scanning. We may well be using the Coverity commercial code today.

Firefox is again the most unstable program... (1)

Futurepower(R) (558542) | more than 8 years ago | (#15895043)

Firefox is, once again, the most unstable program in common use [slashdot.org] .

The 1.5.0.4 version of Firefox was quite stable, if the Flashblock extension was installed. The 1.5.0.6 version is unstable again. The CPU-hogging bug is back!

This comment posted from a copy of Firefox that is constantly using 2.8% of the CPU, even when all pages have been loaded, and there is no active content. That's 2.8% on the way to 70% or more, making it necessary to close Firefox and reboot Windows XP.

There are some bugs found by Coverity [mozilla.org] left unfixed, but so far things have gotten worse since 1.5.0.4, not better.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>