Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

US DHS Testing FOSS Security

kdawson posted more than 6 years ago | from the bug-list-half-empty dept.

Security 203

Stony Stevenson alerts us to a US Department of Homeland Security program in which subcontractors have been examining FOSS source code for security vulnerabilities. InformationWeek.com takes a glass-half-empty approach to reporting the story, saying that for FOSS code on average 1 line in 1000 contains a security bug. From the article: 'A total of 7,826 open source project defects have been fixed through the Homeland Security review, or one every two hours since it was launched in 2006 ...' ZDNet Australia prefers to emphasize those FOSS projects that fixed every reported bug, thus achieving a clean bill of health according to DHS. These include PHP, Perl, Python, Postfix, and Samba.

cancel ×

203 comments

Sorry! There are no comments related to the filter you selected.

Stop, colleberate and listen! (-1, Troll)

Anonymous Coward | more than 6 years ago | (#21963628)

Hammertime! [thepounder.com]

What about MS? (5, Funny)

Anonymous Coward | more than 6 years ago | (#21963638)

Now if they would do the same to Microsoft. Oh yeah...

Re:What about MS? (1)

Anonymous Coward | more than 6 years ago | (#21964622)

Silly question. It simply means DHS and other Federal agencies will have more reason than ever to dump MS and switch to FOSS.

Re:What about MS? (4, Interesting)

filbranden (1168407) | more than 6 years ago | (#21964652)

Actually, it would be really nice if it was possible to do it with Microsoft. Microsoft (or most other companies that produce proprietary software) certainly can't do better than what the open source projects do, and certainly their code contains at least as much issues as the ones found in open source projects.

The ability to do code audits has always been one great advantage of open source software, but until now, it was mostly in theory. Now we start to see big code audit projects such as this one, showing that the advantage is real and that the results of the audit are good, since some of the projects have alread patched all of the issues, and certainly most of others will finish patching them soon. This shows that open source is here to stay, is going mainstream, and will not be stopped by any company's interests.

All issues that currently exist on Microsoft's code, on the other hand, will be unpatched. Unless they hire some consultant company (why not the same?) to do the audit on their code (certainly under NDA). But you can be sure that, if they do, for one, they won't publish the results of how many issues were found. No transparency there. And also, probably many issues won't be fixed as promptly as all of them were fixed in many of the audited open source projects. This is not a speculation, if you only look at how long it takes for them to fix issues for which there are security vulnerability reports issued, then you realise that the ones only they know about will certainly take much longer.

Re:What about MS? (2, Informative)

Shados (741919) | more than 6 years ago | (#21964844)

Well, technically, they don't need to -hire- some consultant companies to do it... While it WILL be under extreme DNA, it is not uncommon for Microsoft's customers to be allowed to get access to the source, if they're big enough.

Now, I realise it doesn't change your point at all, but its not like MS is the only entity with access to their own code: they have dedicated programs to share even their most closed pieces of code with their customers (if they're important enough).

Wow... FOSS looks pretty pathetic (0, Troll)

Anonymous Coward | more than 6 years ago | (#21964710)

So in other words, this thing started in 2006. So if Big Daddy Gubment had not come by with what's essentially a bailout of FOSS, it would STILL be a buggy mess.

Kind of hilarious, how no matter how much of an insecure, buggy, crappy mess FOSS proves to be, they still whine about Microsoft.

Guess it's easier to point the finger than it is to get your own house in order.

Re:Wow... FOSS looks pretty pathetic (5, Informative)

mr_mischief (456295) | more than 6 years ago | (#21965160)

There are industry estimates that say average code in production contains 2 bugs per thousand lines of code. Some say that number is much higher. How many lines do you think are in Vista?

Yes, OSS has bugs. Everything from compilers to content management systems, surely. So do proprietary programs.

The more qualified eyes you get on a bug, the better chance you have of finding and fixing it. You can do that by having a big staff that pores over code again and again. You can do it by having lots of outside help, like in the case of popular OSS projects. One thing that helps is to have a fresh set of eyes look over something, which is much easier in OSS that in closed-source applications.

BusinessWeek had an article from a guy at Coverity back in 2006 about this. In that article [businessweek.com] , Ben Chelf said that 4 of the top 15 programs on the quality scale measured by defects per thousand lines of code were OSS. He said that on average, the major-project OSS software they tested was indeed higher quality software than average. He said, though, that the absolute highest quality code was the cream-of-the-crop proprietary, closed source code from places that make things like fly-by-wire systems. Well, yeah. I'd want my airliner's fly-by-wire system completely bug-free, too.

Commercial software tends to harbor anywhere from 1 to 7 bugs per 1000 lines of code according to the National Cybersecurity Partnership's Working Group on the Software Lifecycle [zdnet.com] . Voluntary testing by Coverity requested (and probably paid for) by MySQL AB revealed that project to have all of 97 flaws, one of which could be a serious security issue. All 97 were to be fixed for the next release.

A similar study (same link) found 985 bugs in over 5,700,000 lines in the Linux kernel, or fewer than one bug per 10,000 lines of code. TFA has data on a newer version of the kernel -- 0.127 bugs per TLOC.

In Apache, 22 bugs total, 0.14 per TLOC, and three fixed so far.

PostgreSQL had 0.041 per TLOC, and have so far fixed 53 of the 90 bugs.

The glibc team fixed 83 of 83 bugs found.

OpenVPN had found one security-related bug in over 69,000 lines of code. As of later yesterday, it's officially security bug free according to the same testing people.

The list of officially security-bug free software [zdnet.com.au] includes Amanda, NTP, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and TCL.

So with Linux (0.127), glibc (0.000), Apache (0.140), PostgresSQL (0.041), Perl (0.024), PHP (0.000), and Python (0.000) powering a web server (numbers according to Coverity [coverity.com] ), you have 0.0474 defects per thousand lines of code across the server. I'd say that's pretty good.

"The" PHP? (1, Funny)

sticks_us (150624) | more than 6 years ago | (#21963658)

I stopped reading after they called it "The PHP."

Re:"The" PHP? (3, Informative)

JorDan Clock (664877) | more than 6 years ago | (#21963770)

..the PHP, Perl, and Tcl dynamic languages...
"The" in this sentence refers to the list, not just PHP.

Re:"The" PHP? (5, Funny)

grcumb (781340) | more than 6 years ago | (#21963828)

..the PHP, Perl, and Tcl dynamic languages...
"The" in this sentence refers to the list, not just PHP.

How could he possibly know that? He said already that he stopped reading after 'the PHP'.

/me ducks and runs...

Re:"The" PHP? (1)

sticks_us (150624) | more than 6 years ago | (#21963878)

This language needs parenthesis, or some better documentation on precedence. I parsed each item in the clause as:

(Samba) (the PHP) (Perl) (Tcl dynamic languages) (Amanda) ...when I guess the intent was

(Samba) the (PHP, Perl, Tcl dynamic languages) (Amanda)

Re:"The" PHP? (0)

Anonymous Coward | more than 6 years ago | (#21963938)

Job.Good = True

Re:"The" PHP? (1, Funny)

Anonymous Coward | more than 6 years ago | (#21964020)

So close. Lets turn those into a proper Tcl list, shall we...

set thislist {Samba} {the PHP} {Perl} {Tcl dynamic languages} {Amanda}

Re:"The" PHP? (2, Funny)

grcumb (781340) | more than 6 years ago | (#21964224)

So close. Lets turn those into a proper Tcl list, shall we...

set thislist {Samba} {the PHP} {Perl} {Tcl dynamic languages} {Amanda}

No, I think he's deliberately speaking with a LISP.... 8^)

Re:"The" PHP? (1)

Unoti (731964) | more than 6 years ago | (#21964046)

Languages like And Such, and the PHP.

Re:"The" PHP? (2, Funny)

bladesjester (774793) | more than 6 years ago | (#21964742)

Languages like And Such, and the PHP.

Security and computer science as explained by a valley girl?

Like totally!

Re:"The" PHP? (1, Insightful)

Anonymous Coward | more than 6 years ago | (#21964078)

PHP essentially stands for "Hypertext Preprocessor", if you ignore the "recursive initialism".

"The Hypertext Preprocessor" sounds as reasonable to me as "The Windows Operating System".

Re:"The" PHP? (1)

Nullav (1053766) | more than 6 years ago | (#21964328)

"The Windows is broken"?

Re:"The" PHP? (2, Funny)

Anonymous Coward | more than 6 years ago | (#21964522)

Learn grammar: "The Windows ARE broken", since all of them are.

Fixed? (5, Funny)

sjbe (173966) | more than 6 years ago | (#21963664)

A total of 7,826 open source project defects have been fixed through the Homeland Security review


Do they mean fixed [wikipedia.org] or fixed [wikipedia.org] ?

Re:Fixed? (0)

Anonymous Coward | more than 6 years ago | (#21964564)

A total of 7,826 open source project defects have been fixed through the Homeland Security review
Do they mean fixed or fixed?
Well since repairing them results in them no longer being vulnerable from that vector, I'd say both.

Looking good, too bad the press didn't understand (5, Insightful)

Bruce Perens (3872) | more than 6 years ago | (#21963670)

The important point here is that proprietary software manufacturers aren't telling you how many security flaws they had. I bet it's more than 1 per 1000 lines, that is an incredibly excellent figure for the first time a scanner like coverity is run. I doubt proprietary work comes close.

You can't ever say that proprietary software is secure, because there's no way to prove it. With Open Source, you can come a lot closer to proving that it is secure, because you can employ every security test that exists.

The fact that a coverity scanner bug is reported doesn't mean it's an exploitable security flaw.

Bruce

Re:Looking good, too bad the press didn't understa (5, Insightful)

QuantumG (50515) | more than 6 years ago | (#21963814)

Although I understand what you're trying to say, it does seem a little irrelevant.

I'm a software security engineer. I can look at source code and tell you if it has some bugs in it that I would consider relevant to security. If I can't find any, I might tell you that it is more secure than if I could... but that's doesn't mean it is secure. I'll never tell you it is secure, because testing simply can't give you that. I can do this on proprietary software or I can do this on Open Source software.. the only difference is that, with the Open Source software, I don't need permission from someone to do the testing and other people don't need permission to check my work.

Does this mean that more people will check the Open Source software for security flaws? Not necessarily. It completely depends on whether or not someone has an interest in the security of that particular bit of software. Even assuming a similar level of interest in the security of comparable proprietary and Open Source software, there's no guarantee that those who have an interest in testing the Open Source software for security flaws will report back the findings. They may simply decide that the Open Source software is too insecure for their use and go with the proprietary solution - assuming they can have it similarly tested by a trusted third party.

All in all, the assumption that Open Source software is more secure than proprietary software is most likely true, but there's no hard data.. because the stats on the insecurity of proprietary software are guarded secrets - and that's probably the best reason to assume that proprietary software is less secure.

Re:Looking good, too bad the press didn't understa (0)

cromar (1103585) | more than 6 years ago | (#21964192)

Does this mean that more people will check the Open Source software for security flaws? Not necessarily.

It sure is nice to be able to do it if/when you feel like it, though!!

Re:Looking good, too bad the press didn't understa (4, Insightful)

Bruce Perens (3872) | more than 6 years ago | (#21964358)

Does this mean that more people will check the Open Source software for security flaws? Not necessarily. It completely depends on whether or not someone has an interest in the security of that particular bit of software.

I submit that people who are only looking for security flaws don't have a motivation to develop a deep understanding of the software. People who are out to modify the software do. And thus there are not just more eyes, but better eyes with Free Software.

There is a class of mathematically provable software languages, and you might be able to say with surety that programs in them are secure. For the languages we usually use, you can only say that you have tested them in the ways you know of. And only a person with access to the source can say that. If you want an independent asessment, Open Source software won't stop one from happening, and won't hinder what can be said with NDAs. That's why I think it's more secure.

Bruce

Re:Looking good, too bad the press didn't understa (1)

netcrusher88 (743318) | more than 6 years ago | (#21964418)

Although I understand what you're trying to say, it does seem a little irrelevant.

I don't really see how it's irrelevant - if a "security defect" exists but cannot be exploited (i.e. if there's a buffer overflow bug but it deals with internal data or data that's already been thoroughly sanitized), it does not present the same risk as a bug that may be easily exploited, for example in the input sanitizing code. It's not really clear how many of these bugs are of each type, and I think it's significant that the phrase "security defect" was chosen instead of "security hole" or some other phrase that is more commonly used for known significant risks.

Of course, "cannot be exploited" is relative - no matter what you do, there's always a real possibility that someone's going to come up with a creative way to get their data in the wrong place at the wrong time and break all your nice sanitizing code and layers you've erected over that heavily protected buffer overflow, so the real resolution is of course to fix it. Still, I think it's an important distinction, especially when dealing with statistics.

Re:Looking good, too bad the press didn't understa (1)

samkass (174571) | more than 6 years ago | (#21964494)

Although I generally agree with the belief that FOSS probably yields better security, I think FOSS has a different characteristic of vulnerability than closed-source software. Specifically, the "ease of exploiting" a vulnerability is increased along with the ease of modification of the software. The most understanding of the system that's out there, the easier it is to take advantage of a vulnerability. I realize that "security through obscurity" is not something you want to depend on, but it is a real effect that has to be considered if you're going to compare the likelihood of actually being attacked versus simply having a vulnerability.

For example, MacOS and Windows had a similar number of critical security patches last year. However, there were dozens of Windows viruses and hundreds of thousands of compromised machines, and zero MacOS viruses. Thus, while a certain measure of vulnerability is comparable, the likelihood of actually being attacked is infinitely higher with Windows.

Re:Looking good, too bad the press didn't understa (2, Interesting)

civilizedINTENSITY (45686) | more than 6 years ago | (#21965000)

"For example, MacOS and Windows had a similar number of critical security patches last year."

Willing to stipulate for the purpose of this discussion.

However, there were dozens of Windows viruses and hundreds of thousands of compromised machines, and zero MacOS viruses.

Likewise willing to stipulate.

Thus, while a certain measure of vulnerability is comparable, the likelihood of actually being attacked is infinitely highder with Windows.

I would suggest this doesn't necessarily follow. It could be. It could also be that while both fixed the same number of holes, the percentage of holes fixed was different. It could be that x holes represented 85% of the mac holes, whereas the same exact number x was only 13% of the available windows holes.

Not saying one or the other interpretation is true. Just that the facts don't necessarily lead to the conclusion posited.

Re:Looking good, too bad the press didn't understa (1)

QuantumG (50515) | more than 6 years ago | (#21964532)

No-one was debating Bruce's last point about Coverity returning many false positives.

As for the use of terminology, excuse me for using an accurate term like "defect" instead of a more popular colloquialism like "hole".

Re:Looking good, too bad the press didn't understa (1, Informative)

Anonymous Coward | more than 6 years ago | (#21964626)

Well put. I wrote about this topic in my recent security concepts ebook:

http://www.subspacefield.org/security/security_concepts.html#tth_sEc24.5 [subspacefield.org]

RTFA (5, Informative)

Pinckney (1098477) | more than 6 years ago | (#21963876)

The important point here is that proprietary software manufacturers aren't telling you how many security flaws they had. I bet it's more than 1 per 1000 lines, that is an incredibly excellent figure for the first time a scanner like coverity is run.
Actually, the first line of the article reads "Open source code, much like its commercial counterpart, tends to contain one security exposure for every 1,000 lines of code, according to a program launched by the Department of Homeland Security to review and tighten up open source code's security."

Mod parent up! (1)

shaitand (626655) | more than 6 years ago | (#21964128)

Most people only read the summary.

Re:Mod parent up! (1)

enoz (1181117) | more than 6 years ago | (#21964310)

Some people even read the summary.
Fixed that for you.

Pessimism in article (5, Informative)

filbranden (1168407) | more than 6 years ago | (#21964802)

Not only did the article say much like its commercial counterpart, but most of the numbers it shows are actually good for open source software.

For instance, most of the projects discussed had less than 1 bug for 1000 lines of code. For instance, the Linux kernel had .127 bugs per 1000 lines, and that on over 3 million lines of code.

Also, the article talks about key projects, such as the glibc (which is basically used by everything on a Linux system) that already fixed all the issues.

Even something huge and complex as Firefox has already fixed half of the issues, and is showing progress on the rest of them (by the fact that some were already verified).

Overall, I didn't get the half glass empty tone that the summary is implying. And what I found strange is that even the comments on the site itself, and many of them on /. itself, are also taking the pessimistic view.

I thought that this news are great for open source software. Shows that it has less security issues than average, that the issues are fixed quickly, and still that some programs are certified by a company for use in security related departments such as the DHS. What could be better than that?

Re:RTFA (1)

falconwolf (725481) | more than 6 years ago | (#21964266)

Actually, the first line of the article reads "Open source code, much like its commercial counterpart, tends to contain one security exposure for every 1,000 lines of code, according to a program launched by the Department of Homeland Security to review and tighten up open source code's security."

The problem is is how do they know how many lines of code are in the closed commercial programs if they can't see the code?

Falcon

Re:RTFA (1)

totally bogus dude (1040246) | more than 6 years ago | (#21964406)

Gee good point! How could they possibly view the code if it's not open to all? I mean, it's not as if there's any possibility they could've gotten a bunch of companies to agree to let them audit their code provided they only released the results in aggregate, without any identifying information.

Just because it's not open source doesn't mean that nobody is ever able to gain access to it.

Re:RTFA (0)

Anonymous Coward | more than 6 years ago | (#21964432)

Spoiler: DHS saw the code.

Re:RTFA (1)

palegray.net (1195047) | more than 6 years ago | (#21964456)

In many cases they can see the code, albeit under pretty restrictive NDAs. Alternately, you could simply ask the vendor for a sloccount. Or perhaps taking the binary file size and using an average lines of code per KB estimate.

how many lines of code? (1)

falconwolf (725481) | more than 6 years ago | (#21964550)

In many cases they can see the code, albeit under pretty restrictive NDAs.

Which is why I consider open source more secure, anyone can find a hole and anyone, well programmers at least, can fix the hole. With closed source code a review can be restricted from informing users of said code the problems it has.

Falcon

Re:how many lines of code? (1)

palegray.net (1195047) | more than 6 years ago | (#21964592)

Agreed on the problem of finding holes but not being able to inform a userbase. I've seen it happen in practice; not pretty.

Re:Looking good, too bad the press didn't understa (2)

unlametheweak (1102159) | more than 6 years ago | (#21963888)

According to McAfee recently (http://yro.slashdot.org/article.pl?sid=08/01/05/0215201) and Microsoft et al, having your code exposed lets the bad guys exploit it's vulnerabilities. Of course if or when a weakness is taken advantage of, it would likely be fixed vary quickly through the FOSS community, instead of on the first Tuesday of every month like as in Microsoft's business model.

Re:Looking good, too bad the press didn't understa (1)

Bruce Perens (3872) | more than 6 years ago | (#21964394)

According to McAfee recently (http://yro.slashdot.org/article.pl?sid=08/01/05/0215201) and Microsoft et al, having your code exposed lets the bad guys exploit it's vulnerabilities

Yes they said that, but you don't really believe it, do you? If so, just look up "security by obscurity" and read about it. To give you a clue, the unavailability of source has not prevented 100,000 Windows viruses.

Re:Looking good, too bad the press didn't understa (1)

ClosedSource (238333) | more than 6 years ago | (#21964502)

I think your logic is a bit confused. The fact that viruses can be created without reading the source code does not prove that there's no value in keeping the code secret. It's like arguing that there's no point in locking your door because 100,000 houses with locks were broken into.

Re:Looking good, too bad the press didn't understa (3, Funny)

Waffle Iron (339739) | more than 6 years ago | (#21964640)

It's like arguing that there's no point in locking your door because 100,000 houses with locks were broken into.

A more apt analogy would be: There's no point in locking your door using a limp spaghetti noodle because a limp noodle makes a completely ineffective lock.

Re:Looking good, too bad the press didn't understa (1)

ClosedSource (238333) | more than 6 years ago | (#21965180)

I see you're trying to imply that Windows is insecure, but I don't see what that has to do with the issue of security through obscurity.

Re:Looking good, too bad the press didn't understa (1)

Bruce Perens (3872) | more than 6 years ago | (#21964788)

My door is locked, but the mechanism of the lock is easily available in the hardware store for others to scrutinize. And so it should be. This is a different sort of information than the pattern of the key.

Bruce

Re:Looking good, too bad the press didn't understa (2, Insightful)

ClosedSource (238333) | more than 6 years ago | (#21965142)

Analogies have their limits, so we shouldn't try to take it too far.

Even those who historically have critized "security through obscurity" never suggested that publishing their design or secrets would lead to better security, but rather that you can't assume your that your design can't be cracked.

Of course, the preferred approach is "security through design" which has nothing to do with correcting bugs. The latter could be called "security through maintenence". Thus while we might argue about whether closed or open source produces better design, examining source code for bugs can't compensate for a design that is insecure.

Re:Looking good, too bad the press didn't understa (1)

unlametheweak (1102159) | more than 6 years ago | (#21964830)

I think your logic is a bit confused. The fact that viruses can be created without reading the source code does not prove that there's no value in keeping the code secret. It's like arguing that there's no point in locking your door because 100,000 houses with locks were broken into.
Fact is anybody can dis-assemble a lock. And of course people can dis-assemble code.
Not too many people would be interested in breaking into a lock on a door (smashing a Window to get into the house is most generally used by non-government intruders).

The greatest value in keeping code secret is making sure it cannot be easily re-produced, and thus subverting other individuals or companies from using it without authorization. It's much like music and DRM: in the end it is the licenses which are enforced in the court of law, and not so much the code itself (the "locks" if you will), that will be able to protect the intellectual property.

Yes there may be value in keeping code secret, but I would argue that the value is minimal compared to the benefits of keeping it open.

Physical locks and security by obscurity (1)

Bruce Perens (3872) | more than 6 years ago | (#21965036)

Here's a good story about examining how locks work, that shows the value of "disclosed source".

Anyone can buy a re-key kit for Schlage locks at the Home Depot. Upon opening the cylinder of the lock with that kit, you will discover that (this is approximate, I don't have the lock in front of me) there are 5 pins, and 5 possible levels per pin, and that the minimum number of possible key patterns might thus be 5 ^ 5 or 3125. Which is enough that nobody's carrying all of the possible keys around and will have time to go through them at my front door.

The re-key kit comes with a set of two identical new keys that do not use the same pin length twice, and thus its number of possible patterns might be 5 * 4 * 3 * 2 * 1 or only 120. Uh-oh! Better not base the keys for my house on the master from the re-key kit! And shame on them for not saying this on the box.

See the benefit of being able to examine how they work?

FYI, yes I know about lock-picking, there's an alarm too.Bruce

Re:Looking good, too bad the press didn't understa (1)

unlametheweak (1102159) | more than 6 years ago | (#21964612)

According to McAfee recently (http://yro.slashdot.org/article.pl?sid=08/01/05/0215201) and Microsoft et al, having your code exposed lets the bad guys exploit it's vulnerabilities
Yes they said that, but you don't really believe it, do you? If so, just look up "security by obscurity" and read about it. To give you a clue, the unavailability of source has not prevented 100,000 Windows viruses.
No I do not believe it. I was just pointing out some (IMHO) rather lame and biased arguments. Openness and transparency (whether it be in software, business models, or just dealing with one's spouse, for example) is generally better than keeping things hidden.

Make the licenses as restrictive as you please, but at least give people the opportunity to know what they are using. Like listing ingredients on processed food, it's good to know that I'm not consuming something that could possibly do me harm (or be beneficial).

There is a level of comfort in dealing with openness. It seems like that's why so many politicians and business leaders are not trusted; because they hide behind PR vetted canned answers (security through obscurity if you will), rather than being articulate and just admitting outright when things aren't working the way they should.

Re:Looking good, too bad the press didn't understa (1)

Ikipou (1193603) | more than 6 years ago | (#21963912)

I'm not from USA, and I'm wondering if the Department of Homeland Security don't also have the code of some proprietary software? To get a certification for homeland security, you don't have to give your code to the government?

Re:Looking good, too bad the press didn't understa (2, Informative)

liquidpele (663430) | more than 6 years ago | (#21964116)

If a company wanted to be audited by them, they would have the techs doing the auditing sign heavy NDA (non-disclosure agreements) and then give them the code. This way, if the techs then give away any details about the code (or often even talk about the code) the company could sue.

So yes, they can get audited if they wanted, but they still would be the only ones who knew how bad the code was (besides the techs doing the auditing of course).

Re:Looking good, too bad the press didn't understa (1)

Pinckney (1098477) | more than 6 years ago | (#21964084)

"Our commercial customers wouldn't like it too much if we aired the number of defects found in their code," said Maxwell, when asked about the results from scans on 400 product lines of the firm's private customers.
So yes, they are scanning proprietary software as well, and they find roughly the same number of security vulnerabilities.

Re:Looking good, too bad the press didn't understa (1)

Bruce Perens (3872) | more than 6 years ago | (#21964678)

Actually, I read that as "We won't tell you how many bugs there are, our customers would not like it". They could well be inflating the reliability of proprietary software for their customers sake.

Re:Looking good, too bad the press didn't understa (5, Informative)

grcumb (781340) | more than 6 years ago | (#21964102)

The important point here is that proprietary software manufacturers aren't telling you how many security flaws they had.

Indeed. FTFA:

"Our commercial customers wouldn't like it too much if we aired the number of defects found in their code," said Maxwell, when asked about the results from scans on 400 product lines of the firm's private customers.

One can only speculate about the, er, source of their discomfort.... 8^)

I bet it's more than 1 per 1000 lines, that is an incredibly excellent figure for the first time a scanner like coverity is run.

1 per 1000 lines is even more impressive as an average across all 180 FOSS applications tested. Most impressive of all are the highlights:

  • SAMBA: 236 defects in 450,000 lines of code. 228 already fixed.
  • Linux Kernel: 0.127 security faults per thousand lines of code. The kernel scan covered 3,639,322 lines of code.
  • Apache: 135,916 lines of code, which yielded a security defect rate of 0.14 bugs per thousand lines of code. Or 1.4 per 10,000 lines of code, if you prefer. 8^)
  • PostgreSQL: 909,148 lines of code, with a 0.041 per 1000 defect rate.
  • glibc: 83 bugs in 588,931 lines of code, all since fixed.

Even some of those with more bugs have at least responded well:

  • KDE: 4,712,273 lines of code, fixed 1,554 defects, verified another 25 and has only 65 to go.
  • GNOME: 430,809 lines of code, fixed 357 defects, verified 5 and has 214 to go.

And my favourite 'backslider' of all, OpenVPN, has yet to fix 100% of the bugs found during this exercise. Of course, that's only 1 bug in over 69,000 lines of code....

These results should be viewed as excellent, by and large. This doesn't mean all this software is bug-free, just that there aren't a lot of easily preventable bugs in the code base. Most encouraging, though, is how fast they got addressed and fixed by the healthier FOSS projects.

Re:Looking good, too bad the press didn't understa (0)

Anonymous Coward | more than 6 years ago | (#21965214)

If it's this easy to find security bugs, why aren't there tons of people doing this for every project? You mean you just set up a piece of software, let it run a scan and you've done your security testing?! SERIOUSLY?! Jesus. Seriously, why wouldn't the linux kernel maintainers, for example, run the scans on every release, then??

Re:Looking good, too bad the press didn't understa (1)

RobBebop (947356) | more than 6 years ago | (#21964614)

Oh man... Bruce Perens. What a pleasure. I couldn't have said it better myself.

(Actually, I was going to make fun of proprietary software for the general idea of having source unavailable).

More to the point though, I received a lecture on this [typepad.com] in a Software Architecture course a couple years ago and it struck a nerve. Even if you never need to review 99.9% of the code you run, it is nice to be able to look through the 0.1% that might be helpful for you to gain a better understanding of what is going on. And that is only possible with Open Source.

Cheers.

What an amazing victory for OSS! (0)

Anonymous Coward | more than 6 years ago | (#21963696)

A third party came in, identified bugs, and they're being fixed...

Meanwhile, any third party can't peruse the code of the non-open software alternatives and can't find the bugs...

Congrats OSS!

(PS-- anyone know- do they count the same bug appearing multiple times as one bug or many?)

AN opportunity to modify the GPL.. (0)

bigattichouse (527527) | more than 6 years ago | (#21963704)

Just thought of this: Make it stipulation of GPL that if you publically report bugs or bug counts in GPL software, that you must also produce a detailed account of how to reproduce the bug, and you must provide that report to the maintainer of the current source (who you got it from, or the root source as listed in the code). Possibly a two-week window between notification (and acknowledgement) and publication. In a way, firms are profiting from the GPL software, but I can guess they aren't reporting all these issues. Sure you could fix stuff in private - but to publically announce it, you should have to allow everyone the oppotunity to fix it.

Re:AN opportunity to modify the GPL.. (1, Insightful)

Anonymous Coward | more than 6 years ago | (#21963848)

We really need a +1 Stupid option

Re:AN opportunity to modify the GPL.. (0)

Anonymous Coward | more than 6 years ago | (#21964360)

We really need a +1 Stupid option
Not really. All the followup posts calling the parent a dipshit more than accomplishes that visibility requirement.

Re:AN opportunity to modify the GPL.. (1)

Lehk228 (705449) | more than 6 years ago | (#21963918)

I would like to consider you an hero of the open source movement

please become an hero.

A slight twist.... (1)

EmbeddedJanitor (597831) | more than 6 years ago | (#21964068)

It is somewhat sad that copyright (the legal muscle in GPL) only covers the writing of software. The testing and verification - which is often as important, if not more important has no legal protection. It is the testing, rather than the coding, that really adds the value in OSS.

Re:AN opportunity to modify the GPL.. (0)

Anonymous Coward | more than 6 years ago | (#21964086)

That is a stupid idea. Even if it was an issue, what difference does it make as long as it get fixed?

Re:AN opportunity to modify the GPL.. (2, Insightful)

WK2 (1072560) | more than 6 years ago | (#21964112)

There are two problems with your suggestion:

a) it is too restrictive, and would disqualify the GPL as free software. Remember, that the GPL is a distribution license, not a list of restrictions. You should be able to talk to other people (even publicly) about software without contacting the maintainer first. The behavior you describe is responsible, and generally recommended, but should not be forced.

b) as you have it worded, if the restrictions were followed, it would enable a maintainer to prevent anyone from disclosing any security bugs. You say that reporters have to wait for an acknowledgment. What if one is never received? What if there is no maintainer? The solution for this problem is obvious (don't require an acknowledgment), but I should point it out, nonetheless.

c) It is not enforceable in most jurisdictions. In the US, and I assume most of the "free world", you can't prevent someone from talking about your products publicly. You can have them sign an NDA, but that doesn't work for publicly available software. McAfee tried something like this some time ago, stipulating in the EULA that you can't benchmark their software. It got shot down in court.

publicly speaking about software (1)

falconwolf (725481) | more than 6 years ago | (#21964352)

c) It is not enforceable in most jurisdictions. In the US, and I assume most of the "free world", you can't prevent someone from talking about your products publicly. You can have them sign an NDA, but that doesn't work for publicly available software. McAfee tried something like this some time ago, stipulating in the EULA that you can't benchmark their software. It got shot down in court.

I haven't seen one in years but doesn't Microsoft's ULA have a clause that you can't publish a review of the software without having MS's approval? I think a year or so ago there was an article on /. about it.

Falcon

Re:AN opportunity to modify the GPL.. (2, Insightful)

Guy Harris (3803) | more than 6 years ago | (#21964302)

Just thought of this: Make it stipulation of GPL that if you publically report bugs or bug counts in GPL software, that you must also produce a detailed account of how to reproduce the bug, and you must provide that report to the maintainer of the current source (who you got it from, or the root source as listed in the code). Possibly a two-week window between notification (and acknowledgement) and publication.

Not all bugs are easily reproducible - and not all bugs are found by tripping over them. Consider, for example, bugs found by various of the warnings enabled by GCC's -W options. I.e., you get reports saying "this code path has these problems", not reports saying "this code path blew up when I did XXX".

I just looked at an old report from Coverity on one of the free-software projects with which I'm involved - one of the problems it found was in a chunk of code

if (cfg->in)use) {
report an error;
return;
}
if (cfg != NULL) {
process what it points to;
} else {
report an error and clean up;
return;
}

where it quite appropriately pointed out that we were checking whether cfg was null after dereferencing it rather than before dereferencing it. We subsequently fixed that problem.

It might be possible to construct a scenario where the application would crash due to that bug - or it might not; that bug is in "framework" code, and if the code using that framework code doesn't happen to pass an argument that would cause cfg to be null, there won't be a crash, but some code in the future might pass such an argument (which might be an argument that comes from user input, so it's not as if passing such an argument is a bug - perhaps the code using the framework code is expecting that code to tell the user of the error).

Even if it's possible to construct such a scenario, the software that found the problem doesn't have a deep enough understanding of the code to say "hey, if you open up the app on a file like with this in it and select this menu item and type this into the dialog box that pops up and then click 'OK', it'll crash", so it's not as if the software that's reporting this problem (non-publicly - to see the reports on an app, you have to be a "member" of the project whose code is being scanned, and sign up for an account [coverity.com] ) can give "a detailed account of how to reproduce the bug".

creators' planet/population rescue kode bug free (-1, Troll)

Anonymous Coward | more than 6 years ago | (#21963708)

no gadgets, or approval from anyone needed. the truly open source nature makes it also quite user friendly. let yOUR conscience be yOUR guide. you can be more helpful than you might have imagined. there are still some choices. if they do not suit you, consider the likely results of continuing to follow the corepirate nazi hypenosys story LIEn, whereas anything of relevance is replaced almost instantly with pr ?firm? scriptdead mindphuking propaganda or 'celebrity' trivia 'foam'. meanwhile; don't forget to get a little more oxygen on yOUR brain, & look up in the sky from time to time, starting early in the day. there's lots going on up there.

http://news.yahoo.com/s/ap/20071229/ap_on_sc/ye_climate_records;_ylt=A0WTcVgednZHP2gB9wms0NUE [yahoo.com]

http://www.nytimes.com/2007/12/31/opinion/31mon1.html?em&ex=1199336400&en=c4b5414371631707&ei=5087%0A [nytimes.com]

is it time to get real yet? A LOT of energy is being squandered in attempts to keep US in the dark. in the end (give or take a few 1000 years), the creators will prevail (world without end, etc...), as it has always been. the process of gaining yOUR release from the current hostage situation may not be what you might think it is. butt of course, most of US don't know, or care what a precarious/fatal situation we're in. for example; the insidious attempts by the felonious corepirate nazi execrable to block the suns' light, interfering with a requirement (sunlight) for us to stay healthy/alive. it's likely not good for yOUR health/memories 'else they'd be bragging about it? we're intending for the whoreabully deceptive (they'll do ANYTHING for a bit more monIE/power) felons to give up/fail even further, in attempting to control the 'weather', as well as a # of other things/events.

http://video.google.com/videosearch?hl=en&q=video+cloud+spraying [google.com]

dictator style micro management has never worked (for very long). it's an illness. tie that with life0cidal aggression & softwar gangster style bullying, & what do we have? a greed/fear/ego based recipe for disaster. meanwhile, you can help to stop the bleeding (loss of life & limb);

http://www.cnn.com/2007/POLITICS/12/28/vermont.banning.bush.ap/index.html [cnn.com]

the bleeding must be stopped before any healing can begin. jailing a couple of corepirate nazi hired goons would send a clear message to the rest of the world from US. any truthful look at the 'scorecard' would reveal that we are a society in decline/deep doo-doo, despite all of the scriptdead pr ?firm? generated drum beating & flag waving propaganda that we are constantly bombarded with. is it time to get real yet? please consider carefully ALL of yOUR other 'options'. the creators will prevail. as it has always been.

corepirate nazi execrable costs outweigh benefits
(Score:-)mynuts won, the king is a fink)
by ourselves on everyday 24/7

as there are no benefits, just more&more death/debt & disruption. fortunately there's an 'army' of light bringers, coming yOUR way. the little ones/innocents must/will be protected. after the big flash, ALL of yOUR imaginary 'borders' may blur a bit? for each of the creators' innocents harmed in any way, there is a debt that must/will be repaid by you/us, as the perpetrators/minions of unprecedented evile, will not be available. 'vote' with (what's left in) yOUR wallet, & by your behaviors. help bring an end to unprecedented evile's manifestation through yOUR owned felonious corepirate nazi glowbull warmongering execrable. some of US should consider ourselves somewhat fortunate to be among those scheduled to survive after the big flash/implementation of the creators' wwwildly popular planet/population rescue initiative/mandate. it's right in the manual, 'world without end', etc.... as we all ?know?, change is inevitable, & denying/ignoring gravity, logic, morality, etc..., is only possible, on a temporary basis. concern about the course of events that will occur should the life0cidal execrable fail to be intervened upon is in order. 'do not be dismayed' (also from the manual). however, it's ok/recommended, to not attempt to live under/accept, fauxking nazi felon greed/fear/ego based pr ?firm? scriptdead mindphuking hypenosys.

consult with/trust in yOUR creators. providing more than enough of everything for everyone (without any distracting/spiritdead personal gain motives), whilst badtolling unprecedented evile, using an unlimited supply of newclear power, since/until forever. see you there?

"If my people, which are called by my name, shall humble themselves, and pray, and seek my face, and turn from their wicked ways; then will I hear from heaven, and will forgive their sin, and will heal their land."

meanwhile, the life0cidal philistines continue on their path of death, debt, & disruption for most of US. gov. bush denies health care for the little ones;

http://www.cnn.com/2007/POLITICS/10/03/bush.veto/index.html [cnn.com]

whilst demanding/extorting billions to paint more targets on the bigger kids;

http://www.cnn.com/2007/POLITICS/12/12/bush.war.funding/index.html [cnn.com]

& pretending that it isn't happening here;

http://www.timesonline.co.uk/tol/news/world/us_and_americas/article3086937.ece [timesonline.co.uk]
all is not lost/forgotten/forgiven

(yOUR elected) president al gore (deciding not to wait for the much anticipated 'lonesome al answers yOUR questions' interview here on /.) continues to attempt to shed some light on yOUR foibles. talk about reverse polarity;

http://www.timesonline.co.uk/tol/news/environment/article3046116.ece [timesonline.co.uk]

goverment helping FOSS (1)

shadylookin (1209874) | more than 6 years ago | (#21963718)

I guess this benefits open source software since any bug fix is a good thing, but why on earth would the department of homeland security be studying software. shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?

Re:goverment helping FOSS (2, Insightful)

KillerCow (213458) | more than 6 years ago | (#21963768)

Computer terrorism. They don't want a send-mail bug to allow a beachhead for compromising more sensitive systems.

Re:goverment helping FOSS (1)

QuantumG (50515) | more than 6 years ago | (#21963858)

Didn't you watch Die Hard 4? DHS obviously did.

Re:goverment helping FOSS (1)

ianare (1132971) | more than 6 years ago | (#21963882)

In the 21st century, you do have to worry about cyberattacks. The DHS uses some of these tools [linux.com] , and it is a Good Thing (tm) they are making them more secure. They help propriatary software vendors [fcw.com] too, the difference is that with OSS everyone benefits.

Re:goverment helping FOSS (1)

WK2 (1072560) | more than 6 years ago | (#21964126)

shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?

Good point. It's too bad they can't do both.

Homeland Security (1)

falconwolf (725481) | more than 6 years ago | (#21964372)

I guess this benefits open source software since any bug fix is a good thing, but why on earth would the department of homeland security be studying software. shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?

Such as hurricanes?

Falcon

Re:Homeland Security (1)

civilizedINTENSITY (45686) | more than 6 years ago | (#21965166)

Actually I seem to recall DHS paid a CIA analyst to do an impact study on global warming and climate change. Instead of looking at where temperatures are changing (which scientists are concerned about) the question posed was what will these changes mean. The result was that cities are not that robust. More than hurricanes, but I'm sure they were a part of it...

Re:goverment helping FOSS (0)

Anonymous Coward | more than 6 years ago | (#21964424)

I guess this benefits open source software since any bug fix is a good thing, but why on earth would the department of homeland security be studying software. shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?
Because in a rare moment of sanity one of the senior government officials realized that the DHS could do things other than crush civil liberties, profile minorities, or make our lives hellish going through airports or border crossings. That official will be quickly replaced.

Re:goverment helping FOSS (1)

dbIII (701233) | more than 6 years ago | (#21964822)

It's the Uber deptment of everything from Rubik's cube copyright enforcement to financially punishing airlines for carrying musicians that have converted to Islam. If they want to study software they can - there is no leadership or supervision in place to limit them. Theoretically the President or Cheney could do something to influence them but in practice they are uncontrolled.

Re:goverment helping FOSS (1)

unlametheweak (1102159) | more than 6 years ago | (#21965016)

I guess this benefits open source software since any bug fix is a good thing, but why on earth would the department of homeland security be studying software. shouldn't they be worrying about things like preventing biological attacks or improving how they handle natural disasters?
Software is used in important areas that are vital to a country's basic infrastructure and operation, such as power plants (nuclear or otherwise), radio and television stations, cellular phones, the Internet, banking, etc. I think the Internet would be one of the most important as it is a major source of commerce and communication.

Example:

Matthew Kovar, a senior analyst at the market research firm Yankee Group, generated some publicity when he told reporters the attacks caused USD $1.2 billion in global economic damages.
- http://en.wikipedia.org/wiki/MafiaBoy [wikipedia.org]

The attack was aimed at DNS root servers. Since that time the router's software has been upgraded to prevent such wide-scale damage. I remember that incident because I was literally unable to access any Web site at the time. I had later learned that this was because this one attacked clogged up most of the Internet.

Re:goverment helping FOSS (1)

jayp00001 (267507) | more than 6 years ago | (#21965240)

Mod parent up- this is exactly right. If we have people bright enough took work on FOSS code can't we move them to the airports to replace the morons searching the handicapped and infants for security?

Ummm (1)

renegadesx (977007) | more than 6 years ago | (#21963728)

Samba was not listed as having a clean bill of health, there were bugs that have yet to be verified.
I think the bug rate for alot of these projects (Linux, Samba, etc) remarkable

Re:Ummm (1)

karlto (883425) | more than 6 years ago | (#21963948)

The second article lists Samba as being certified at the top level (Rung 2)

Must be run by Engineers... (4, Funny)

ComputerSlicer23 (516509) | more than 6 years ago | (#21963784)

Uh.. from the article, the software is called "Prevent Software Quality System"... Wow, I can't think of a bigger misnomer for something that should help improve software quality. I sure don't want to prevent software quality in my own products.

But...but (0)

Anonymous Coward | more than 6 years ago | (#21963794)

I thought that the Bush Administration could do nothing good by Slashdot standards?

Re:But...but (1)

Torvaun (1040898) | more than 6 years ago | (#21963974)

They usually don't. But, enlightened individuals that many Slashdot users are, we do not feel the need to rip apart good things done by bad people. We will look for motive, yes, but we will also accept the good that they do.

Re:But...but (1)

Copid (137416) | more than 6 years ago | (#21964332)

I thought that the Bush Administration could do nothing good by Slashdot standards?
I can't tell for sure, but I strongly suspect that whichever anonymous coward posted this also complains loudly when people post irrelevant anti-Bush trolls.

The Actual Scan Site (2, Informative)

gQuigs (913879) | more than 6 years ago | (#21963796)

Wow important stuff (3, Funny)

OzPeter (195038) | more than 6 years ago | (#21963908)

I checked out the Coverity website [coverity.com] and saw on the list of projects the aalib ASCII art library [sourceforge.net] which according to the history hasn't been updated for something like 7 years.

Damn we better protect ourselves from Terrists hiding their WMD's in ASCI art

Re:Wow important stuff (1)

enoz (1181117) | more than 6 years ago | (#21964420)

Damn we better protect ourselves from Terrists hiding their WMD's in ASCI art
Damn right! [goatse.ch]

I wouldn't sweat it. (1)

rindeee (530084) | more than 6 years ago | (#21963950)

DHS is a dysfunctional mess for the most part when it comes to INFOSEC/IA. They negative for the sake of negativity approach does not surprise me in the lest. If it's any comfort, DoD takes FOSS quite seriously and makes use of many great FOSS tools and platforms. It really is a cultural difference. Those in the DoD that get the job done are prone to use 'the best tool for the job'. FOSS is a gimme in many (and an ever increasing number of) cases.

False positives (2, Interesting)

clem.dickey (102292) | more than 6 years ago | (#21963984)

The article did not seem to give any data on false positives. A story here [internetnews.com] has Coverity claiming a 10% false positive rate. But there is no independent confirmation. It would also be interesting to know how hard it is to prove a false positive vs. how hard to fix a true positive. In other words, it it worth Coverity's time to further reduce the false positive rate.

Well... (4, Insightful)

Otter (3800) | more than 6 years ago | (#21964008)

This seems like a genuinely useful activity for DHS, certainly more valuable than x-raying my shoes and confiscating my saline solution.

Re:Well... (1)

ColdWetDog (752185) | more than 6 years ago | (#21964074)

This seems like a genuinely useful activity for DHS, certainly more valuable than x-raying my shoes and confiscating my saline solution.

Of course, you realize you are not setting the bar very high...

Re:Well... (1)

drgould (24404) | more than 6 years ago | (#21964202)

This seems like a genuinely useful activity for DHS, certainly more valuable than x-raying my shoes and confiscating my saline solution.
Of course, you realize you are not setting the bar very high...

Of course you realize you are talking about the DHS.

This seems like a genuinely useful activity for (1)

falconwolf (725481) | more than 6 years ago | (#21964440)

DHS, certainly more valuable than x-raying my shoes and confiscating my saline solution.

What would be more valuable would be to get rid of DHS.

Falcon

Once again, free enterprise wins. (-1, Flamebait)

Anonymous Coward | more than 6 years ago | (#21964034)

So now it takes the Department of Homeland Security to find the thousands of bugs present in open source software. Our tax dollars at work. Instead of protecting our nation from terrorsts, these federally-funded workers have to spend time vetting a bunch of hippie geeks' pet projects. It's pethetic.

Compare this with just two of the many examples of the excellence of free enterprise. Microsoft XP operating system and XBox 360 game platform, two of the most successful computer products ever made, testament to their greatness by the billions of dollars' worth of these fine products that have been sold. Did WinXP or the 360 need a government handout to survive? The answer is no. Proof once again that "free" software is no substitute for the free market! [wikipedia.org]

L, A and P, but where's M? (5, Interesting)

ThreeGigs (239452) | more than 6 years ago | (#21964168)

From TFA:
The popular MySQL open source database was not included in the scans for reasons that were not immediately evident.

Any suggestions as to why MySQL has no results? I'm stumped and wondering why one whole corner of a LAMP foundation was left unchecked.

Re:L, A and P, but where's M? (1)

The MAZZTer (911996) | more than 6 years ago | (#21964340)

Scanners can have bugs too. Maybe feeding the MySQL source code into it caused it to error or crash for whatever reason.

Or maybe licensing issues? Although I doubt it, IIRC MySQL is GPL or something.

those scanning tools are very unreliable (0)

Anonymous Coward | more than 6 years ago | (#21964204)

i have tested and evaluated that particular tool as well as several competitors; they all had high false positive rates.

props for DHS? (0)

Anonymous Coward | more than 6 years ago | (#21964276)

Does anyone want to say thanks to the US government for this service?

some notes on the article (5, Interesting)

ehovland (2915) | more than 6 years ago | (#21964870)

First off, prevent is not strictly a security flaw static-analysis checker. It is a static-analysis checker that checks for all sorts of defects. Some of which are directly related to security. Second, I have used prevent extensively over the past year and have found it to be an invaluable tool. It has a pretty low false positive rate and fixing the defects it finds means your code is better. On the code I work on, I find that we have a much lower defect count. But we also have pretty mature code and we really do attempt to make it as bullet proof as possible. But we still have defects.

My experience is with the C/C++ version of tool. We have also been evaluating the java version of the tool and it is good. But some of the free alternatives like findbugs are still better. I would use findbugs w/ prevent for java if I wanted good coverage.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>