Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Study Finds Bug Bounty Programs Extremely Cost-Effective

Unknown Lamer posted about a year ago | from the open-sores-hates-jerbs dept.

Bug 95

itwbennett writes "U.C. Berkeley researchers have determined that crowdsourcing bug-finding is a far better investment than hiring employees to do the job. Here's the math: Over the last three years, Google has paid $580,000 and Mozilla has paid $570,000 for bugs found in their Chrome and Firefox browsers — and hundreds of vulnerabilities have been fixed. Compare that to the average annual cost of a single North American developer (about $100,000, plus 50% overhead), 'we see that the cost of either of these VRPs (vulnerability reward programs) is comparable to the cost of just one member of the browser security team,' the researchers wrote (PDF). And the crowdsourcing also uncovered more bugs than a single full-time developer could find."

Sorry! There are no comments related to the filter you selected.

Woot (2)

zoomshorts (137587) | about a year ago | (#44240445)

Ohhh, code review is useful???

Incentives (5, Informative)

Todd Knarr (15451) | about a year ago | (#44240481)

The major problem is that on-staff developers are usually discouraged from going on bug-hunts. Management would rather have them developing new features, so they won't allocate time towards finding bugs. When what the company policy towards finding bugs is conflicts with how your manager assigns you tasks, guess which one wins. Worse, most of the time an employee who ignores his to-do list to go find problems ends up penalized either explicitly (by bad reviews) or implicitly (negative impact from people being annoyed that he made work for them). Outsiders in these bounty programs don't have to worry about a manager assigning them 100% to new features and 0% to finding vulnerabilities and they don't have to worry about the impact of bad reviews or negative comments by managers about the extra work they created for everybody.

Re:Incentives (0)

Anonymous Coward | about a year ago | (#44240533)

Doesnt that support the argument to use VRPs? Seems to make sense to have your employees work on new features and let the public find your bugs for pennies.

Re:Incentives (2)

Alumoi (1321661) | about a year ago | (#44240981)

Hmm, just like every major software company is doing: sell first version of software, let the paying crowd find the bugs, sell second version of software with (some) bug fixed. Rinse, repeat. Profit!

Re:Incentives (4, Informative)

VorpalRodent (964940) | about a year ago | (#44240563)

This.

And not just bug hunts. I have a laundry list of things that need to be refactored, but every time we think we might have a chance to do so, project management decides something else is more important. We have people complaining about things being slow, but when told that we need to spend time to make it faster, we instead get directed at new features or, worse, tweaks for the sake of a single non-representative customer that happens to have the ear of the project owner.

Re:Incentives (1)

Anonymous Coward | about a year ago | (#44240915)

Do you work at the same middleware company as me?

Re:Incentives (1)

Todd Knarr (15451) | about a year ago | (#44241319)

Yes. At work we have a long-outstanding issue open about performance. I nailed down where the bottleneck was, and put in the library module needed to fix it. Tests using the new module were showing a minimum order-of-magnitude speedup compared to the old (minimum because the test cases were biased to be as favorable to the old method as possible). Yet we've been directed to postpone actually switching to the new module (I suspect politics).

Re:Incentives (0)

Anonymous Coward | about a year ago | (#44242315)

The squeaky wheel as it were. I love that type of management. ( He said while rolling his eys so far back into his head he could see who was standing behind him.

Re:Incentives (4, Insightful)

CastrTroy (595695) | about a year ago | (#44240627)

Exactly. I think if you found the right kind of employee and told them to hunt for bugs all day long and get paid for it, They'd probably uncover quite a few bugs. Give them complete access to the code, source control, and test suites, and they could probably find bugs much more efficiently than getting somebody to find vulnerabilities from the outside.

Re:Incentives (0)

Anonymous Coward | about a year ago | (#44241155)

I've no problems finding bugs at work. The problem is the code is so bad that I find bugs faster than I can fix all of them. While fixing bugs I find more bugs.

It's amazing our customers still find the product useful enough to pay for it.

It's like a big building made of dried shit. It sort of works in some conditions. You know what happens if it rains a bit, but if you remove all the shitty bits there's not much building left... And to rebuild the building from scratch would take years.

Re:Incentives (2)

SethJohnson (112166) | about a year ago | (#44245727)

It's amazing our customers still find the product useful enough to pay for it. It's like a big building made of dried shit. It sort of works in some conditions. You know what happens if it rains a bit, but if you remove all the shitty bits there's not much building left... And to rebuild the building from scratch would take years.

Wow. Aren't you worried that by posting this online that you might get fired from your position on the Microsoft SQL Server dev team?

Re:Incentives (0)

Anonymous Coward | about a year ago | (#44241811)

This is my job. Management is still a problem.

Re:Incentives (1)

cheesybagel (670288) | about a year ago | (#44240835)

Yes this happens. The trick to get around it is to sell the bug fixing or refactoring as a new feature. i.e. you refactor the code in order to fix the longstanding bug and you add a new feature to boot. If your manager is too stupid to figure out the bug really, really needs to be fixed, you can always convince the client to "persuade" them instead of you.

Re:Incentives (1)

VorpalRodent (964940) | about a year ago | (#44242291)

I've done this as well - but I don't like effectively misleading management by saying that we need to do X in order to achieve what is only a loosely related Y. Yes, X would make Y easier, or improve Z and W, but it isn't truly essential.

That's not to say things are entirely bad. I've also had management see the true value of some stuff that isn't technically being requested at that very moment, and push things upwards by tying the work to something else in just this manner.

I guess the moral is that I don't want to mislead project owners, but I'm okay if my boss does it for me?

Re:Incentives (1)

cheesybagel (670288) | about a year ago | (#44242955)

I do not like to mislead project management either. The less information people have about the actual de facto state of the project the more likely it is that things will go off track and development will end up in failure because you either blew up the deadline or funding. However it is also important that projects have a certain degree of quality in them. Otherwise the clients will end up not renewing their contracts and going somewhere else next time. Balancing these needs is certainly not easy.

I once had to do more work for what effectively was a misfeature requested by the client. No amount of arguing I did could convince him of that. I knew his misfeature had no actual performance impact although it made no practical sense. In fact it made life worse for the system administrators. In the end, after the software was deployed, the system administrators forced the rollback of the misfeature because it made their jobs hell. Sometimes only time can solve such problems and no amount of forcing will do it.

Re:Incentives (0)

Anonymous Coward | about a year ago | (#44241167)

Management would rather have them developing new features, so they won't allocate time towards finding bugs.

I'd rather have developers develop (new code/features or fixes to any bugs) and offload the task of finding non-obvious bugs to QA. Whether management wants to put resources into QA is another question. They'll have to decide how often they want hear about bugs from external sources...

Re: Incentives (0)

Anonymous Coward | about a year ago | (#44247155)

Qa can find only max 30% of bugs. Code review 90% according to the studies. Developers can also find concurrency bugs more easily and they can find non Functional bugs e.g. bad variable names that Might cause new bugs.

Re:Incentives (0)

Anonymous Coward | about a year ago | (#44241637)

Maybe, but if they encouraged their employees to write stable, safe software, in the first place, or discouraged writing of bad software with penalties, shaming, w/e, they'd neither have to organise external bug hunts and pay out bounties, nor pull people off enhancement-type projects.

Yes, shaming and penalties, that's the way! (0)

Anonymous Coward | about a year ago | (#44242399)

RIght, with shaming. That's a good idea:

"Implement un-specced feature X in unreasonable time Y on top of an unstable foundation and no you can't take the time to refactor it we essentially want you to produce lava-code, and no you can't have time to test it, we promised it to the customer yesterday and no we don't do QA around here: 'just do it', 'make it so' and get it into production.

Oh. Oh look. There's a software defect!

Well you must be a very incompetent developer then 'innit mate. Perhaps if I engage in "shaming behaviour", express my "dissapointment" at your inability to defy logic and the laws of physics, threaten you with penalties and complain a lot for you to "go faster" I can hide the fact that I "manage" software projects for a living without, you know, actually knowing what software is. Oh and by the way give me a cast-iron reliable estimate on the following utterly vague and large unknown feature set right now, just pull it out of your ass and i'll hold you to it and no you can't have any time for planning."

Right dude, seriously: I guess you never worked as a developer then. You'ld also make a terrible manager.

Addendum:

"Right OK guys, meeting with the client went well, they've hired us for another project and it's a big one so we really need you to perform for the team on this... Guys? Guys? Hey, where'd everyone go?"

Re:Incentives (1)

fermion (181285) | about a year ago | (#44242551)

It is a waste of money for developers to go on bug hunts. If a customer reports a bug and a QA person confirms it,then time should be allocated to fix it, and not only fix it but hopefully to fix the process that allowed it to happen. The benefits of bug bounties, I think, is to encourage sophisticated users to give a detailed analysis of the prob,em therefore saving QA and developer time. In OSS of course the end user can fix the bug, but so can any volunteer, so not sure where the cost savings would be there.

that depends (1)

publiclurker (952615) | about a year ago | (#44243149)

on the type of bug. If it's security related, you can never be sure that the customer that finds it will report it instead of exploit it.

Re:Incentives (1)

SethJohnson (112166) | about a year ago | (#44245785)

It is a waste of money for developers to go on bug hunts.

Nonsense. The outside world can only fuzz against the product looking for a security vulnerability and they are only paid for security vulnerabilities.

Bug hunts reveal architecture missteps that will break the product during upgrades or other usage. Internal developers are aware of the architecture, so they are more able to focus on searching and finding both security vulnerabilities as well as general bugs. A testing matrix cannot predict all the things that can possibly be impacted by an upgrade, so QA is not going to find every bug. And it gets really ugly when a company releases a patch and then immediately has to release a second one to fix the things the previous patch broke.

Re:Incentives (1)

dveditz (11090) | about a year ago | (#44245531)

As the Firefox Security Manager I completely and vehemently disagree. I employ a team that spends 100% of their time "going on bug-hunts" looking for security bugs in Firefox, and I know my counter-part at Google is doing the same for Chrome. Our Bug Bounty programs (VRP? ugh, so very corporate) are an incentive for people who stumble on neat stuff to pass it on, not a substitute for doing the work ourselves.

What's an interview have to do with it? (2)

feddas (1979736) | about a year ago | (#44240589)

Mostly shows how being good at finding bugs is a different skill than being good at job interviews.

IF A TREE FALLS IN A FOREST !! (0)

Anonymous Coward | about a year ago | (#44240645)

and no one is there to hear it does it make a sound ??

NO !!

Bounties are like telling hit-men to take you out but then paying them if they tell you about it first !! Bounties are Yahoo-think !!

Wasn't that obvious? (0)

Anonymous Coward | about a year ago | (#44240647)

The bounty system is the ideal tool to exploit thousands of people working for free, so that only one has to be paid. The best thing is that you only have to pay for results.

Re:Wasn't that obvious? (1)

viperidaenz (2515578) | about a year ago | (#44245131)

And you don't have to worry about employment laws.

math problem? (2)

rst123 (2440064) | about a year ago | (#44240651)

isn't $570,000 / $150,000 about 3.8 people? (articles numbers.) Still probably a good deal, but not quite as good.

Re:math problem? (1)

mybinarylife (1921532) | about a year ago | (#44240763)

Close divide the $570,000 by 3 for the number of years the program has been running. You get $190,000 a year in bounties vs the $150,000 a year for a developer. Still not equivalent, but at least a little closer.

Re:math problem? (1)

Mouldy (1322581) | about a year ago | (#44240871)

You also need to bare in mind you still need to pay the developer to actually fix the issues uncovered by the bounty program...and any other staffing the verify incoming bugs are valid and worth paying for.

Re:math problem? (2)

plopez (54068) | about a year ago | (#44247023)

No. The developer just reports the bugs to the development team. Perhaps we should give that developer a special title like "Quality Assurance Engineer".

Re:math problem? (1)

Anonymous Coward | about a year ago | (#44240869)

isn't $570,000 / $150,000 about 3.8 people? (articles numbers.) Still probably a good deal, but not quite as good.

$570,000 was over 3 years. $150,000 was salary plus overhead for 1 year.

Instead of 570K/150K = 3.8 developers, you get 570K/450K = 1.27 developers

Re:math problem? (1)

JoshRosenbaum (841551) | about a year ago | (#44241005)

That number is over 3 years according to the summary, so you need to divide by 3.

I'd point out the numerous flaws... (0)

Anonymous Coward | about a year ago | (#44240695)

...in their methodology, but first, pony up.

dilbert (4, Funny)

Joe_Dragon (2206452) | about a year ago | (#44240723)

Re:dilbert (4, Interesting)

CastrTroy (595695) | about a year ago | (#44240767)

I wonder if anything like this is going on internally. Let's say a developer at Google knows about a problem. He could either fix it, and get his regular pay, or he could tell his friend about the bug, and split the bounty with his friend who "discovered" the bug. Either way the bug gets fixed. And it probably get's fixed faster this way, since it's now an externally known vulnerability.

That's the only way for it to be worthwhile (0)

Anonymous Coward | about a year ago | (#44241147)

I wonder if anything like this is going on internally. Let's say a developer at Google knows about a problem. He could either fix it, and get his regular pay, or he could tell his friend about the bug, and split the bounty with his friend who "discovered" the bug. Either way the bug gets fixed. And it probably get's fixed faster this way, since it's now an externally known vulnerability.

Let's put it this way, consider ALL the knowledge, experience, and hours and hours of trying to figure out these bugs.

I wonder what the actual hourly benefit is to these bug catchers.

Composer analogy: A music prof in college I had described a piece he was commissioned to write. He said he was paid $2,500 ('Ooos' from class); which based on the number of hours he put into it amounted to roughly $2.50 per hour.

Not so good.

I see some of these bounties and all I can say is that these folks ain't doing it for the money.

On the other hand, the notoriety of finding a bug with GOOGLE, I'm sure, is a GREAT resume builder!

Re:That's the only way for it to be worthwhile (2)

SJHillman (1966756) | about a year ago | (#44241525)

For a lot of these people, it might be a hobby. If it weren't for bug hunting for a bounty, they might be working on open source software instead with no payout at all. For those people, the payout is infinitely greater even if it amounts to $2.50/hr. Most people are just happy to have a hobby that breaks even, nevermind nets a profit.

Re:That's the only way for it to be worthwhile (2)

TheRaven64 (641858) | about a year ago | (#44241541)

Many of the people working on these things will also have full time jobs as security researchers. The extra financial incentive for a bug just means that they'll be applying their bug-finding technique to your codebase instead of to someone else's.

Re:dilbert (1)

shentino (1139071) | about a year ago | (#44279609)

That works well until the guy gets caught red handed passing off vulnerabilities to an outsider and not only canned but possibly jailed.

Google has shown itself to be very strict about enforcing its NDA policy, and divulging exploits to outsiders in general is a major legal risk.

Re:dilbert (0)

aaronb1138 (2035478) | about a year ago | (#44241463)

I don't understand that comic, is the funny part where the salaried employees (dumb slaves) realize that the money is in consulting?

Re:dilbert (1)

jittles (1613415) | about a year ago | (#44241545)

I don't understand that comic, is the funny part where the salaried employees (dumb slaves) realize that the money is in consulting?

No the funny part is that the boss doesn't realize that he's just written the software developers a blank check that they can write any amount they want in.

Re:dilbert (1)

GiganticLyingMouth (1691940) | about a year ago | (#44241573)

Assuming you're being serious, the joke is that they're going to introduce a lot of bugs, then get payed to fix them.

Re:dilbert (1)

parkinglot777 (2563877) | about a year ago | (#44242929)

The catch phrase is in the last sentence "I am going to write me a new minivan this afternoon." In other words, the guy is going to write a full load of bugs software and fix them. Each bug he found (intentionally left in the software) and fixed would be worth $10. If he is going to be able to buy a new minivan, how many bugs does he need to create & find?

VRPs are the new sweatshops (3, Interesting)

OleMoudi (624829) | about a year ago | (#44240733)

This is indeed true specially for popular companies with rather mature SecOps that pay minimum wages for vulnerabilities that are indeed hard to find or require a pretty darn good skill level to discover. Some of them even only offer swag in exchange of finding serious threats such as persistent XSS or authentication bypass. They maybe feature the researcher in some blog post to publicly thank him and attract the wannabe crowds.

Having said that, I myself have participated in several of these programs (with varying success) and come to realize that probably Google and Facebook are the only VRPs currently paying reasonable wages for bugs in terms of cost efficiency for the researcher.

On the other hand, some of us just enjoy from time to time trying to find security bugs for fun (maybe because we are huge nerds) so these programs offer a great opportunity to test things and not risking ending up in jail.

Re:VRPs are the new sweatshops (1)

LordLimecat (1103839) | about a year ago | (#44243435)

Then dont spend your time finding vulnerabilities for those companies?

Im not seeing the comparison to a sweatshop here.

Re:VRPs are the new sweatshops (1)

plopez (54068) | about a year ago | (#44247043)

A Ferengi would find an exploit, sell it on the black market, and then shortly there after report it. Profit!

from a moral standpoint its deplorable. (1)

nimbius (983462) | about a year ago | (#44240839)

as a corporation is abdicates you from the responsibility of things like health insurance in countries like america that have very expensive coverage individuals typically cannot afford. In more advanced countries like sweden or canada, youre indirectly allowing a government to subsidize a component of your under-the-table employment of coders and hackers. expenses like retirement, life insurance, dental coverage and the cost of work-related activities like ice cream socials are then realized as a savings. In my opinion coders and hackers must be very careful when engaging in bug bounty as the cost of a programmer including benefits is often not fully reimbursed when they find and patch a bug. even if that is not a primary consideration, the ethics of fixing googles problems are worth considering

small projects like mozilla should get to do it, as theyve consistently demonstrated a moral and ethical commitment to protecting the internet for all humankind. Google, a major multinational corporation that lobbies congress for H1B legislation, is in a bit more of a grey area. Chrome is an offering in which its user becomes the product, the final objective to sell the subjects data to various other corporations and earn a profit.

Re:from a moral standpoint its deplorable. (1)

BVis (267028) | about a year ago | (#44241433)

Morals are expensive. If you can get the same work for less money (by using a VRP) then you're doing it right, as far as the organization is concerned. They don't care about anything else.

Re:from a moral standpoint its deplorable. (1)

Aikiplayer (804569) | about a year ago | (#44242469)

I'm not sure I agree with you. I don't get the sense that they're outsourcing hacking (it's really more dev than QA but not really either). They're both crowdsourcing it and attempting to incentive the finders to report it to them vs. organizations who will use that information to create exploits, etc. I don't believe they've given up on investing their employees in attempting to create secure software, I think they're supplementing those efforts this way (and other ways as well).

QA Finds Bugs, Devs Fix Them (1)

Anonymous Coward | about a year ago | (#44240845)

Maybe they should have compared the salary of a QA person instead of a developer. As a developer, I find lots of bugs, and then fix them. I also fix the bugs that QA finds, but usually spend a lot of time trying to figure out how to reproduce the issue ("uhh, first I clicked on this and then I clicked on that and then something weird happened").

Anywhile, it's hard to crowd source a product that has not been released yet and most companies don't have the fan-bois and gurls to even consider this strategy.

Re:QA Finds Bugs, Devs Fix Them (1)

Captain Hook (923766) | about a year ago | (#44241365)

but usually spend a lot of time trying to figure out how to reproduce the issue

You need better test employees.

Re:QA Finds Bugs, Devs Fix Them (1)

mattack2 (1165421) | about a year ago | (#44246277)

Wait, you're saying QA isn't writing bugs that have detailed steps to reproduce? If they are ones that can be easily reproduced, then they're not writing good bugs. (Obviously there are lots of bugs that are worth writing up that DON'T yet have reproducible cases yet.. sometimes a conglomeration of those not-reproducible cases can lead to a reproducible case too...)

BTW, while I definitely think that 'bug bounty' isn't as good as a company finding its own bugs, I wish ALL companies had an official way to report bugs. I'm a huge Tivo fan for example, but there are lots of reproducible issues in them.. and only recently did they start their official forum. (That's not the same as a bug reporting system, but I have used it as such.. and in fact have thought of starting a thread at the major *unofficial* Tivo discussion forum just to have links to known bugs written up on their official forum.)

It does not work if Wally is in the team. (0, Redundant)

140Mandak262Jamuna (970587) | about a year ago | (#44240861)

When the PHB announced a bug bounty program, Wally vowed to write himself a new car that afternoon.

Re:It does not work if Wally is in the team. (1)

Anonymous Coward | about a year ago | (#44241933)

I believe it was a minvan

Cost of fiunding bugs != cost of fixing them. (5, Insightful)

140Mandak262Jamuna (970587) | about a year ago | (#44240911)

Browsers have very large installed base. There are enough bug spotters even if a very small fraction of them actually hunt and report bugs. Even then, the bounty is for finding the bugs, not fixing the bugs that includes the cost of coming up with a fix, verifying it fixes the problem, testing to make sure it does not create new problems and rolling out the fix.

Re:Cost of fiunding bugs != cost of fixing them. (0)

Anonymous Coward | about a year ago | (#44241215)

So then, after coding in bugs to screw with the boss we can get paid to debug the bugs? Fantastic!

Re:Cost of fiunding bugs != cost of fixing them. (0)

Anonymous Coward | about a year ago | (#44243705)

Still 90% of the effort in fixing a bug is often finding the cause. I presume they pay better for bug reports that list the cause (and code lines) than those that simply state the buggy behavior.

Fail logic.... (0)

Anonymous Coward | about a year ago | (#44240913)

So if they cut the developer & the tester (bugs get found/fixed in teams), they get a lot more bugs released into the wild, a lot more bad PR, and lot more bad user experience.... oh, but it's fine, they're saving a few thousand bucks a year..

this is very bad (1)

stenvar (2789879) | about a year ago | (#44240965)

That means that there is a strong incentive for companies to create insecure, crappy software and then let so-called "white hat hackers" fix their bugs at a discount. And because any other form of disclosure is illegal, the companies are pretty well protected from negative consequences of their bugs and deflect from their own negligence by blaming "black hat hackers".

...at creating mystery meat. (1)

sethstorm (512897) | about a year ago | (#44240987)

Given that disclosure is also at the terms of the payer, you also get less transparency versus independent disclosure.

Advertising Benefits (1)

thejahn (617263) | about a year ago | (#44241031)

This study may neglect to mention the benefits of advertising and exposure even if only ten people found the bugs.

Why employees don't find these bugs (2, Interesting)

ulatekh (775985) | about a year ago | (#44241037)

Because the sort of programmer that's good at finding/fixing these bugs...is not the sort of programmer that the interview process determines would be a "good fit" for the organization.

Re:Why employees don't find these bugs (-1)

Anonymous Coward | about a year ago | (#44241691)

Oh dear fucking god stop being so pretentious. You don't have to be some sort of sniggering sociopathic lunatic with a power complex living in a cave to find security bugs.

Re:Why employees don't find these bugs (0)

Anonymous Coward | about a year ago | (#44241791)

Grow up.

Re:Why employees don't find these bugs (0)

Anonymous Coward | about a year ago | (#44242209)

they wouldn't be good at finding the bugs if they were employed there, they would be tasked with other stuff.
they wouldn't be let to have the time and would be bitched to if they did use the time. the guy who wrote the bugs would think if it's worth fixing them or writing new code that might bury them.. so he orders you to write new code, not to fix the old design he did earlier.

Re:Why employees don't find these bugs (0)

Anonymous Coward | about a year ago | (#44242767)

Nonsense. The sort of programmer who wants to poke around finding bugs is not a programmer, he's a tester.

Ineffective, unfortunately (3, Interesting)

gweihir (88907) | about a year ago | (#44241049)

This is effective for the low-hanging fruit, i.e. the easy (relatively) to find security-related bugs. For things that require advanced techniques or expensive tools (like Fortify), it fails. Unfortunately, the harder to find bugs are still well within reach of spy agencies of all kind, including a number that is allowed to do industrial espionage (like the US or France).

So while this looks good on the surface, it is really just making the problem worse. The only exception is software that has very low security needs.

For reliability, it is about as ineffective, as only easy to identify bugs will be tracked down.

$100,000 bug finder (1)

jimbolauski (882977) | about a year ago | (#44241103)

I would hope Google is smart enough to know that you don't need an experienced developer to find bugs in their code. Aspiring developers fresh out of college are more then adequate. At 50k a pop google could have hired 7 PFYs spending 14,000 hours scouring code, hell give them 1k bonuses for each bug to keep them motivated.

No shit Sherlock (2)

Fuzzums (250400) | about a year ago | (#44241243)

What I'm really shocked about is that you need a university to figure this out. Or rather do research on this. Companies figured this out quite some time ago and anyone with a functioning brain can see why.
What I'm more interested in is that king of people spend their time in participating in programs like this. The chances that you find a bug are not that big. The financial reward, given the amount of time you will spend on finding a bug is probably also relatively small.
From a company's point of view on the other hand, it's great. Many people working for you. For free. A job well done :)

Even better if the checks aren't cashed (1)

WillAdams (45638) | about a year ago | (#44241367)

or one instead offers ``certificates of deposit'' in the (fictional) ``Bank of San Seriffe'': http://en.wikipedia.org/wiki/Knuth_reward_check [wikipedia.org]

William
(who is quite bummed that he didn't get his reward check back when Dr. Knuth was using Wells Fargo as his bank: http://www.truetex.com/knuthchk.htm [truetex.com] )

Cost without quantification (0)

Anonymous Coward | about a year ago | (#44241455)

These amounts of money were paid for the bugs that bounty hunters bothered to report.

The question is, how many bugs still exist in the software, and of those, how many have been discovered and not reported?

All this article is doing is confirming that, in this case, if you make an effort to minimize spending, your costs can be reduced.

It does not say what kind of value you are receiving for what you pay.

Not a replacement (2)

gmuslera (3436) | about a year ago | (#44241643)

Is good to reward people that find security holes, at the very least because is a safer bet than selling them in the black market, or keeping them for yourself or the government to exploit them. But it should not be a replacement for actually having dedicated people activelly working for your security that will report to you if something weird is there, some could actually go to the black market (or be found by government teams and never disclosed that it is there because is an useful cyberweapon) and you must be proactive from your side

Not as easy as it sounds (0)

Anonymous Coward | about a year ago | (#44241733)

Developers are not paid to be dedicated bug hunters. They deliver new code and maintain the code base. They should be trained in security, but the expectation of flawless perfection in secure coding is absurd.

At many software companies, a dedicated person or team performs the audit and pen testing function. The number of issues the team finds are exponentially more than are reported. And many are never acknowledged, but are fixed in the latest releases. Stay up to date, really.

The article gives examples of companies that have just a few products. Consider the budget of running a bug bounty program for a company that has 50-300 products - it would easily run into the millions.

Bounty programs are great if the money is there (I'm looking at your billions in profit Microsoft!) But also think about companies who barely profit and have greater needs for survival.

Every piece of software you use likely is vulnerable to something that is not yet discovered, fixed, or publicly disclosed. It's about what the company is doing to reduce those vulnerabilities and how well they handle the inevitable vulnerability report.

Average? Where? (1)

Anonymous Coward | about a year ago | (#44241801)

average annual cost of a single North American developer (about $100,000)

It sure isn't the average in Canada.

Re:Average? Where? (1)

HornWumpus (783565) | about a year ago | (#44242117)

So what? You've got 3 developers. Not like your going to move the average.

Re:Average? Where? (2)

cbhacking (979169) | about a year ago | (#44242555)

High-tech regions tend to be high cost of living, too. In Silicon Valley, 100K USD may well be better than an entry-level salary for a dev with a 4-year degree. The cost of living is so high that this is less impressive than it sounds, though. It's a little less bad up around Seattle ("Silicon Forest") where starting salaries are more commonly in the 70-90k range, but people break six digits very quickly. I haven't job-hunted anywhere else, but at least on the west coast, a 100k estimated average might actually be low. People with more than 5 years in the industry can probably pull at least 50% more than that if they're any good. Also, that's just for a BS; get a MS or a PhD and you can definitely start at or above 100k (and yes, in this field there are high-paying jobs in industry for folks with doctorates).

Typical business-centric bullshit reporting (2)

musth (901919) | about a year ago | (#44241845)

It's not surprising at all that piecemeal work, with no provision for healthcare, vacation etc. - much less reliable, ongoing income - is more profitable for business.

Why should technology workers be intrigued or inspired by this? Why is this information presented to technology workers as another avenue to praise Google's or Mozilla's cleverness? And why do technology workers so consistently dig their own graves by latching onto this kind of ideology and failing to fight for labor rights?

Well, yes, it'll work for browsers... (2)

gestalt_n_pepper (991155) | about a year ago | (#44241941)

where you have millions of folks looking at your free software for long periods of time. If you're a commercial software vendor, however, with a $10,000 non web-based package and at most a few thousand users (There are still a *lot* of these), then this approach is very unlikely to succeed. Commercial software users are rarely interested enough to report a bug that doesn't actively interfere with their daily work.

QA? (1)

Pro923 (1447307) | about a year ago | (#44242251)

What the hell does QA do these days anyway?

Re:QA? (1)

lister king of smeg (2481612) | about a year ago | (#44245239)

get outsourced to india or cut entirely?

Of course (0)

Anonymous Coward | about a year ago | (#44242515)

Considering that they've both been caught stiffing the crowd when crowd-sourcing, it's little wonder that they're saving money with this approach: you stiff an employee, and you'll get sued.

Once again... (0)

Anonymous Coward | about a year ago | (#44242537)

... software developers contribute to their own devaluation. Imagine plumbers or electricians getting together in their spare time to repair or upgrade a corporate building, in the vague hope that the corporation in question might give them a few beers, or a cap with a logo. This is why the management class continues to see IT / development as a bunch of easily satiated morons, who are more than willing to work hard for free pizza and swivel chairs.

The longer I'm in this field, the more convinced I am that it is not a long-term career option for grown-ups. Become a consultant (and hope no one reads past the weasel-words) or choose a technical career with a focus on physical deliverables.

cost effective (0)

Anonymous Coward | about a year ago | (#44242567)

Ha! Because the real money is in selling these bugs to foreign governments!
Cost effective!? Yeah, I'll fix your bugs for $1 million, since that's what I can make
selling them to France.

Can't see the forest for the trees (0)

Anonymous Coward | about a year ago | (#44242585)

So what this encourages shops to do is NOT find/fix bugs (that could ruin their credibility) and instead simply release the software to the general public (lurking zero-day exploits et al) for the masses to find and suggest bug fixes. All the while, innocent consumers of the software have little guarantee that a good-faith effort was made on the part of the software-maker that they are PURCHASING software that will perform as advertised without exposing the user to undue risk.

Tell me companies won't be stupid enough to overlook this fact?

A contest would be more efficient yet! (0)

Anonymous Coward | about a year ago | (#44242689)

Major companies should have a contest, because then they'll get free work from everyone who enters and only have to give out one award. The current way they're doing it could be made more efficient.

Lottery is still a tax on idiocy (0)

Anonymous Coward | about a year ago | (#44242785)

While this bounty practice is now more or less restricted to software bugs, I sure hope it will never extend it's reach to other areas of software development and that it'll prove to be a vain and passing fad. Sure, the high bounties are appealing and so is the prestige that comes with them but this sort of "winner takes all" approach is tremendously wasteful regarding the work values of discarded competitors - not to mention, it's against the very idea of collaboration and one of the best way to polarize the software development community. Of course, it's no surprise that this activity is highly profitable to the lottery owners (is there really such a thing as a bankrupt casino?). In the end, I really hope those bounty programs will be regarded only with contempt and disdain by developers for it's really a farce towards our profession. (And for those wondering: no I'm not bitter about these bounty programs because I've wasted my time on the; I make a point not to participate in it).

   

Re:Lottery is still a tax on idiocy (0)

Anonymous Coward | about a year ago | (#44247613)

Bug bounty is a job!

Developrs don't find bugs testers do. (0)

Anonymous Coward | about a year ago | (#44242871)

Testers do most of the testing, and they're cheaper. Also, you get ehat you pay for, naive testers file stupid baive bugs, not the ones that really ferret out the big problems.

Probably not a replacement for full time employees (1)

Error27 (100234) | about a year ago | (#44242901)

I get paid to audit code, so I'm biased.

The article says that no one employee could find hundreds of bugs and that's true. But when you hire employees you are building a process. Improving the process by writing a new QC script can eliminate hundreds of bugs over a couple years. These are not attributed to one employee and since the offending code is not committed then they aren't even counted as bug fixes.

Offering a bug bounty, on the other hand, is a unpredictable thing and you'll get random fixes. It is valuable because it provides a fresh perspective.

My guess is that if you collect a few bug bounties then Google will send you a recruiting email. It might be more expensive to hire you to work full time it's still a worthwhile thing.

developers don't create bugs........(kinda) (1)

chris.alex.thomas (1718644) | about a year ago | (#44243019)

developers don't "create" bugs, we don't sit down and say, hey, lets create a bug! and then go about making one, most of the time, we believe our code doesn't have bugs, cause if we thought it had bugs, we'd write it a different way, or we'd know about the bug in the first place and it'd be in our list of things to fix, normally those things are fixed quickly because we knew it was there, but things we don't know about, well, how do you expect us to find it? I didn't find it whilst I was writing the code and I'm the brain doing the typing, if I can't find it then, what do you think the chance of me finding out afterwards will be?

other people are very good at finding bugs in my code, if they exist, because they have a different mindset and think about things differently than I do, they think of a circumstance I didn't think of, great, you found the bug! but I didnt know it was there.....

so I think it's kind of normal that other people will find bugs in your code that you didn't know existed, so it makes perfect sense to reward those people to find them, paying ME to find them is going to not pay off in a big way, cause if I knew how to find bugs in my own code, I would have done it already and fixed it.

Researchers should compare vs QA (1)

HockeyPuck (141947) | about a year ago | (#44243135)

So UC Berkeley should compare the number of bugs found by researchers vs the number that Google's Internal QA Dept has found.

Then we'll know if it was really worth while. Since Google would never publish the number of bugs they find internally, all this data is worthless.

It is nice though getting people to QA your projects for basically free.

Not a bulletproof solution (0)

Anonymous Coward | about a year ago | (#44243385)

In the world where the Microsofts(among) others demand an NDA to go along with the bug, many reported bugs stay unfixed.

What I've seen by software companies (0)

Anonymous Coward | about a year ago | (#44247303)

I have some experience with (very closed source) software companies and bugs. They had a bugs list that users would report. They would rank customers by seat licenses, and give customers votes based on the number of seat licenses they had to vote on which bugs should be fixed. The company had an idea of how hard it would be to fix each problem, and if a bug didn't have the right threshold of votes, it wouldn't be fixed.

It's not about the bounty...it's about coverage (0)

Anonymous Coward | about a year ago | (#44250759)

As the saying goes, an extra pair of eyes goes a long way...having numerous pairs goes even further, up until a certain point of diminishing returns and lower signal to noise ratio. Even so, I believe the real value of these programs is the fact that they're tapping into endless combinations and permutations of testing environments that in-house developers do not have access to. I've heard this approach labeled 'in the wild' testing that differs from 'in house or in the lab' approaches. Interesting and apparently quite effective.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?