Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Microsoft Fuzzing Botnet Finds 1,800 Office Bugs

timothy posted more than 4 years ago | from the running-through-the-possibilities dept.

Programming 111

CWmike writes "Microsoft uncovered more than 1,800 bugs in Office 2010 by tapping into the unused computing horsepower of idling PCs, a company security engineer said on Wednesday. Office developers found the bugs by running millions of 'fuzzing' tests, a practice employed by both software developers and security researchers, that searches for flaws by inserting data into file format parsers to see where programs fail by crashing. 'We found and fixed about 1,800 bugs in Office 2010's code,' said Tom Gallagher, senior security test lead with Microsoft's Trustworthy Computing group, who last week co-hosted a presentation on Microsoft's fuzzing efforts at the CanSecWest security conference. 'While a large number, it's important to note that that doesn't mean we found 1,800 security issues. We also want to fix things that are not security concerns.'"

Sorry! There are no comments related to the filter you selected.

xkydgtufhlofhil (5, Funny)

Anonymous Coward | more than 4 years ago | (#31705036)

ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn

Re:xkydgtufhlofhil (3, Interesting)

troll8901 (1397145) | more than 4 years ago | (#31705200)

ghulkgiplgbvihlnk luioguilgil.bjohj110-o; Huto;bn

I don't understand this Score:4 Insightful comment. Can someone explain?

Re:xkydgtufhlofhil (5, Informative)

sucker_muts (776572) | more than 4 years ago | (#31705212)

don't understand this Score:4 Insightful comment. Can someone explain?

Even though your name does look quite suspicious, I'll try to explain anyway.

The parent is showing how fuzzing works:
Using random 'data' to test the various functions of software, so we can find out if a certain piece of input triggers undesirable behavior.

In this case you could say that he's not only giving an example, but is testing the slashdot user comments code as well.

But it's perhaps more an attempt at humor. :-)

Re:xkydgtufhlofhil (2, Funny)

Anonymous Coward | more than 4 years ago | (#31705326)

Windows: "It's not a bug, it's a feature."

GNOME: "It's not a bug, it's a design decision."

Re:xkydgtufhlofhil (1)

TimSSG (1068536) | more than 4 years ago | (#31705538)

COREL: It's not a bug. It's a WAD (Works as Designed). Tim S.

Re:xkydgtufhlofhil (0)

Anonymous Coward | more than 4 years ago | (#31705546)

Apple: It's not a bug. would you like to pay more?

Re:xkydgtufhlofhil (0)

Anonymous Coward | more than 4 years ago | (#31705594)

Mozilla: "It's not a bug, it's a feature."

Re:xkydgtufhlofhil (3, Insightful)

elronxenu (117773) | more than 4 years ago | (#31705878)

Linux: "It's not a bug, not any more."

Re:xkydgtufhlofhil (1)

DMUTPeregrine (612791) | more than 4 years ago | (#31708298)

Python: Oh, I fixed that last night, it's all ready in the repo.

Re:xkydgtufhlofhil (5, Informative)

jonadab (583620) | more than 4 years ago | (#31705558)

Except that, in most cases, random letters in the ranges a-z and A-Z are not where you're going to find most of your problems. The major sources of bugs that can be uncovered by random data are assumptions that programmers (sometimes subconsciously) make about what the data are going to be like.

The most obvious of these are assumptions like "a newline can't occur in a single-line field" (a mistake web developers often make, because they assume the data are coming from an HTML input element that only allows single-line data; but an attacker can in fact send anything they want in an http request), or "nobody's going to have a single-quote character in their name" (hello, SQL injection attack). This sort of thing is probably not a major factor in Office, because it's common for documents to have those kinds of characters in them. There might be a couple of weird old control characters (like the ASCII NUL, 000), but those bugs were probably found aeons ago.

A second major category of problematic assumptions assumptions has to do with languages and code pages and character sets. When software that was written to assume a particular character set (like ASCII, or Latin-1) or even just one code page at a time (like, whichever one is the system default) has to be extended to support more (like, especially, Unicode), you run into all kinds of nasties. Again, though, Office probably had to deal with these issues a couple of versions ago. They may have found a few more, but at this point it's probably not the most fertile ground any more.

When you're dealing with file formats, however, there are also things like "the value at offset 0x003C from the beginning of the object header contains the size of the object, which can never be more than 0xFFFF" and "an object can embed another object by referencing it, but there are never any circular references, because the software doesn't allow the user to put an object inside itself". These sorts of assumptions pop up every time you write or change code that reads a file format, so they never go away really. This sort of thing is probably *most* of what the Office team found, I suspect.

Re:xkydgtufhlofhil (3, Informative)

Helen O'Boyle (324127) | more than 4 years ago | (#31707016)

"nobody's going to have a single-quote character in their name" (hello, SQL injection attack)

Hey, I resemble that remark! And yes, it's resulted in chuckles over the years. Microsoft, DevelopMentor, random e-commerce sites... many have fallen to the Irish. When talking to security professionals, I introduce myself as "the woman whose name is a SQL injection attack", and it seems to help them remember me.

Re:xkydgtufhlofhil (0, Offtopic)

jank1887 (815982) | more than 4 years ago | (#31705766)

somewhat relevant xkcd:

Exploits of a Mom
http://xkcd.com/327/ [xkcd.com]

Re:xkydgtufhlofhil (1)

troll8901 (1397145) | more than 4 years ago | (#31706206)

Even though your name does look quite suspicious, I'll try to explain anyway.

Thank you for your explanation.
And your benefit of the doubt.

Re:xkydgtufhlofhil (2, Funny)

mobby_6kl (668092) | more than 4 years ago | (#31706998)

>In this case you could say that he's not only giving an example, but is testing the slashdot user comments code as well.

It's testing not just the user comments code, but also the moderation system code and the moderators themselves. In this case, it appears that he found a bug which causes the comment to be moderated Insightful by providing a certain combination of random characters as input. I will now attempt to replicate this problem.

______TEST DATA FOLLOWS______
TvaHokVAwgZGLrzPnDsIzHnKwuOOQEgaFskFJx-9JH@eIbwWSYhujyXDekeBP-9YQlfiZtdOZXlupfvy
UYXenTsWzzF#SScvbvWXtMMcbMg@xIsRC!OiViEDnt-9fQRGXEgvbfdlBATolRyiVYmcKyHi-9bLVcYx
JrPmw

Re:xkydgtufhlofhil (2, Informative)

msclrhd (1211086) | more than 4 years ago | (#31705218)

Fuzzing is a technique where you modify the data sent to a file, protocol or data parser (e.g. code that reads an xml file) by changing random bits. Thus, if you have a 'text' command, a fuzzer could change that to 'next', or if you have a quoted striing "test", the fuzzer could change the end quote to something else, e.g. ' "tests '.

Hence, what you can end up with is something that looks like random garbage.

Or user a sales. (2, Interesting)

leuk_he (194174) | more than 4 years ago | (#31705262)

It is an alternative to the monkey test: Take a sales person from across the ahlloway and let him click on your application. If it does not crash or give absurd error messages you can do the actual testing.

GIGO!

Re:xkydgtufhlofhil (1)

jackal40 (1119853) | more than 4 years ago | (#31705462)

Well color me red, here I thought this kind of testing should have been done prior to release. Guess the new model of software development is to have the users discover the bugs (can I get a smiley on this) instead of paying a QA team to test.

Re:xkydgtufhlofhil (2, Insightful)

nmb3000 (741169) | more than 4 years ago | (#31707254)

Well color me red, here I thought this kind of testing should have been done prior to release. Guess the new model of software development is to have the users discover the bugs (can I get a smiley on this) instead of paying a QA team to test.

No, color you stupid. Office 2010 hasn't been released yet.

Nice try though.

Dont wait for Office 3000 (0)

Anonymous Coward | more than 4 years ago | (#31705038)

1800 in Office 2010? then 210 issues to go...

Hey, Microsoft! (5, Funny)

geminidomino (614729) | more than 4 years ago | (#31705040)

"We also want to fix things that are not security concerns."

It's 5AM EST. April Fools' day is over everywhere but a few pacific islands. Give it up already.

Re:Hey, Microsoft! (2, Funny)

somersault (912633) | more than 4 years ago | (#31705052)

While a large number, it's important to note that that doesn't mean we found 1,800 security issues

Don't worry, we all know that you haven't fixed any security issues.

Re:Hey, Microsoft! (4, Insightful)

PolygamousRanchKid (1290638) | more than 4 years ago | (#31705056)

Note that he said "want" and not "will".

Re:Hey, Microsoft! (1)

game kid (805301) | more than 4 years ago | (#31705264)

...which brings us back to where we started; they've clearly fixed n security issues, where 0 <= n <= 1800. :)

Re:Hey, Microsoft! (0)

Anonymous Coward | more than 4 years ago | (#31707470)

As somebody who will not be forking out for Office 2010, it would be nice if they fixed some of the flaws in their previous Office products. The ability to insert multiple images into Office 2003 without Clippy putting the pics where he would like them rather than where I would like them would be a start.

Re:Hey, Microsoft! (1, Funny)

Arancaytar (966377) | more than 4 years ago | (#31705070)

It would actually be believable except for the "also". :P

New bugs (5, Insightful)

El_Muerte_TDS (592157) | more than 4 years ago | (#31705046)

I wonder how many "new" bugs they'll create by fixing the found bugs.

Anyway, nice to see that they're performing fuzzing tests, not enough people/companies do that. There's also quite little tool support for it.

Re:New bugs (2, Interesting)

GF678 (1453005) | more than 4 years ago | (#31705152)

I wonder how many "new" bugs they'll create by fixing the found bugs

Yeah, just like the numerous regressions I see in the Linux kernel, WINE, Ubuntu releases etc.

Re:New bugs (1)

LinuxIsGarbage (1658307) | more than 4 years ago | (#31705774)

I wonder how many "new" bugs they'll create by fixing the found bugs

Yeah, just like the numerous regressions I see in the Linux kernel, WINE, Ubuntu releases etc.

Why is this modded offtopic? It's cool and popular to poke fun at Microsoft but heaven forbid you point out Linux, WINE, and Ubuntu have regressions?

Re:New bugs (1)

John Hasler (414242) | more than 4 years ago | (#31707020)

> Why is this modded offtopic?

Because it is.

> It's cool and popular to poke fun at Microsoft but heaven forbid you point
> out Linux, WINE, and Ubuntu have regressions?

Why is the fact that other software also has regressions relevant? Do you think that is news to anyone here?

Re:New bugs (4, Funny)

beakerMeep (716990) | more than 4 years ago | (#31705206)

fuzzing tools probably wont ever gain wide spread acceptance outside of the furry community though.

Re:New bugs (0)

Anonymous Coward | more than 4 years ago | (#31705480)

fuzzing tools probably wont ever gain wide spread acceptance outside of the furry community though.

Of course not. You need a yiffing tool for that.

Re:New bugs (0)

Anonymous Coward | more than 4 years ago | (#31705426)

I wonder how many "new" bugs they'll create by fixing the found bugs.

Just how is this an insightful comment?

I wonder how many "new" bugs any software developer create by fixing the found bugs.

Re:New bugs (1)

anexkahn (935249) | more than 4 years ago | (#31706624)

My teachers in school used to say for every bug you fix, you have a 50% chance of creating two more :)

Fewer than 1800. (0)

Anonymous Coward | more than 4 years ago | (#31708414)

Any company that develops software has to start with the premise that developers fixing bugs create fewer bugs than they fix, otherwise they should get out of that business and start selling donuts instead.

Hmmm (1, Flamebait)

jocabergs (1688456) | more than 4 years ago | (#31705066)

Not sure if I buy the part about them trying to fix the non-security issue bugs... I think the proposed fix for bugs in 2007 is $300 for 2010, but its by no means a comprehensive fix.
(I'm coming from a bitter place, I've been stuck going through idiotic publisher files for the last 3 days and I'm certain it was designed by monkeys(or for them))

If only this was easier... (2, Interesting)

Manip (656104) | more than 4 years ago | (#31705072)

This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms. While I'm sure we could all drop entirely random data into a parser and see if it fails, to REALLY conduct a test you have to do the same thing broken down by data element in the file format and then for each of those test both realistic and unrealistic test cases.

Then you throw on top of that UI and Web-Page fuzzing and you now have to somehow hook every element on a site and throw in random data which is not realistic with a large rich application.

Re:If only this was easier... (4, Informative)

somersault (912633) | more than 4 years ago | (#31705104)

The whole point of the data is that it's unrealistic. There are a few tools out there for doing this type of testing, or easily modified to do it. I haven't used many testing tools but you could take something like Skipfish [google.com] and add in some fuzz testing pretty easily.

Re:If only this was easier... (5, Insightful)

owlstead (636356) | more than 4 years ago | (#31705178)

As with all testing tools, the more of them you use, the better. There are many reasons why you don't want to employ all tests, e.g. lack of knowledge, lack of manpower, lack of money or lack of time. The good thing is that if you can get them automated, then they quickly become affordable.

For an example: I was thinking if it was wise to put findbugs (which works on compiled byte code) next to checkstyle (which works on source code level) in my Java project. Obviously I put them both in; they duplicate bugs but who cares ? I'll just look at checkstyle first and findbugs second. If I can put in a pre-build fuzzing component I probably will.

But fuzzing tools are different than unit tests. Fuzzing can never cover every nook and cranny. They will produce reports that are much less readable, and that cannot be directly tied to particular events (e.g. during regression testing). If anything, they'll put some pressure on developers to put in more unit tests; if the fuzzing tool finds many bugs in a component, it should be a good indicator that even the basic unit tests have not been created.

Re:If only this was easier... (0)

Anonymous Coward | more than 4 years ago | (#31705334)

Using testing tools is good, but when you uncover that many bug, it means something is wrong in your developing method. Parsing a well formed language is not that hard, implementing a sane syntax tree and graceful failures isn't either.

Microsoft is notorious for taking 5 or 6 versions of a product to get it "right enough", I strongly suspect that they would greatly improve their quality by having a solid theoretical ground first, then code instead of trying to get the 1.0 out as quickly as possible. Of course, I am not saying this is wrong business-wise, just that this is wrong from a software quality point of view.

Re:If only this was easier... (2, Interesting)

digitig (1056110) | more than 4 years ago | (#31705836)

The solid theoretical ground would be fine if they were starting from scratch now (and looking at some of the research coming out of MS, they do seem to be trying for it -- we'll have to wait and see whether it delivers). One big problem for them, though, is maintaining compatibility with ealier versions of Office which were not written using what is now current best practice. Once you start trying to implement code with behaviour that's not properly understood, or pulling in code that's not properly understood, then that best practice is some help, but it doesn't give you the robustness you might want. The alternative would be to abandon back-compatibility, but that would throw away all their (perceived) lock-in and make it too easy for customers to jump ship, so that would probably be a bad business decision.

Unit tests don't find everything either (1)

tepples (727027) | more than 4 years ago | (#31705392)

But fuzzing tools are different than unit tests. Fuzzing can never cover every nook and cranny.

Neither will unit tests [c2.com] .

Re:If only this was easier... (2, Informative)

SharpFang (651121) | more than 4 years ago | (#31705248)

A fuzzer isn't really hard to write.

Pick a word-based variant of Dissociated Press [wikipedia.org] that requires similarity a random number of words back/ahead and allows split on special characters (separators) besides whitespaces. Feed it a lot of your actual files. Actually, the amount of data it can produce may be vastly bigger than the amount of data it takes in, because it can jump back and forth in the input files recombining their fragments multiple times.

Of course then you need a test unit that feeds the fuzz to your program.

Re:If only this was easier... (1, Informative)

Anonymous Coward | more than 4 years ago | (#31705266)

This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms.

Microsoft runs huge (and I mean huge) server farms for all kinds of internal testing - unit tests for rolling builds, automated functional tests for the same, performance regression tests, compatibility tests (what if we run it on Vista without SP1 and with Office 2003 with latest updates installed?..) - you name it, it's there.

But, even with all the servers, it still takes hours for a complete test run.

One would think that this is the case... (2, Interesting)

WD (96061) | more than 4 years ago | (#31705296)

What you describe is "smart" or "generational" fuzzing, where you have a detailed knowledge of the target that you are fuzzing. The thing is, dumb (mutational) fuzzing is still effective. Very effective. Check out Charlie Miller's CanSecWest presentation - An analysis of fuzzing 4 products with 5 lines of Python
http://securityevaluators.com/files/slides/cmiller_CSW_2010.ppt [securityevaluators.com]

In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.

Re:One would think that this is the case... (1)

amorsen (7485) | more than 4 years ago | (#31705482)

In 3 weeks of (really) dumb fuzzing, 174 unique crashes in PowerPoint were discovered.

The fuzzing was dumb, but the picking of files as basis for the fuzzing was smart. Unfortunately Charlie Miller doesn't present a tool for doing that.

Re:If only this was easier... (0)

Anonymous Coward | more than 4 years ago | (#31705568)

This is a great methodology of testing but to be honest I'm not sure it is within the scope of most software firms.

Any firm whose software takes input should use it. The input can be from input files, stdin, the network, or command-line arguments. If you have a CLI utility, and you try entering a 2457 byte string into a command line argument for a file path (given that POSIX's PATH_MAX is 1024 and NAME_MAX is 255), how will your program deal with it? Will there be a buffer overflow or will it fail safely?

While I'm sure we could all drop entirely random data into a parser and see if it fails, to REALLY conduct a test you have to do the same thing broken down by data element in the file format and then for each of those test both realistic and unrealistic test cases.

Ideally you would have a spec of the file format, and you could automated generating generating and consuming the files via library calls.

Re:If only this was easier... (2, Interesting)

BitZtream (692029) | more than 4 years ago | (#31706160)

Its only a great model for testing if you've exhausted the extensive list of known bugs that people hit every day under common circumstances.

Finding bugs in the file format is great and all, but fixing the bugs that users actually see every day is far more important and you can reset assured it will be released with a bucket load of very obvious bugs that should have been fixed rather than dicking around throwing random data at it.

I know there are potential security issues to deal with and those are important, but they still aren't as important as the users experience with the software and actually getting their own job done. Saying 'don't open word docs from someone else until this is fixed!' is a lot more practical than hearing that person say 'I'm not using Word, this retarded table layout bug is pissing me off, can we find something else to use instead of Word?'

I'm not saying they shouldn't, I'm just saying their priorities are wrong on a scale that is hardly imaginable.

As far as being realistic with a large rich application ...

Citation Needed.

Given enough processors sitting around the size of the application or its feature set becomes of little concern. It may take longer but thats no excuse to not do it.

Re:If only this was easier... (2, Interesting)

plover (150551) | more than 4 years ago | (#31706526)

I wouldn't knock what they're doing. As we've recently seen with Adobe, exploits in the payload format can be used to manipulate users and even launch code. And remember how we used to be all panicky about Word macro exploitations until the defaults were changed to shut them off? "Good times", indeed.

Consider that Microsoft dominates the market, and that the ".DOC" format is widely accepted across companies. Nowadays .DOC files are readily passed by email filters, web filters, etc. Office workers open them in previewers and Word without giving a second thought to security.

A buffer overrun or other fault in the handling of .DOC files could offer a hacker a way to deliver a malicious payload through channels that are today trusted worldwide. For all we know, these could already be exploited by phishing attacks.

It's definitely worth Microsoft's time and effort to execute these tests.

"Botnet?" (1)

oldhack (1037484) | more than 4 years ago | (#31705076)

What's the connection with "botnet"?

Re:"Botnet?" (4, Funny)

nacturation (646836) | more than 4 years ago | (#31705088)

FTFA:

Microsoft was able to find such a large number of bugs in Office 2010 by using not only machines in the company's labs, but also under-utilitized or idle PCs throughout the company. The concept isn't new: The Search for Extraterrestrial Intelligence (SETI@home) project may have been the first to popularize the practice, and remains the largest, but it's also been used to crunch numbers in medical research and to find the world's largest prime number.

"We call it a botnet for fuzzing," said Gallagher, referring to what Microsoft has formally dubbed Distributed Fuzzing Framework (DFF). The fuzzing network originated with work by David Conger, a software design engineer on the Access team.

Odd that they would call it that publicly, given the negative connotation of the word. I would have called it "fuzzy clouds grid computing" or something like that.

Re:"Botnet?" (1)

tagno25 (1518033) | more than 4 years ago | (#31705120)

I would have called it a "Fuzzed Cluster"

Re:"Botnet?" (4, Funny)

El_Muerte_TDS (592157) | more than 4 years ago | (#31705184)

"Cluster Fuzzed" would be much better, specially when somebody finds a remote exploit in their cluster code, then Microsoft will be cluster fucked.

Re:"Botnet?" (2, Funny)

laederkeps (976361) | more than 4 years ago | (#31705190)

So the project is a "Cluster fuzz" ?

Re:"Botnet?" (1)

flyingfsck (986395) | more than 4 years ago | (#31705228)

A 'Cluster Fsck' would be even better.

Re:"Botnet?" (1)

anarche (1525323) | more than 4 years ago | (#31705328)

or Cluster Ftck?,

its only one bit they're flipping...

Re:"Botnet?" (1)

Nadaka (224565) | more than 4 years ago | (#31705488)

If you are flipping only one bit in a "Cluster Fsck", you are missing the point and could be having a lot more fun.

Re:"Botnet?" (2, Informative)

shutdown -p now (807394) | more than 4 years ago | (#31705274)

Odd that they would call it that publicly, given the negative connotation of the word. I would have called it "fuzzy clouds grid computing" or something like that.

Developers tend to name things that are used internally in a way that is short and more to the point, which is not necessarily something perfect for marketing/PR.

Sometimes these things slip through.

Re:"Botnet?" (4, Funny)

Mathinker (909784) | more than 4 years ago | (#31705096)

Let me explain: Microsoft discovered that all of their desktop computers were zombied with malware, and after wresting control from the botnet C&C, decided to take advantage of this increased ability to remotely administer their computers to run QA tests, on the off chance there might be some need for it.

</joke>

WTF (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#31705136)

you have a close tag but no open tag? Lrn2HTML, faggot.

Re:"Botnet?" (5, Funny)

benjamindees (441808) | more than 4 years ago | (#31705140)

They had to infect the computers with Office 2010.

In soviet Russia... (0)

Anonymous Coward | more than 4 years ago | (#31705084)

...Office' bugs made you fuzzy.

Speaks to the complexity (0)

Anonymous Coward | more than 4 years ago | (#31705112)

I don't mean to sound for or against any particular vendor's products, but I think finding that number of bugs speaks to the complexity of the software. A full Office suite has so many functions that I bet the average user isn't even aware of 10% of them. And the most avid of users still discover new features fairly regularly.

Re:Speaks to the complexity (4, Insightful)

zippthorne (748122) | more than 4 years ago | (#31705236)

Your point being? In 10 years since I started using it, I still don't know all the Vi commands and Emacs is so daunting I never even attempted it.

Re:Speaks to the complexity (1)

AHuxley (892839) | more than 4 years ago | (#31705238)

http://searchsecurity.techtarget.com/news/article/0,289142,sid14_gci1506909,00.html [techtarget.com]
Nothing really new, you just want your OS to be 'Unix' like when one app or new networked lifestyle cloud is compromised.
You really hope your fav 'application' does not open up your OS and start pumping your personal data out.
Apple seems itoy distracted, Windows seems Win7 PR happy.

Re:Speaks to the complexity (2, Interesting)

140Mandak262Jamuna (970587) | more than 4 years ago | (#31706358)

So why don't you do something instead of constantly griping? Find some open source project that comes close to what you want and contribute to it. Even if you are not a developer, work on documentation, testing, bug reporting or something.

1800 down, 10,000,000 to go (1, Troll)

flyingfsck (986395) | more than 4 years ago | (#31705222)

The problem is that they write such crappy code to begin with. There really is no good reason for that. I use Word 2007 at work and it is very buggy. If I had my way, I would not use it, even OOo is better.

Re:1800 down, 10,000,000 to go (2, Funny)

swilver (617741) | more than 4 years ago | (#31705292)

The same as I thought. Tip, meet iceberg.

Re:1800 down, 10,000,000 to go (1)

GeckoAddict (1154537) | more than 4 years ago | (#31705344)

The large Microsoft products seem to go through the same pattern. First version with a very large change kind of sucks but it works well enough to release and get bug reports on, and the second version is a polished, faster, more robust version of the same. See: Windows ME->XP, Vista->Win7, and now Office2007->Office 2010.

Re:1800 down, 10,000,000 to go (1)

RebelWebmaster (628941) | more than 4 years ago | (#31706140)

Windows ME was a descendant of Win95/Win98. Windows XP was a descendant of NT4/Win2K.

Re:1800 down, 10,000,000 to go (1)

jbengt (874751) | more than 4 years ago | (#31706528)

Your ME example is wrong; ME actually bucks the trend by being a worse version of a previous product.
ME was based on '95/'98 and XP was not based on ME.

Re:1800 down, 10,000,000 to go (0)

Anonymous Coward | more than 4 years ago | (#31705500)

Exactly. And how many new bugs were introduced with these fixes? Considering how poor the pre-patch code was, I don't believe the new patches will be written any more carefully.

Seriously, 1800 bugs just on the data parsing section? That's some shit coding right there. I mean it's just data parsing you morons.

Re:1800 down, 10,000,000 to go (1)

Blakey Rat (99501) | more than 4 years ago | (#31708130)

I use Word 2007 at work and it is very buggy.

Undoubtedly true.

If I had my way, I would not use it, even OOo is better.

"Better" in what sense? It's certainly not less buggy... hell, some components of OpenOffice.org (ok, I'll name it: Impress) seem to have never been actually tested.

Re:1800 down, 10,000,000 to go (1)

TheLink (130905) | more than 4 years ago | (#31709120)

> I use Word 2007 at work and it is very buggy. If I had my way, I would not use it, even OOo is better.

I agree Word 2007 is buggy. But OOo is NOT better. It's far buggier.

FWIW, it's not so much the bugs in Word 2007 that annoy me than the way it does formatting and selection of certain stuff. Yes I know you can customize the behaviour but I still find I have to "battle" with it a bit too often.

OpenOffice on the other hand has rather blatant bugs like these:

Launch openoffice writer.
Type three lines of: "This is a test testing 1 2 3"

Select one of the lines.

Press ctrl-f.

Enter test in the "search for" field
Enter foo in the "replace with" field
Click on More Options.
Click on current selection only.
Click on replace.

Bug: openoffice replaces the ENTIRE selection with "foo", instead of the first found term.

It is not possible to search and replace terms within a selection. You either have the entire selection replaced (if you click replace), or openoffice forgets your initial selection (if you click find).

This bug has been around for ages: http://qa.openoffice.org/issues/show_bug.cgi?id=15501 [openoffice.org]

I can't be bothered to see if it's still present in 3.2.

Just go the the openoffice bug site and look at the open bugs there and you'll see how behind they are. But hey it's free...

Another April Fool? (0, Troll)

epo001 (558061) | more than 4 years ago | (#31705234)

I mean Microsoft proactively looking for bugs? It's a bit far-fetched.

Come on it's April 2nd here, stop it!

Re:Another April Fool? (1)

owlstead (636356) | more than 4 years ago | (#31707368)

Any other time you might have been modded funny for a lame joke like that, but - as you - mention, it is 2nd of April, we're still trying to recover from yesterday. No lame jokes today, please.

..they were all sitting in front of the screen.... (1)

uweg (638726) | more than 4 years ago | (#31705242)

SCNR....

Software firm test his software? (1)

Tei (520358) | more than 4 years ago | (#31705258)

*pat in the back* good one Microsoft, now you test your software. What about now to change and respect standards, PLEASE.

Re:Software firm test his software? (1, Troll)

kronosopher (1531873) | more than 4 years ago | (#31705286)

I'd like to ask them to stop killing people [timesonline.co.uk] first, much less respecting standards.

Re:Software firm test his software? (1)

maxume (22995) | more than 4 years ago | (#31705338)

Aren't you a piece of work.

Ah, reminds me of the old days (1)

smchris (464899) | more than 4 years ago | (#31705290)

Of Imprise Delphi 4 and Corel WordPerfect 9.

Just kidding. Seems like a good initiative on Microsoft's part.

This is pretty standard in Haskell (1)

Z8 (1602647) | more than 4 years ago | (#31705444)

QuickCheck [chalmers.se]

But for some reason random data testing is less popular for the other languages I'm familiar with.

Did they fix... (1)

RemoWilliams84 (1348761) | more than 4 years ago | (#31705492)

Did they fix that bug where the useful menus get replaced by that horrible ribbon thing?

I know there are downloads to revert to the menus, but can't do that at work.

Re:Did they fix... (1)

BitZtream (692029) | more than 4 years ago | (#31706190)

Yes, they called it the ALT key, and it was fixed in 2007 too.

wow imagine that (1)

hesaigo999ca (786966) | more than 4 years ago | (#31705532)

Coming from a programmer's point of view, this is like saying a nurse used an alcohol swab before plunging a needle into your arm, to avoid you contracting anything....i think in today's world this is all pretty standard even for the smaller budget companies, most are unit test driven and have intensive test environments to stress test their apps. I have seen this even in a company as small as 3 programmers. Do they want a medal, seriously....i am thinking of saying something....maybe someone would care to finish my thought
I have blasted them enough already....?

Re:wow imagine that (1)

natehoy (1608657) | more than 4 years ago | (#31705736)

Programming 101: TEST YOUR GODDAMNED INPUTS!

Programming 102: If you missed the lesson in Programming 101 where you have to test your inputs, you fail and have to repeat Programming 101. Some people never get out of Programming 102. I've worked with a few of them. "It ain't that pretty at all".

This is clumsy after-the-fact testing at best, just throwing random garbage at the program and hoping to hit a condition. Having said that, I do want to applaud Microsoft for at least, finally, taking some steps toward input control. After so many Patch Tuesdays where "stack overflow", "buffer underrun", and other things caused by nothing more and nothing less than careless programming, it's good to see that they are at least going back with a little more rigor and testing for relatively obvious stuff before their code blows up in a customer's face.

Oblig. XKCD: http://xkcd.com/327/ [xkcd.com]

Re:wow imagine that (1)

Jer (18391) | more than 4 years ago | (#31706418)

Computer Science 395 (Software Engineering): Remember how back in Programming 101 we told you how to perform testing on your code? Turns out we grossly oversimplified our discussion of how to go about doing that for pedagogical reasons. While it's nearly trivial to perform the test/debug cycle on code that you wrote for a class project that does one well-defined thing and will only ever be seen by you and your grader/professor, the scope of testing and debugging transforms radically when you attempt to scale this up to more general applications. Now, let's talk a bit about how real-world testing is done when you're debugging a giant project, with a substantial user base, spread over dozens of programmers, with code that contains elements that originated over a decade ago, some from programmers who left the company long before you finished Programming 101...

Re:wow imagine that (2, Insightful)

natehoy (1608657) | more than 4 years ago | (#31707100)

Yes, I've taken that class.

I'm not talking about testing, I'm talking about design. If you expect a URL in a field and someone puts executable code in there, you should not be executing the code - you should be rejecting the URL. Data of that nature should not be put in a memory area where an instruction can be sent to run it.

Stack overflows, buffer underruns, and things of that nature are not things that should be caught in testing. They are things that should be prevented in the first place. If your code can't write data from strangers in places it can execute it, you can't be caught with your pants around your ankles when someone sends you executable code in a text field.

I'm not saying this testing is a bad thing, it's great, and necessary, and wonderful, and all that! But I sincerely hope Microsoft learned the lesson and Office 2012 or whatever the next version is will at least get some protected mode lovin' so they can separate data space from execution space and stop crossing the streams.

Maybe then Patch Tuesday will stop being so darned exciting.

Re:wow imagine that (1)

hesaigo999ca (786966) | more than 4 years ago | (#31707966)

Yet so many people take this class and then some, to learn UML, Unit testing,
variants of stress testing, even how to stand their ground against unwanted elements in the environment, but one thing will never change, if the boss says to do it his way, you can either finally do it his way, or hit the highway, and usually this is were the problem lies.

I have come across a few times in my life that the necessary steps were avoided on purpose and left many with critical bugs in their apps, only because of some higher up decision being taken to cut costs and save resources, to only have to spend more in the end.

I have heard many times M$ stood for develop and ship now, debug and patch later...however never having worked there I could never confirm such blatant disregard for proper protocol.

Re:wow imagine that (1)

BitZtream (692029) | more than 4 years ago | (#31706282)

this is like saying a nurse used an alcohol swab before plunging a needle into your arm, to avoid you contracting anything....

You clearly don't understand what fuzzy testing is.

Fuzzy testing and unit testing are only related in that they are tests.

Unit tests are well defined tests for known conditions.

Fuzzy tests, if done perfectly, would appear to be random and thrown entirely random data at the applicaiton in order to increase the chances of finding out what happens when the app gets something the developers never expected.

I don't know of anyone who does regular fuzzy testing. Everyone that matters does unit testing.

Going back to your nurse/needle analogy, a corrected version would be something like the nurse putting a needle of random length into a vial of a random medicine he/she pulled off the shelf into a random location on your body and injecting a random amount of the drug ... just to see what happens and figure out if its safe to do that in the future.

I won't finish your thought as its pretty clear you don't have them all that often, but before you start blasting someone it might actually help if you had at least some idea about what you were discussing, you clearly do not.

No one fuzzy tests. Okay, so no one is obviously false, but the number of people who do fuzzy testing is unimaginably low. Go find me an OSS package that does fuzzy testing as part of their automated build process. IF you find one, $5 says its a tool for doing testing and not some other type of app.

Re:wow imagine that (0)

Anonymous Coward | more than 4 years ago | (#31706458)

Mozilla regularly fuzzes its JavaScript engine: https://bugzilla.mozilla.org/show_bug.cgi?id=jsfunfuzz [mozilla.org]

Re:wow imagine that (1)

hesaigo999ca (786966) | more than 4 years ago | (#31707988)

Funny i read the word "buffer overflow" and to my knowledge this is
considered part of the unit testing and not the fuzzy.
As for many not doing fuzzy testing, well i never considered the 2 seperate
issues in my unit testing, so you will have to pardon my lack of
empathy for the situation

Re:wow imagine that (2, Interesting)

Blakey Rat (99501) | more than 4 years ago | (#31708216)

I don't know of anyone who does regular fuzzy testing. Everyone that matters does unit testing.

Just FYI, Microsoft does fuzz testing in all areas of business, not just Office. The "news" here is really that the Office fuzz testing is done with a cluster of the developers' own computers. (Although it's definitely a good story to get out to all the shitty software houses out there that don't already do fuzz testing.)

When I worked in Xbox game testing back when the Xbox 360 was shiny and new, we had a large pile of Xbox 360s that did nothing but fuzz-testing of new titles by feeding them random controller input.

that doesn't mean we found 1,800 security issues (3, Insightful)

Geminii (954348) | more than 4 years ago | (#31705928)

it's important to note that that doesn't mean we found 1,800 security issues.

"...we have absolutely no idea where THOSE are."

No surprise, with that "format"! (3, Insightful)

Hurricane78 (562437) | more than 4 years ago | (#31706576)

Have you even seen the “specification” that MS tried to make a standard. It’s a horribly convoluted mess, that can only be described as an upside-down pyramid of always patching new stuff onto the old framework, while never doing a needed complete re-design. Like Windows ME.

Hey Microsoft! If there are more bugs than features in your file format, maybe you should do a re-design, hm? ;)

Dumbest possible way to not find errors (2, Insightful)

Ancient_Hacker (751168) | more than 4 years ago | (#31706712)

Remember the very obvious maxim of Dykstra: testing can only tell you there ARE errors, it can't tell you there AREN'T errors.

Randomly poking at data only find you the very dumbest errors. It takes some real thinking and mulling to realize, hey, if a xml field crosses this buffer boundary, and the last 4-byte Unicode code was cached, it's going to get bashed by the next 3-byte escape code. Or 255 bytes of code-page Yen symbol (255) followed by a 254 will lead to sign-extension and access to an address in the kernel trampoline DLL. Those kind of combinatorial errors are not going to be discovered by random poking at the data.

So they're going to (and have) given everybody a false sense of security, when the basic method can do nothing of the sort. it can only fin errors of the most trivial sort. It can't find errors that thousands of unemployed Russian hackers can dream up of testing for, and it can only FIND errors, not tell you there aren't an unlimited number of remaining errors.

Re:Dumbest possible way to not find errors (1)

owlstead (636356) | more than 4 years ago | (#31707402)

Yeah, well, they find bugs and fix them, regardless of all the other testing they perform. If they are not deprecating other tests that (may not be fully covered) then yes, I can see a problem here. And don't forget, even the dumbest bug can become a vulnerability.

Re:Dumbest possible way to not find errors (0)

Anonymous Coward | more than 4 years ago | (#31707800)

Fuzzing is not a dumb method of finding bugs at all, it is an EXCELLENT method. Fuzzing when combined with unit testing, code reviews and full test sweeps as Ms are doing here is actually pretty damn awesome. Fuzzing helps devs and testers find bugs and security holes that their standard testing just didn't think of, it also helps improve the quality of defined tests as each time a fuzz breaks the product you can look at how and why and then implement test cases around those. anyone that says fuzzing is dumb obviously has very low level of security/dev knowledge, every extra tool that finds more bugs is a good thing, just becaue your little mind can't grasp that doesn't make it dumb.

It's not a botnet. (2, Insightful)

NotBornYesterday (1093817) | more than 4 years ago | (#31706974)

It's distributed computing.

Wait, I suppose it could be a botnet, if MS's IT department distributed the required software by exploiting security holes in the victim OS instead of just using admin rights to install the new app. Come to think of it, that might be easier ... [me scurries off to develop new easy-to-use set of malware-based admin tools].

There must be some funny counting here (1)

iceco2 (703132) | more than 4 years ago | (#31707250)

Even though some of us would easily believe MS office has 1800 security issues that need fixing.(and in my opinion every crash due to malformed input is a security issue)
I find it hard to believe they found 1800 of these by generating random data, what is far more likely is that they recorded 1800(or more) crash events
and after fixing two or three programming errors(problematic hidden assumptions about the input) 1800 of them were not reproduced.
This hardly counts as solving 1800 bugs.
The technique itself is very problematic since you need to generate random data that passes the tests your software has in place but still causes an unexpected error
due to something you forgot to check. just feeding random files and trying to parse them won't get you very far as practically all of them will be rejected, so have to make your garbage generator slightly more sophisticated, but without feeding in the same misconceptions that caused you to write buggy code in the first place.

  Me

Fuzzing is not needed if you have the source code. (0)

Anonymous Coward | more than 4 years ago | (#31707846)

You can write unit tests and then look at the code coverage of the tests until you have added tests that walk every code path. You can make sure that each code path is hit with garbage data. You can do this in just a few weeks of time. Certainly the fuzzing tests are good too, but unless the fuzzing findings are being added to the unit tests as they are found, then it is just wasted effort.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?