Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

CERT Releases Basic Fuzzing Framework

timothy posted more than 4 years ago | from the this-field-cannot-be-left-blank dept.

Security 51

infoLaw passes along this excerpt from Threatpost: "Carnegie Mellon University's Computer Emergency Response Team has released a new fuzzing framework to help identify and eliminate security vulnerabilities from software products. The Basic Fuzzing Framework (BFF) is described as a simplified version of automated dumb fuzzing. It includes a Linux virtual machine that has been optimized for fuzz testing and a set of scripts to implement a software test."

cancel ×

51 comments

Sorry! There are no comments related to the filter you selected.

Fuzzing is only useful, if only moderately so (1, Insightful)

Anonymous Coward | more than 4 years ago | (#32370834)

Anything that you write that uses a regex you should beat on with some fuzzing logic, since they can tend to increase in computational time non-linearly, and next thing you know you got a DOS on your hands.

TIP OF THE DAY for you FROM ME

Re:Fuzzing is only useful, if only moderately so (1, Interesting)

Anonymous Coward | more than 4 years ago | (#32370994)

This man speaks the truth. Just yesterday I had to deal with a Perl script whose execution time blew up once it had to process files larger than 1 KB in size. It'd work fine for 500-character files, but give it more than 1000 characters and the runtime would take over half an hour! (Yes, we had one user sit there and wait over 30 minutes for it to finish.)

In the end, it was a poorly-written regular expression that was to blame. It was easy enough to fix, and we've since ditched the Indian team that developed the script. I think it's excessive, but our manager has now ordered a review of ALL the regexps in all of our scripts.

Re:Fuzzing is only useful, if only moderately so (0)

h4rr4r (612664) | more than 4 years ago | (#32371048)

You paid for cheap code and you got it. What did the pointy hair expect?

Re:Fuzzing is only useful, if only moderately so (0)

Anonymous Coward | more than 4 years ago | (#32371128)

You assume that someone who is writing code (the GP) is simultaneously in a managerial position.

Hint: Programmers cleaning up cheap code often didn't ask for the task. It was a PHB.

Re:Fuzzing is only useful, if only moderately so (1)

h4rr4r (612664) | more than 4 years ago | (#32371152)

Pointy hair is the PHB, what do you think PHB stands for?

That is why I asked what the PHB expected, not what he thought the outcome would be.

Re:Fuzzing is only useful, if only moderately so (0)

Anonymous Coward | more than 4 years ago | (#32372314)

Then you should not have said "you" if you were talking to the PHB.

Re:Fuzzing is only useful, if only moderately so (0)

Anonymous Coward | more than 4 years ago | (#32373002)

Umm, I took that as the "group" "you" not the personal you.

It made sense to me and I'm drunk as fuck, whats your excuse?

Re:Fuzzing is only useful, if only moderately so (0)

Anonymous Coward | more than 4 years ago | (#32371428)

I claim BS.
Unless their perl code looked like this:

# // PATTEERN MATSH
system("java -Xmx1000m -Xms500m PatternMatchRoutineTakingTwoArguments.class SureName=$String MatchAgainst=\"Sri* OR sri*\" ThirdArgumentIsNotUsed");

Re:Fuzzing is only useful, if only moderately so (1)

ysth (1368415) | more than 4 years ago | (#32372388)

Nope, no BS. Compare perl -we'("a" x 10) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/' and perl -we'("a" x 100) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/'

Re:Fuzzing is only useful, if only moderately so (2, Informative)

mr_mischief (456295) | more than 4 years ago | (#32373166)

$ time /usr/local/bin/perl -we'("a" x 100) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/'
37.00user 0.01system 0:37.81elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+438minor)pagefaults 0swaps

$ time /usr/local/bin/perl -we'("a" x 10) =~ /(a*)(a*)(a*)(a*)(a*)(?i:b)/'
0.00user 0.00system 0:00.00elapsed 75%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+438minor)pagefaults 0swaps

For a purposely selected pathological case on a Pentium M 1.6 GHz laptop with little free RAM, I'd say that's not bad for a system that has specifically been chosen to support grouping, alternation, backreferences, conditional changes (case sensitivity, prematch, postmatch, etc) on only parts of the expression, greediness and nongreediness, lookahead, and lookbehind. Perl "regular expressions" are definitely not actually regular.

That's perl 5.12.0 BTW, which is much improved over older series (pre 5.10 anyway) of perl systems regarding regexes.

Note that if you're okay with intentionally trying and failing to get a case-sensitive rather than case-insensitive 'b' after your pathological quantifiers on the 'a' characters, then you have no such time problem.

$ time /usr/local/bin/perl -we'("a" x 100) =~ /(a*)(a*)(a*)(a*)(a*)(?:b)/'
0.00user 0.00system 0:00.00elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+428minor)pagefaults 0swaps

$ time /usr/local/bin/perl -we'("a" x 1000000) =~ /(a*)(a*)(a*)(a*)(a*)(?:b)/'
0.00user 0.00system 0:00.00elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+673minor)pagefaults 0swaps

$ time /usr/local/bin/perl -we'("a" x 1000000000000) =~ /(a*)(a*)(a*)(a*)(a*)(?:b)/'
0.00user 0.00system 0:00.00elapsed 80%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+426minor)pagefaults 0swaps

I'm sure the p5p would welcome a patch that delivers the promised matching semantics without performing so poorly on pathological cases.

Re:Fuzzing is only useful, if only moderately so (1)

ysth (1368415) | more than 4 years ago | (#32376268)

The point is that it doesn't scale linearly. 1000 is much worse, and 10000 much worse than that.

Yes, any particular case *could* be "fixed" in the regex engine, but there will always be one more possible pathological regex.

Re:Fuzzing is only useful, if only moderately so (1)

mr_mischief (456295) | more than 4 years ago | (#32387478)

Yeah, and checking specifically for a possible type of pathological regex in order to work around it would require slowing the engine for the common case. So I'd say the power of the engine to do so many things is a good trade for the easily avoided pathological cases.

There are only two situations in which such pathological cases are likely to be given to the regex engine anyway. One is when long regexes are built automatically by a program based on a set of rules and a particular data set, which should only trigger a pathological case rarely based on random chance. The other is when people set out to specify something pathological on purpose just to demonstrate the issue. The rest of the time, a programmer should be interested in using the regex engine to actually do something useful and there's really no use in any of the contrivances used for such torture tests.

Re:Fuzzing is only useful, if only moderately so (0)

Anonymous Coward | more than 4 years ago | (#32388160)

If there are *pathological* regular expressions, that means that the algorithm is poorly defined. A regular expression is *by definition* something that can be handled by a DFA (and your pathological case can be handled by a DFA), and so also *by definition* polynomial (I'm not absolutely sure, but I think that a DFA should always be expressible with a quadratic, or maybe it's a cubic, algorithm). That sure as hell doesn't look polynomial to me. It looks at least sub-exponential. That tells me that somebody doesn't know how to write a simple walk algorithm.

Re:Fuzzing is only useful, if only moderately so (1)

ysth (1368415) | more than 4 years ago | (#32394636)

Perl regexs are not "by definition" regular expressions, and are not handled by a DFA. By choice, not out of ignorance.

Re:Fuzzing is only useful, if only moderately so (1)

mr_mischief (456295) | more than 4 years ago | (#32395228)

Thanks for reading the thread... oh, wait, you didn't. I already said [slashdot.org] that Perl's extended "regexes" are not regular. Well, thanks for weighing in anyway.

Re:Fuzzing is only useful, if only moderately so (1)

ysth (1368415) | more than 4 years ago | (#32394658)

No, it actually happens in the wild (see the post ^ that had BS claimed on it) typically when parsing a string up into many substrings where the boundaries aren't specified in such a way that backtracking is prevented.

Re:Fuzzing is only useful, if only moderately so (1)

mr_mischief (456295) | more than 4 years ago | (#32395220)

Sure, it happens, but with what frequency?

Any artificial language you're parsing should be designed with this in mind, or at least a different tool could be clearly chosen for it.

If you're parsing any sort of natural language, well first you kind of deserve what you get. ;-) The variability of the language would seem to preclude this sort of massive backtracking in the common case, and the parser could use multiple separate regexes to help lower the chances of triggering the problem.

It still seems from my experience that the common case is of reasonable performance is much more common, although rare incidents might happen.

Re:Fuzzing is only useful, if only moderately so (1)

elfprince13 (1521333) | more than 4 years ago | (#32372648)

I'm sorry you're uneducated about Perl's notoriously poor regex implementation, that has unfortunately become the standard in most languages. Reference: http://swtch.com/~rsc/regexp/regexp1.html [swtch.com]

Re:Fuzzing is only useful, if only moderately so (1)

elfprince13 (1521333) | more than 4 years ago | (#32372662)

Note particularly the units on the vertical scale of those two graphs. The Perl algorithm is measured in seconds, the Thompson algorithm is measured in microseconds.

Re:Fuzzing is only useful, if only moderately so (1)

Menkhaf (627996) | more than 4 years ago | (#32373120)

Thanks for the link, it seems like a good read for a cloudy day.

Do you know if anything has changed since the article in 2007?

-menkhaf

Re:Fuzzing is only useful, if only moderately so (0)

Anonymous Coward | more than 4 years ago | (#32374280)

Perl development has stagnated pretty badly since about 1997. Sure, they get new releases out, but they only contain bug fixes.

Re:Fuzzing is only useful, if only moderately so (1)

elfprince13 (1521333) | more than 4 years ago | (#32385324)

As far as I know, the same poor implementation is still in use. I don't use Perl, but I do use a lot of languages which make use of PCRE which uses the same faulty algorithm.

Re:Fuzzing is only useful, if only moderately so (1)

daemonburrito (1026186) | more than 4 years ago | (#32373380)

Thanks, that was really interesting.

Cool (1)

gweihir (88907) | more than 4 years ago | (#32370854)

And urgently needed. So far the CMU/CERT software I had a look at was pretty good....

bleh (1, Informative)

Anonymous Coward | more than 4 years ago | (#32370980)

Sort of like this [sourceforge.net] ?

Re:bleh (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#32370988)

Yep except for this is written by actual professionals not some basement dwelling open sores turd.

Re:bleh (1, Interesting)

Anonymous Coward | more than 4 years ago | (#32371314)

I wouldn't be so quick to automatically label researchers as professionals. If there is one place I've seen worse code than OSS, it would be in academia.

Bizarrely, this is also where I've seen the most brilliant code.

Re:bleh (1, Funny)

Anonymous Coward | more than 4 years ago | (#32371458)

After writing that, I had to compare the two.

Fuzz's code resides in a massive 34KB C file, undocumented and unformatted, liberal copy-pasta and no re-use, no grasp of language, about as readable as an assembly dump. Basically one big hack.

BFF is a nicely written, concise, small, and extensively documented perl script.

Yep... OSS fails to impress once more.

Re:bleh (2, Insightful)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#32371598)

Dare I inquire as to the thought process behind the notion that the inferiority of an OSS program called "Fuzz" and the superiority of an debian-based VM, running a GPLed perl script automating a WTFPLv2-licenced fuzzer proves the unimpressiveness of OSS?

Re:bleh (0)

Anonymous Coward | more than 4 years ago | (#32371808)

It doesn't. But there isn't a clever way to distinguish between hobbiest OSS code and "professional" OSS code (by organization). Former represents OSS more than the latter does, so I (ambiguously) called BFF 'OSS'.

I guess I could have called it, community developed OSS. It usually stands that projects made by internal development team are of higher code quality.

Re:bleh (0)

Anonymous Coward | more than 4 years ago | (#32371640)

The Perl code is not that great, and here and there stuck in the late 90's.

Re:bleh (1)

ysth (1368415) | more than 4 years ago | (#32372410)

Pet peeve: use strict, but not warnings enabled. Yes, strict is really important; BUT WARNINGS ARE MUCH MORE SO.

Re:bleh (2, Insightful)

Daniel Dvorkin (106857) | more than 4 years ago | (#32371776)

If there is one place I've seen worse code than OSS, it would be in academia.

Bizarrely, this is also where I've seen the most brilliant code.

If you look closely, you'll find that the "brilliant code" is most often written by academics who have industry programming experience. Similarly, in industry, you will find that the best code is written by experienced programmers with rigorous academic backgrounds. In contrast, the academics who insist that computer science has nothing to do with programming, and the self-taught hackers who proudly proclaim their lack of all that fancy book-larnin', are two sides of the same worthless coin.

Some of that may be true (0)

Anonymous Coward | more than 4 years ago | (#32372302)

But for me it's simply about learning the conventions/idioms of whatever language/system you're programming in and taking some fucking pride in your work.

My code has actually become worse since I moved from academia to industry -- at least, I've been "forced" into a lot more hacks. Two reasons: Shortsightedness on the part of stakeholders/PHBs and the fact that you cannot afford to rewrite absurdly broken code written by your coworkers. Sad but true.

Re:bleh (1)

NewbieProgrammerMan (558327) | more than 4 years ago | (#32372284)

If there is one place I've seen worse code than OSS, it would be in academia.

You've only seen code from those two, then? Because there's no shortage of hard-core WTF code in private business and government.

Re:bleh (0)

Anonymous Coward | more than 4 years ago | (#32372518)

I can attest to this, I have seen code written by a major multi billion dollar DoD contractor better known for aircraft than software that didn't account for the fact that in javascript "8" and "9" my be valid numbers but "08" and "09" are not. Therefore it was impossible to submit forms with dates on the 8th or 9th of any month, or in August or September without turning off browser script support. This was in delivered production code.

Re:bleh (1)

Eskarel (565631) | more than 4 years ago | (#32372974)

In their defense, interpreting strings beginning with a zero as being in octal is fucking retarded, and that's the cause of that particular bug. It's even been deprecated in the spec.

axfuzz (5, Interesting)

shird (566377) | more than 4 years ago | (#32371160)

in their whitepaper they referenced my 'axfuzz' tool I wrote years ago and even used a modified version of it in their testing. Hope they didn't judge me on that code, it was a pile of crap that I kept hacking together until it finally worked, with no thought to proper software design.

Re:axfuzz (2, Insightful)

nextekcarl (1402899) | more than 4 years ago | (#32372098)

I think the fact they are using a modified form means they did judge you, and found it good enough to use as a start. That should count for something.

Re:axfuzz (3, Funny)

TubeSteak (669689) | more than 4 years ago | (#32372106)

Hope they didn't judge me on that code, it was a pile of crap that I kept hacking together until it finally worked, with no thought to proper software design.

That sounds like exactly the kind of code a fuzzer should be used upon.
Oh the recursion!

Re:axfuzz (1)

Menkhaf (627996) | more than 4 years ago | (#32373256)

Hi,

Do you know if the whitepaper available somewhere for free?

Another link for further information about BFF, is the CERT Vulnerability Analysis Blog, found at https://www.cert.org/blogs/vuls/2010/05/cert_basic_fuzzing_framework.html [cert.org]

-menkhaf

Re:axfuzz (1)

shird (566377) | more than 4 years ago | (#32374002)

I see Dranzer is an older ActiveX fuzzing tool, somewhat unrelated to the "BFF". I didn't really read the article properly. :P

Well, then "my hats off" to you man... apk (0)

Anonymous Coward | more than 4 years ago | (#32378850)

"in their whitepaper they referenced my 'axfuzz' tool I wrote years ago and even used a modified version of it in their testing. Hope they didn't judge me on that code, it was a pile of crap that I kept hacking together until it finally worked, with no thought to proper software design." - by shird (566377) on Thursday May 27, @09:48PM (#32371160) Homepage

Per my subject-line above: My "hat's off" to you man, in terms of "good job on your end". It's always nice (and, a treat) to see that someone from this website (or rather, someone that frequents it) has actually produced something of use that's for "the common good", or used by others even (right up there with being imitated, as far as "flattery")!

(Change, REAL CHANGE? Starts with YOU, & you exemplify this via your works here!)

APK

P.S.=> The funniest part is, is this (and, I am sure you are aware of it, based on your works): Anyone AND EVERYONE has this type of potential, IF they apply themselves!

You seem to "berate yourself" for your code... don't!

(& yes, I know how that is, looking back on code you wrote YEARS (if not decades) before & saying to yourself "WTF? I did THAT?? I am amazed it works @ all!"... but, this is only your own personal growth/progress showing through, because you're now aware of how you could have done it better (a great sign really - imo @ least? It means you're growing & improving!)).

You KNOW the deal, i.e.-> That You'll fail a few times, @ first, but then? THEN, you get it working @ least. For starters... then you can improve on it, IF you wish, & that, as I am also sure you know, takes time & observations + inference (been there recently on an app I wrote to remove duplicates from a HOSTS file I build. I wrote the 1st version in 2002, it utterly sucked. I later rewrote it (I used Borland Delphi because it's THE fastest in strings + math work, doubling MSVC++ even in Visual Basic Programmer's Journal 1997 issue "Inside the VB5 Compiler Engine")).

Near the end, & got it down to 1 hr. processing time over 1 million records, down from 2 hrs. processing/parsing time... Anyhow/anyhow: I "hit the wall", & couldn't get the code any faster (inline asm & all, + compiler optimizations & by hand ones via profiling and identifying the slowest area) - this "perturbed me".

HOWEVER, at the very end, it took me looking @ THE DATA ITSELF, & how I was parsing + processing it (1 large file) & it made a HUGE diff. in how fast it processed removal of HOSTS file duplicates (yes, a SELECT DISTINCT type query in an indexed DB engine can do it faster, but, I don't even use indexing in mine (don't feel like implementing a b-tree either) & it's really a tool for those of us that don't "do SQL" (I do, but meaning others here), or possess a DB engine to do so (though mySQL server is free, not everyone's mastered SQL).

So, how'd I improve it? AGAIN - Not in the code itself, but in the data!

Heck - I ended up "busting up" the 1++ million records into 8 equal parts, all sorted, & now I have the thing down to 24 minutes processing time, vs. its original mark of 2++ hrs.!

(Sure, I could have altered the algorithm in code I used, seeing it favors "smaller/smallish" datasets over larger ones for deduplication of entries, but this worked gangbusters, so I went with it!).

SO, I guess you could say "I KNOW THE FEELING" (on what I quote from you)...

In the end? Great job, you've done something SO WELL, others "bit off your style" man... kudos! This world needs more "creative types" like you out there, helping improve it (that's MY "bottom-line" here), & once more? Always cool to see another "creative type" here on this website (you're a rarity imo & observations here @ least, the past 7++ yrs. or so visiting here)...apk

hmmm... (2, Funny)

thatskinnyguy (1129515) | more than 4 years ago | (#32371340)

The worst case scenario is talking about worse case scenarios thinking about worse case scenarios and letting them possess you.

Linky? (3, Informative)

Anonymous Coward | more than 4 years ago | (#32371568)

Oh FFS, you couldn't even link [cert.org] to the damn framework?

BFF? (2, Insightful)

Fnord666 (889225) | more than 4 years ago | (#32371618)

BFF? What an unfortunate choice of acronyms.

Re:BFF? (3, Funny)

Daniel Dvorkin (106857) | more than 4 years ago | (#32371730)

Because it's, like, the security researcher's BFF OMG ponies!

Good!! (1)

Panaflex (13191) | more than 4 years ago | (#32372146)

I propose that every website which handles private data (credit, ssn, health, etc) should be integrating these kinds of tools into normal test procedures, both in development and on production mirrored sites.

Hear hear!

Which license? (0)

Anonymous Coward | more than 4 years ago | (#32372978)

Bitches! If I ever see somebody IRL who tells me he's a slashdot editor, god I swear I will punch him.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>