EFnet Paralyzed By Vulnerability 156
An anonymous reader writes "EFnet member Fionn 'Fudge' Kelleher reported several vulnerabilities in the IRC daemons charybdis, ircd-ratbox, and other derivative IRCds. The vulnerability was subsequently used to bring down large portions of the EFnet IRC network."
By crafting a particular message, you can cause the IRC daemon to call strlen(NULL) and game over, core dumped.
Sigh... (Score:5, Funny)
1998 called and want their attack vector back.
Re:Sigh... (Score:5, Funny)
Re: (Score:3)
A 1998 attack vector for a 1992 network.
Re: (Score:2)
string s;
if (s.Length....
blammo - you should have checked if s was null first... Oh, and use that fancy try/catch stuff, all the cool kids do!
Objective-C nil (Score:2)
blammo - you should have checked if s was null first
Unless you're using Objective-C, where the "nil" is a special object that implements all messages (that is, methods) as a no-op that returns a nil.
Re: (Score:2)
Ugh, this really sounds like the way SAS responds when you try to access a field that doesn't exist. It will just create it and initialize it to missing value.
I'm sure the objc solution is creating a _lot_ more bugs than it "fixes".
Re: (Score:2)
Ugh, seriously? Nothing like forced non-failure of software instead of a failure that could prevent more serious consequences...
Re: (Score:2)
it's better to react to an error than try to ignore it. Though I think I'd prefer a log entry to a core dump. I prefer to write bulletproof code, and I frequently benefit from it. Just this morning in fact I got a warning email that a directory create failed on a script. It wasn't a shell crash, it was an authentic "assertion failed" routine of mine getting called to fire off an email, when an rm -d returned nonzero.
It was something that "should never fail". ("so why bother testing the return code?") Un
Re: (Score:2)
blammo - you should have checked if s was null first... Oh, and use that fancy try/catch stuff, all the cool kids do!
The solution is *obvious* (duh), but the problem is not. You can't be putting an exception handler around every function call.
What most likely happened, is by the time the string got to the strlen function, it was either assumed to have been security checked and data validated, or, the set of validations run was not complete.
Shit happens.
Re: (Score:2)
The difference is, you can't catch a segfault in C++.
Re: (Score:2)
Yeah, he screwed up his example, i'm pretty sure it would compile if he set s to null explicitly.
Despite his mistake though his point stands, the problem of you (or a library you use) using null to represent "not known", "not present", "not found" or whatever and then passing the variable to a routine that assumes it will not be null impacts many languages that are far more "modern" than C.
Re: (Score:3)
You're laughing but I had to tell the developers of a small indie MMO, with-in the last year, that their ircd was vulnerable to remote exploits and it was running on the same thing that hosted the game itself, true story.
C strings strike again! (Score:3, Insightful)
Re: (Score:2, Insightful)
Re: (Score:2)
Yes too much management needed in C.
Even if you had a *ptr holding the data type. You have to check.
Pitty intel didnt implement string functions in the CPU.
But damn, the libc guys should check input pointers themselves.
Re:C strings strike again! (Score:4, Informative)
Pitty intel didnt implement string functions in the CPU.
They did. [pku.edu.cn] Welcome to decades ago.
Re: (Score:2)
Re: (Score:1)
Re: (Score:3)
And do what? Return a clearly wrong value so the caller can blindly continue? Burn some cpu on every call so that a broken caller can run a little longer and mysteriously crash later on or maybe just produce garbage output?
That said, undefined behavior is undefined, if I was writing a libc it'd have to return 97 just for the heck of it.
Re: (Score:2)
Re: (Score:3)
0 would be terrible, since it isn't a zero length string and that's lying to the caller. Crashing is a better option, at least you stop the damage there and then rather than the program generating garbage output. An exception is better still, but C doesn't have those - segfaulting sort of works as one if you squint.
-1 isn't an option because the return value is a size_t which is unsigned.
Sure it's far from an ideal setup, but C is language built on a philosophy that the compiler and libc should be relativel
Re: (Score:2)
And strcmp compares strings. NULL does not point to a string, therefore you shouldn't expect string functions to work.
C expects the programmer to know what he's doing, that's all.
Re: (Score:3, Informative)
An uncaugh NullPointerException on a call to aString.length() in java would have the same effect and kill the running Thread, the program if it is the main Thread.
http://stackoverflow.com/questions/5796103/strlen-not-checking-for-null [stackoverflow.com]
Re: (Score:1, Flamebait)
Hmm
Option 1 : 50 dozen try/catches every where making java slower, and coding more manual.
Option 2 : If length() returned zero even on a null object, it still is sort of correct, the string is zero sized.
Coding should be made easier not bloated.
Should there be 100000000 instances of checking for pointer before calling strlen(), or should strlen() it self check for ptr?
This is where libc sucks, so just make your own strlen() from libc, as a static strlen(), which behaves nicer. A length of a 'NULL' obj
Re: (Score:3)
Option 2 : If length() returned zero even on a null object, it still is sort of correct, the string is zero sized.
null isn't an object in Java, which is where the problem is coming from.
Re: (Score:2)
Re: (Score:1)
So, you have a pointer to string S, and you want *S.length() to check whether S is null? Do you propose that all functions on null pointers return 0? If so, how would you determine if it's a null pointer, or if the answer really was 0?
If (s==null)
Re: (Score:3)
Re: (Score:2)
I don't know much about the Arduino, but my understanding is that it's basically a microcontroller with the I/O pins wired to something that it's easy to connect stuff to. As such, the CPU is probably on the same order of complexity as a 6502, so about as easy to program, although the tools have improved a lot. It also doesn't have any of the coprocessors (e.g. for sound and video) that a C64-era machine typically had. Note, however, that I was talking about fully understan
Re: (Score:3)
Re: (Score:2)
An uncaugh NullPointerException on a call to aString.length() in java would have the same effect and kill the running Thread, the program if it is the main Thread.
The JVM would continue running as long as there are running non-daemon threads. Whether or not it is the main thread is irrelevant.
You are correct, however, that calling .length() on an object of type String that is null will throw a NullPointerException; however, it may or may not kill the current thread. It depends on whether or not the exception is caught, which is often the case, especially in servers where robustness is important.
Re: (Score:2)
especially in servers where robustness is important.
Ditto for C/C++ applications "where robustness is important". Implementation looks different, but same in the essence.
Some memory debuggers and code analyzers (Insure for example) can even catch the error during compilation (IIRC so called "wild pointers" in the Insure speak).
Experience we made with HA software, null pointer problems are (often but only) nasty in C/C++ and are horrible in Java. Some stats. As new release went into tests/production, we found in total about 200 *crashes* due to null point
Re: (Score:2)
Experience we made with HA software, null pointer problems are (often but only) nasty in C/C++ and are horrible in Java. Some stats. As new release went into tests/production, we found in total about 200 *crashes* due to null pointers in C/C++ apps. In Java part (about half the size of the C/C++) there were about 20 null pointer exceptions - with one which was occurring only in the production. This sole exception cost about 6 month of work to localize - because unlike core dump, Java's stack trace (due to your "which is often the case") was already unwound and couldn't point to the location of the show-stopper problem. This particular case (and it's not a sole Java debacle) became famous, because CTO of the customer, during a meeting, exclaimed that he wished the software was written in C instead, so that developers can see from core dump where the problem is.
That's interesting. My experience has actually been the opposite. I spent a few decades programming in C, and about a decade ago switched to Java.
In my experience -- which is, of course, just anecdotal -- C presented more difficulties. Core dumps are useful, of course, but it meant the entire process -- including all the threads -- went down. For high reliability requirements, we typically used a watcher daemon to restart processes that crashed unexpectedly.
In Java, unless you go out of your way to catch Nu
Re: (Score:2)
Core dumps are useful, of course, but it meant the entire process -- including all the threads -- went down. For high reliability requirements, we typically used a watcher daemon to restart processes that crashed unexpectedly.
We allow to start several instances of the same process: if one goes down, others are still there.
However, I'm not recommending Java over C, or vice-versa, though. I consider the application domains where C and Java compete to be pretty small these days, so typically I consider the correct choice to be dictated by the application domain.
In all fairness, I'm not a specialist in Java. Probably your experience is more relevant than mine.
But I have seen already enough of spaghetti Java code to stop believing entirely the fairy tail of "better language will solve all problems."
In Java, unless you go out of your way to catch NullPointerExceptions and then purposefully eat the attached stack trace, you should know precisely where and why it happened.
Simple response: interface layer of libraries. It's pretty much given to find there some exception-munging code: after all we do not want to expose with exceptions the inna
Re: (Score:2)
But I have seen already enough of spaghetti Java code to stop believing entirely the fairy tail of "better language will solve all problems."
Easier/safer languages are a double edged sword. In the hands of a master, it makes the master that much more productive, because they're not wasting wetware cycles on details like "will concatenating this string overflow my destination buffer?". But, at the same time, it makes programming more accessible to people who really shouldn't be doing software development.
I like to think I'm a good software developer and that using Java makes me more productive, but I readily admit that your average C developer is
Re: (Score:2)
[...] In the hands of a master, it makes the master that much more productive, because they're not wasting wetware cycles on details like "will concatenating this string overflow my destination buffer?".
Bad example.
A good sign of a non-master is attempt at concatenating string. Think of it, even Java *cannot* concatenate strings. :)
Contemporary C master wouldn't even think about it. Far cry from StringBuilder, but all the functions for safe string handling are already there - one only has to use them. (Although some thought might go into defining the limits for the buffers.)
Simple response: interface layer of libraries. It's pretty much given to find there some exception-munging code: after all we do not want to expose with exceptions the innards of the library.
This used to be a more serious problem, but fortunately, these days you can wrap an exception in a new exception and not lose the underlying stack trace. It's the common idiom now. For example: catch (IOException e) { throw new MyLibraryException(e); }
A link/name for the idiom I can pass to our Java people?
Re: (Score:2)
A link/name for the idiom I can pass to our Java people?
I think I would just point them to the Java API documentation [oracle.com] and point out these constructors for java.lang.Exception:
Exception(String message, Throwable cause)
Constructs a new exception with the specified detail message and cause.
Exception(Throwable cause)
Constructs a new exception with the specified cause and a detail message of (cause==null ? null : cause.toString()) (which typically contains the class and detail message of cause).
These two additional constructors were added to Jav
Re: (Score:2)
It isn't a matter of checking every object.
Just the ones that come from outside.
That takes lazy^stupid to miss, not just lazy/stupid.
Re: (Score:2, Interesting)
This was a NULL-pointer exception, not a buffer issue. But I do agree that it makes more sense to invest in building IRCd software which is written in string-safe and pointer-safe languages. Mozilla's rust language [rust-lang.org], for example looks promising for use in IRCd. The main thing is that we need a language which provides scalable data structures, as servicing IRC messages involves many data lookups.
However, it is easier for most people hacking on IRCd to just pick up a 2.8-derivative.
Re: (Score:2)
The main thing is that we need a language which provides scalable data structures, as servicing IRC messages involves many data lookups.
Are you CTO per chance? VP of tech? You talk like one.
But if you want it to be addressed in a programming language.
Until language developers suck it up and start making languages that allow for opposite concepts in the same language, it would remain the same.
For example.
Most stupid errors in strict typed languages are in the code for some type weakness - because, duh, strictly typed languages provide zero standard facilities to implement weak typing when you need it. So you have to roll out your own
Theoretical way to get size of C string (Score:2, Interesting)
Using error handlers, & two pointers (this goes for ANY array, & strings are just arrays of characters/array of char):
---
1.) You send two pointers into/@ the array/string buffer allocated, as follows:
2.) 2nd "double-sized" one (positionally) is ALWAYS double the size (position) of the 1st.
3.) When the 2nd "double size of the first" FAILS (& the err handler catches it, ala try-catch/try-except type constructs)?
4.) The error handler passes back the size of the 1st "half-size pointer" location,
Re: (Score:2, Funny)
Can these solutions be used in a Hosts file?
Re: (Score:2)
That's not sporting at all. He actually posts something informative about something else and you troll him? OK, I'm known for teasing apk, but he does something positive and you try to pull him down? Not fair. Rachel. Not bloody Barbie or whatever, Rachel.
Re: (Score:2)
"Screw you, apk, and the horse you rode in on. If I ever see you post here again, I'll bomb you as AC from Tor, meaning I'll NEVER run out of posts because I can change endpoint."
You know damn well I posted this in anger because you were accusing me of doing it anyway, and of being "Barbie". You, and anyone else who can read this, will notice that I posted with my UID and when I do post I don't post AC. I have posted embarassing stuff about myself IRL to prove this incorrect but instead you used this agains
Re: (Score:2)
Bullshit, most of that. You're so wordy, and you've got it so wrong. Fortunately, I have an IRL to go back to that encourages me to contribute positive change around me (unlike this place and people like you).
Thanks for ruining my evening after I simply told someone it was bad form to drag you down to that level after you posted something positive.
Re: (Score:2)
This all started because I completely disagreed about the use of HOSTs as compared to Adblock or similar, and you started personally insulting someone I know. Now, you're doing the same to me.
Defend myself, I'm a stalker. Ignore you, people might think you were correct. What a dilemma. You're one evil, uncaring guy, apk. Security tools = positive change? Try www.timberrecycling.org for positive change. Or www.manchester.gov.uk/elections. Or call Greater Manchester Police and ask them about Rachel Wilson. If
Re: (Score:2)
From your link:
"It makes the function look complicated as it has multiple exit points (return statements)."
hmm... you mix up 2 things. It is more like you add complexity by introducing multiple exit points.
There is ways around it you know... Beginning your fucntion with a temporary variable set to NULL and returning that temporary variable in one exit point at the end of your function is one.
Re: (Score:2)
Re: (Score:1)
This is the problem you get when your strings don't know their allocated size like in that ghastly language Pascal.
I would have to say that is an outright falsehood and misunderstanding of the language.
This is the problem you get when the programmer makes a serious error, who is the one and ONLY one who should be keeping track of the size of strings.
The entire C language is based around the idea that the programmer knows best. That's why it doesn't do unneeded things for you a bunch of times for each call.
So of course when the programmer does something stupid like not keep track of the size of strings, and pass what th
Re: (Score:2)
Yea, adding unneeded checks to every operation because of lazy/shitty programmers is a much better idea.
Guess what, exploits happen in languages with managed arrays too!
Languages can't fix shitty programmers or mistakes, deal with it and stop blaming the language.
Re: (Score:2)
Yea, adding unneeded checks to every operation because of lazy/shitty programmers is a much better idea.
I think the point that you are missing is that those checks are needed because the majority of programmers are both lazy and shitty once they are dealing with a large codebase. Here we have an obscure input causing the problem because its not obvious (lazy, shitty) that a variable might be null.
Re: (Score:2)
This is the problem you get when your strings don't know their allocated size like in that ghastly language Pascal.
At least in the case of Turbo Pascal, there is an allocated size and an actual size (length) for string variables.
Re: (Score:2)
Pascal is still a great language to learn how to code. I will never knock it.
EFnet is already paralyzed (Score:2)
Re:EFnet is already paralyzed (Score:5, Interesting)
Re:EFnet is already paralyzed (Score:4, Informative)
There has been a lot of work in this area with a few projects now... Microsoft's IRCX, then IRCNEXT, IRCPLUS and now atheme.org's IRCv3 [ircv3.org]. IRCv3 is becoming the defacto standard at this point, supplanting the traditional IRC protocol, as almost all vendors that are noteworthy have adopted support for revision 3.1 of the protocol already.
Both Atheme and Anope can be interacted with via RPC from scripts allowing for web integrations. Also, there are immersive web clients which provide a lot of useful metadata to clients.
Re: (Score:3, Interesting)
"Anope can be interacted with via RPC"... Yes, Anope 1.9. That's the development release, though. As much as I'd like to share your idealized view of IRC becoming a better place (except for the fact that I definitely wouldn't want to create an IRCd from scratch these days -- linking protocol and especially these goddamn CAP negotiations are just as bad as doing SSL from scratch. Who had the idea of message intents anyway?), fact is that the majority of networks that use Anope (and there are, due to the fact
Re: (Score:2)
Well, I still like the distributed design of it so keep it please. See picture here:
http://en.wikipedia.org/wiki/Internet_Relay_Chat#Client_software [wikipedia.org]
Re:EFnet is already paralyzed (Score:5, Funny)
While you're at it, please redesign SMTP.
Re: (Score:3)
IRC needs a redesign from scratch...
Sounds like a great idea... I even have an idea of what we could call the redesign: Jabber / Conference Room.
Re: (Score:2)
MUC is extremely inefficient though. There is no multicast notion in XMPP, so MUC works around this by sending duplicate messages to each user directly.
Re: (Score:3)
Jabber + PSYC [about.psyc.eu]
Re: (Score:2)
PSYC suffers the same issue as MUC.
Re: (Score:2)
MUC is extremely inefficient though. There is no multicast notion in XMPP, so MUC works around this by sending duplicate messages to each user directly.
That's also pretty much how IRC works (though the message formatting is rather different). For real efficient multicast, you need to implement it at the network level so that messages are sent directly to the other listening parties by the hardware, but IP multicasting is complex and full of all sorts of "interesting" problems when you use it at larger scales than the LAN. Re-transmitting messages (even with updated IDs and other such info) is far simpler in practice. Really.
Re: (Score:3)
Yeah, but redesigning something that has been around forever is hard (see: IPv4). Way too much legacy stuff out there dependant on the messed up way IRC works.
That said, gradual introduction of saner behavior is possible (see: Atheme/IRCv3).
If you build something from scratch you'll just end up with a potentially technically improved (or just broken in different ways) but largely ignored solution (and for this, see: jabber conference!)
Re: (Score:2, Funny)
Google wave?
Re: (Score:1)
Can we shorten the amount of text to about 40 or 20 characters? Better artificial limits can improve the quality of our chat content. Besides, it can help us achieve web-scale!
Re: (Score:2)
Re: (Score:2)
EFnet does not provide NickServ or ChanServ, which is a common criticism of the network. Due to not providing centralized authentication or channel ownership services, the other aspects of the modern IRC protocol mostly do not apply.
Re: (Score:2)
Also when I checked there a year or so ago they still don't cloak the ip address on your hostmask like a lot of modern irc networks do now by default as well.
Re: (Score:2)
Full disclosure takes a hit (Score:1)
which projects are now going to close their doors to full disclosure? this was posted on the ircd's own bug reporting systems and was publicly visible. if it were not, and only the developers and higher level users (such as the nodes of efnet or freenode) should be able to see reports of this nature, this sort of attack may not have happened and the ircd's could have been silently patched without anyone knowing.
on the other hand, if you close your doors, you obviously have something that requires hiding, dr
Re: (Score:3)
you do not know what you are talking about. the actual bug was disclosed in the IRC channel for charybdis. there was no "bug report."
Re:Full disclosure takes a hit (Score:4, Interesting)
Disclosed how? by example?
Sounds like when I found that if you put %s in a CounterStrike player name, the client would crash when showing the scoreboard, I tried %s after I noticed "%dxxxxxxx%" became "32401xxxxxxx%".
I reported it to Valve as soon as I figured out what it was, but it was a fun few days until they fixed it, "Hey look at that guys name!", server empties as people hit tab to show the scoreboard. I was server admin anyway, but it still was fun.
Re: (Score:1)
You cannot "close your doors" to full disclosure, that doesn't make any sense. If someone wants to publicly announce a problem, they will always be able to do that. If someone wants to privately bring something to the attention of devs, they can always do that too. You cannot force people to use a disclosure method that they don't want to use.
Re: (Score:2)
I don't see any reason to hide from bugs like this, people make mistake, code suffers from bitrot, etc. It's sure as shit embarrassing when a mistake in your code ends up on /. though.
Re: (Score:2)
Because EFNet is more or less non-functional since it was publicly disclosed?
Because no thousands of people pretty much can't chat, because someone else like you was too short sided to understand that some actions have larger overall consequences?
Because its fucking stupid to tell the world how to break something until its been fixed and thats only left for fucking jackasses to do.
Re: (Score:2)
He did not disclose it, the person who disclosed the vulnerability did so in a public space for development discussion on charybdis. Once it was out in the open, we quickly jumped into action to start mitigating it across all fronts including EFnet. The ratbox developers were notified at least an hour before the exploit was unleashed, with a patch to deploy and everything, so really we did everything we could possibly do to mitigate the possibility of fallout.
Once people started running the exploit on EFn
Re: (Score:2)
1 hour? Give me a fucking break. Doing it in public is as good as launching the attack yourself.
The patch could have been discussed behind closed doors. If you wanted to get someones attention about the real threat, you could have popped a single server ONCE without making it public.
Do you have any idea what its like running a network like EFNet and coordinating upgrades across all servers? Do you think there are admins awake and ready to be your bitch and patch their servers on your command in all time
Re: (Score:3)
EFnet was fixed within an hour or two for the most part. More importantly the hub servers were not impacted which helped with getting things working sooner.
Indeed the patch could have been discussed behind closed doors, but some
Great! (Score:5, Funny)
Now I can finally get that nickname I have been wanting since 1999 !!
Re: (Score:2)
the cause was bitrot... (Score:3)
This is a good case of bitrot here, code that had made assumptions about what the other parts of the code were doing..and fail. Brown paper bag day for me on EFnet :P
Re: (Score:1)
Hey now, I'd call it closer to a flaming brown paper bag day, filled with dog crap.
Re: (Score:2)
So it was you who left that on my doorstep this morning!!! :D
Rename idea (Score:2)
They should change the name from EFnet to EFFYOUnet.
Assigned CVE-2012-6084 for this issue (Score:3)
Re: (Score:2)
The problem isn't performance as much as it is accessibility. Almost every UNIX system has a C/C++ toolchain installed, not so much with Lisp, Java or C#. Also, C and C++ are generally the lowest common denominator for contributors. Almost everyone knows a little bit about C, not so many people know about Lisp (which is a travesty in and of itself, but not my problem).
Re: (Score:3)
Re: (Score:1)
Throwing C++ into the same category as C is a bit retarded, is it not?
It's a lot retarded. But that never stopped anyone before.
Re: (Score:3)
What language would you advocate, exactly?
The *EXACT* same problem (abrupt program termination) would have plagued the software if had been written in either C# or Java unless exception-handling code was in place to prevent it. Not so coincidentally, the effort spent writing such exception handling code in C# or Java could just as easily be spent on a simple null pointer check before passing a value to strlen in C. Not to mention the fact that neither C# nor Java even existed when ircd was invented.
Re: (Score:2)
What language would you advocate, exactly?
Lisp
Re: (Score:2)
Interesting call... Lisp existed back then.
Granted, this particular crash would not have occurred had it been written in Lisp, but the failure on the part of the programmers to anticipate the kind of input that might produce a crash in C could just as likely to cause catastrophic failure in a program written in any language, even Lisp. Ultimately, the problem was not programming language choice - which is my whole point.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
The major difference between exceptions and null checks is the granularity. You can for example have a try/catch around the whole function parser that will kick in on any sort of failure and just say "whoops, something bad happened parsing this command" and it won't really matter what part inside that failed as long as you can recover, in that case probably by just ignoring that command and keep running. With null checks you must have one in every possible place you might call a null pointer. Particularly f
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Validate your fucking inputs, you moron.
That's the short version, the full version is
Validate all possible valid combinations of inputs are valid in all subroutines.
Re: (Score:2, Funny)
EFnet is some crappy IRC network people go to for help until they find out about freenode.