Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

One Broken Router Takes Out Half the Internet?

kdawson posted more than 5 years ago | from the brain-gone-punky dept.

The Internet 412

Silent Stephus writes "I work for a smallish hosting provider, and this morning we experienced a networking event with one of our upstreams. What is interesting about this, is it's being caused by a mis-configured router in Europe — and it appears to be affecting a significant portion of the transit providers across the Internet. In other words, a single mis-configured router is apparently able to cause a DOS for a huge chunk of the Net. And people don't believe me when I tell them all this new-fangled technology is held together by duct-tape and baling wire!"

cancel ×

412 comments

Sorry! There are no comments related to the filter you selected.

Intelligence Op (2)

Philip K Dickhead (906971) | more than 5 years ago | (#26879383)

Looking to make the big blackout, when needed.

See Also: Severed Mediterranean Cables.

BGP (5, Informative)

winkydink (650484) | more than 5 years ago | (#26879451)

The internet's dirty little secret. It's amazing it works at all.

"One Bad Apple" (1)

Philip K Dickhead (906971) | more than 5 years ago | (#26879561)

...Don't Spoil the Whole Bunch, Girl!

Re:Intelligence Op (5, Funny)

agm (467017) | more than 5 years ago | (#26879943)

They need to replace it with a network that is designed to survive a nuclear attack. Oh wait, hang on....

Re:Intelligence Op (-1, Offtopic)

hardwarefreak (899370) | more than 5 years ago | (#26880241)

They need to replace it with a network that is designed to survive a nuclear attack. Oh wait, hang on....

Wish I had mod points today. Parent should already be SCORE:5 Funny. Apparently not enough Slashdotters know the history/evolution of the net.

Few stories back... (5, Funny)

Anonymous Coward | more than 5 years ago | (#26879387)

A couple of Nuclear Subs probably cut an underwater cable...

Re:Few stories back... (5, Funny)

morgan_greywolf (835522) | more than 5 years ago | (#26879433)

Nuclear Sub? Is that a new sandwich from Subway?

"The all new Subway Nuclear Sub: It glows in the dark! Get a lotta green for a little green! Now only $5.99 for a 12-inch! Subway: Eat Fresh!"

I lost a router (5, Funny)

Philip K Dickhead (906971) | more than 5 years ago | (#26879507)

And took out THE _WHOLE_ INTERNET!!!!!

It's true! Ask my wife!

Re:Few stories back... (2, Funny)

Kell Bengal (711123) | more than 5 years ago | (#26879635)

A nuclear sub would be one with extra tabasco sauce.

Re:Few stories back... (4, Funny)

mschuyler (197441) | more than 5 years ago | (#26880211)

That is actually correct. The sub shop in Bremerton (West Coast port for Trident Ballistic Missile Submarines, SSBN-726, etc.) sells the Trident Nuclear Submarine Sandwich with an extra serving of horseradish somewhere in the middle of it. It'll light your hair on fire, or, in my case, turn my scalp red.

Re:Few stories back... (1)

ottothecow (600101) | more than 5 years ago | (#26879731)

Don't you know that all foot long subs are only $5 right now?

get with the program

Re:Few stories back... (0)

Anonymous Coward | more than 5 years ago | (#26880217)

Nuclear Sub? Is that a new sandwich from Subway?

Goes well with four-alarm buffalo wings on the side.

Re:Few stories back... (1)

Klowner (145731) | more than 5 years ago | (#26880219)

Five.. Five dollar.. Five dollar nuke suuuuub

Hey I know how you could fix it: (0)

kcbanner (929309) | more than 5 years ago | (#26879405)

A reverse ssh tunnel [caseybanner.ca] !

Can you think of some reasons (0)

Anonymous Coward | more than 5 years ago | (#26879413)

why people don't believe you? I would like to hear your side of the story first.

Half the internet? Are you serious? (5, Informative)

Anonymous Coward | more than 5 years ago | (#26879423)

A router takes out 'half the internet' and I learn this from Slashdot?

Seriously, what is/was the impact? I work for a large e-commerce provider and haven't seen a thing that would indicate a problem today.

Ditto the A.C. (5, Informative)

khasim (1285) | more than 5 years ago | (#26879527)

It must have been the "half the Internet" that I don't use. Which would be an interesting half because many of the sites I visit regularly are based in Europe.

From the thread, it looks like AS 47868 was the route being lost.

http://en.wikipedia.org/wiki/Autonomous_System_Number [wikipedia.org]

Re:Ditto the A.C. (4, Funny)

roc97007 (608802) | more than 5 years ago | (#26879841)

> It must have been the "half the Internet" that I don't use.

The non-pr0n half.

Re:Ditto the A.C. (5, Funny)

besalope (1186101) | more than 5 years ago | (#26880045)

> It must have been the "half the Internet" that I don't use.

The non-pr0n half.

Such a place exists? 0.o

Re:Ditto the A.C. (0)

Anonymous Coward | more than 5 years ago | (#26880067)

non-gay pr0n

The half that doesn't know how to filter their BGP (1)

wsanders (114993) | more than 5 years ago | (#26880255)

Everyone knows that BGP stand for "Border Gateway Politics".

Or, what you give up in reliability you gain back in increased complexity.

Sorry, those are the only BGP jokes I know.

Re:Ditto the A.C. (2, Informative)

0xygen (595606) | more than 5 years ago | (#26879927)

I think AS47868 was causing the routes to be lost.
It was making mass BGP announcements about really long incorrect routes.

Re:Ditto the A.C. (4, Informative)

petecarlson (457202) | more than 5 years ago | (#26880049)

It wasn't just AS47868, it was kicked off by AS47868 sending real long routes like you can get to a by going through b, c, d, e, f ,g, h... and so on and so forth. Older versions of IOS wack out with the crazy long routes and lose their BGP sessions so it is possible that he lost half of the internet while you were on a network segment which was not seeing the issue. If the OP were to post the ASN or IP block he was on we could run BGP play and see just how much of the net he really lost. I'm going to guess about .5%.

Re:Half the internet? Are you serious? (4, Funny)

Frosty Piss (770223) | more than 5 years ago | (#26879631)

A router takes out 'half the internet' and I learn this from Slashdot?

Non, no, no. You messed up the troll and got modded "Insightful". Let me fix that for you:

A router takes out 'half the internet' and this is front page news at Slashdot? Slow news day?

Thank you, I'll be here all week...

Re:Half the internet? Are you serious? (0)

Anonymous Coward | more than 5 years ago | (#26879861)

I work at a hotel it made our connection over many t1's seem like 9800 baud we have to do some of our night close out work online to close out the day on our credit card processor it took me 30 min to do this when it usually only takes up to 5 min as it was very very slow the hole hotel had little to no internet access witch made our guests very upset at around 2 am it mysteriously came back and was working fine wish i had waited till 2 am do do the end of day close

Re:Half the internet? Are you serious? (3, Insightful)

jjrockman (802957) | more than 5 years ago | (#26879939)

Punctuate much?

Re:Half the internet? Are you serious? (1)

glwtta (532858) | more than 5 years ago | (#26879949)

Your connection also seems to be dropping all of your punctuation characters. Very annoying problem to have.

Re:Half the internet? Are you serious? (5, Funny)

Anonymous Coward | more than 5 years ago | (#26880259)

A router takes out 'half the internet' and I learn this from Slashdot?

Seriously, what is/was the impact? I work for a large e-commerce provider and haven't seen a thing that would indicate a problem today.

Well I'm not sure about you.

Personally, I have BIGGER news! A single router in a remote rural US state managed to take down the ENTIRE INTERNETS!!!!

Yes, indeed when I noticed my cat had unplugged the power adapter, I replaced it. Then the ENTIRE internet came back! It was amazing how I single-handedly brought back the whole internets. Al Gore would be proud.

Sorry (4, Funny)

Alcoholist (160427) | more than 5 years ago | (#26879425)

My bad. I never should have cut that tape.

Hmm... (2, Interesting)

Vectronic (1221470) | more than 5 years ago | (#26879445)

I suppose that a networking event with one of our upstreams [merit.edu] was behind that router?

3/11 (invalid or corrupt AS path)

Or maybe I'm behind that router?

Re:Hmm... (1)

Vectronic (1221470) | more than 5 years ago | (#26879547)

Ignore that, it's working now, just took 4 minutes.

Pre-FUD propaganda (3, Interesting)

marco.antonio.costa (937534) | more than 5 years ago | (#26879475)

No, we DON'T NEED A NEW INTERNET! Stop pitching it, statist drones.

The internet works fine, and that's what the RIAA/MPAA/etc are trying to fix.

Re:Pre-FUD propaganda (1, Funny)

Anonymous Coward | more than 5 years ago | (#26880007)

>Stop pitching it, statist drones.

When you say 'statist drones' I picture you in your mother's basement wearing her mascara and listening to Rage Against the Machine loudly. How close am I?

Re:Pre-FUD propaganda (4, Funny)

Hecatonchires (231908) | more than 5 years ago | (#26880207)

You left out 'updating your myspace page', 'writing poetry about how no-one understands' and 'cutting yourself'

You get Duct tape? (4, Funny)

Forge (2456) | more than 5 years ago | (#26879483)

Lucky Yankees with all your fancy technology. If I told you what we use, nobody would respond for fear that in attempting to respond I would cause a few fatalities.

Yep, Its true (4, Informative)

Bryansix (761547) | more than 5 years ago | (#26879485)

Our Hosted VOIP service took a dump today at 8:40 AM PST. Supposedly it was a server in the Czech Republic. From the Carrier

2009-02-16 0945 PST CP experienced a core network connectivity issue due to a world wide BGP issue that affected all BGP interconnected networks. A rouge machine in the Czech Republic was making bad AS advertisements that caused systems world wide to fail. We have worked with our providers as well as our internal Engineering department to effectively block this node and restore service to our network. This is an ongoing issue that is still being worked to get a 100% correction. There is a workaround currently in place until a complete fix is available.

Re:Yep, Its true (5, Funny)

radish (98371) | more than 5 years ago | (#26879619)

A rouge machine in the Czech Republic was making bad AS advertisements that caused systems world wide to fail.

Now I really don't know all that much about large-scale networking so maybe someone could explain this to me. What difference does it make if the router is rouge, versus say, green? or black?

Thanks for any insight :)

Re:Yep, Its true (5, Funny)

ChunderDownunder (709234) | more than 5 years ago | (#26879795)

Since folks on Slashdot seem to like car analogies, I'll just mention that Red Cars Go Faster [google.com] and assume that the same law applies for routers.

Re:Yep, Its true (5, Funny)

pyite (140350) | more than 5 years ago | (#26879821)

Now I really don't know all that much about large-scale networking so maybe someone could explain this to me. What difference does it make if the router is rouge, versus say, green? or black?

So they announced a route that was, shall we say, malformed. Part of the problem is that due to a Cisco bug (CSCdr54230), some routers choke on it instead of ignoring it. The bug is fixed. It was fixed some time ago. Nonetheless, it's a pretty bad bug, labeled as "1 - catastrophic" by Cisco (in red letters, even). Routers still running affected code versions are having issues.

And it's only at this point in writing my reply that I realize you were taking advantage of a pun by way of misspelling. I'll leave my reply anyway ;-)

Re:Yep, Its true (4, Funny)

Anonymous Coward | more than 5 years ago | (#26879963)

Everyone knows rouges are overpowered, just ask any mage.

Re:Yep, Its true (4, Funny)

Hecatonchires (231908) | more than 5 years ago | (#26880215)

Yes, Mages are known for powdering their cheekbones. Rogue's on the other hand, like to stab people in the back.

Re:Yep, Its true (1)

Anonymous Coward | more than 5 years ago | (#26879699)

Wow, those Czechs have machines for rouge? I guess that explains why they have so many fashion models...

Re:Yep, Its true (1)

Jurily (900488) | more than 5 years ago | (#26879783)

A rouge machine in the Czech Republic was making bad AS advertisements that caused systems world wide to fail.

Wasn't the internet designed to withstand WW3? And now a single machine takes it down?

Please enlighten me, how is that possible?

Re:Yep, Its true (0)

Anonymous Coward | more than 5 years ago | (#26879951)

Because it's not true. One border router can cause some problems here and there, but the rest of the internet will be fine.

Re:Yep, Its true (1)

Zironic (1112127) | more than 5 years ago | (#26880013)

Internet is meant to be able to route around broken nodes and it seems to do this rather well, just not as quickly as some would like.

Re:Yep, Its true (3, Insightful)

mail2345 (1201389) | more than 5 years ago | (#26880027)

WW3 is an external problem.
A misconfigured machine is an internal problem.
The internet can survive cut cables, provided that there are other routes.
But if it can't find said routes, then there is a problem.

Re:Yep, Its true (1)

ScrewMaster (602015) | more than 5 years ago | (#26879859)

A rouge machine in the Czech Republic

That's the problem. You shouldn't use rouge on your routers.

Re:Yep, Its true (5, Insightful)

myowntrueself (607117) | more than 5 years ago | (#26879997)

That's the problem. You shouldn't use rouge on your routers.

I think that a rouged router would possibly be overly promiscuous.

No wonder problems like this can spread like the clap in a port town!

Am I being too vauge? (5, Funny)

HTH NE1 (675604) | more than 5 years ago | (#26880003)

That's the problem. You shouldn't use rouge on your routers.

They think a rouge router is in vouge, but they're out of their leauge. We should haranuge them! A plauge on them! Rip out their tounges so they cannot aruge! Them and their colleauges. Nothing but demagouges and idealouges I say. There can be no dialouge on this matter. Send them to the moruge!

Are you intriuged by my ideas and want to subscribe to my travelouge?

Re:Yep, Its true (1)

Threni (635302) | more than 5 years ago | (#26880065)

Are you sure it's not just affecting a couple of shitty boxes in Prague or wherever? I've noticed no problems today. I'm sure there are a few pissed bedroom boys getting high ping on their Open Arena servers, but it's hardly the end of the world.

Re:Yep, Its true (1)

Hadlock (143607) | more than 5 years ago | (#26880291)

Our ecommerce site went down at 10:40am CST (-6 GMT) due to an insane amount of traffic coming from China (different from your location). We finally blocked the IP at the firewall but an interesting coincidence.

AS 47868 (5, Informative)

Anonymous Coward | more than 5 years ago | (#26879495)

There is a post in nanog and on isc.sans.org.

AS 47868 causing AS paths to become too long...

http://www.merit.edu/mail.archives/nanog/msg15472.html

Oops (3, Funny)

kbob88 (951258) | more than 5 years ago | (#26879509)

Sorry, I *told* Mustafa not to drop the anchor there! But does he listen to me? No...

You're doing WHAT with the wire? (0, Offtopic)

d3ac0n (715594) | more than 5 years ago | (#26879511)

Just a short "Spelling Nazi" break: It's spelled "baling" wire, not "bailing" wire.

Baling wire is wire used for strapping bales of hay together. I'm not entirely how one would use wire for bailing out a sinking boat, but if you can figure it out then you can call it "bailing" wire. Until then, please spell correctly.

/Spelling Nazi

Re:You're doing WHAT with the wire? (0)

Anonymous Coward | more than 5 years ago | (#26879759)

I'm not entirely how one would use wire

If you post correcting someone's post, be sure to ensure your post is correct.

Re:You're doing WHAT with the wire? (1)

pitterpatter (1397479) | more than 5 years ago | (#26879781)

Just a short "Spelling Nazi" break: It's spelled "baling" wire, not "bailing" wire.

Baling wire is wire used for strapping bales of hay together. I'm not entirely how one would use wire for bailing out a sinking boat, but if you can figure it out then you can call it "bailing" wire. Until then, please spell correctly.

/Spelling Nazi

I affirm your "Spelling Nazi" correction, but would like to encourage you to proofread carefully when being pedantic. In a spirit of helpfulness, I'd like to provide the "sure" that I think you dropped between "entirely" and "how." You're welcome.

Re:You're doing WHAT with the wire? (3, Funny)

Vectronic (1221470) | more than 5 years ago | (#26879787)

Main Entry: bail
Function: verb
Date: 1613
transitive verb
1 : to clear (water) from a boat by dipping and throwing over the side usually used with out
2 : to clear water from by dipping and throwing usually used with out

Bailing Wire = Internet Tubes

Re:You're doing WHAT with the wire? (0)

Anonymous Coward | more than 5 years ago | (#26880055)

Ummm--I agree and have used much of it. However, the handle of a bucket is also called a bail, and when, in the course of human events, our bucket's handle breaks, why then we must repair with our Bailing Wire--or--in the vernacular--"balun war"...

Re:You're doing WHAT with the wire? (1)

gyrogeerloose (849181) | more than 5 years ago | (#26880243)

why then we must repair with our Bailing Wire--or--in the vernacular--"balun war"

Balun war? You mean people are fighting over impedance-matching transformers now?

I'm tellin' ya, this world is going to hell in a hand basket--with a wire handle.

Trust (3, Insightful)

TubeSteak (669689) | more than 5 years ago | (#26879535)

Until the internet evolves away from its trust-everyone roots,
one well placed server will be able to cause massive damage.

There would be a lot more impetus to force the change if hackers were nuking things from orbit for lulz instead of infiltrating systems for business reasons (spamming, bot herds, etc).

Re:Trust (3, Informative)

lotaris (34307) | more than 5 years ago | (#26880203)

This only hit people running old unpatched versions of IOS. Known and patched long ago.

baling wire, not bailing wire (4, Informative)

bugi (8479) | more than 5 years ago | (#26879539)

http://en.wikipedia.org/wiki/Baling_wire [wikipedia.org]

I think you mean baling wire. One uses buckets for bailing.

Re:baling wire, not bailing wire (1)

Tomun (144651) | more than 5 years ago | (#26879565)

No only that, but leave duct tape alone. It's an excellent way of holding things together. I'd trust your life to the stuff.

I don't know what is sadder (0)

EmbeddedJanitor (597831) | more than 5 years ago | (#26879661)

Spelling Nazi getting feathers out of place over bailing vs baling.

That there is a Wikipedia article on baling wire.

Re:baling wire, not bailing wire (1)

onkelonkel (560274) | more than 5 years ago | (#26879789)

Could I use baling wire to make a new bail for my bailing bucket? If so, would my wire bail made from baling wire become bailing wire?

Concerned and puzzled.

Oblig. I.T. Crowd (4, Funny)

XanC (644172) | more than 5 years ago | (#26879563)

What is Jen doing with The Internet??

Re:Oblig. I.T. Crowd (3, Funny)

Anonymous Coward | more than 5 years ago | (#26879917)

Googling Google on google?

Re:Oblig. I.T. Crowd (1)

bazald (886779) | more than 5 years ago | (#26880001)

Has she tried turning it off and on again?

Re:Oblig. I.T. Crowd (1)

DecepticonEazyE (1165265) | more than 5 years ago | (#26880163)

The Elders of the Internet will be very upset!

Re:Oblig. I.T. Crowd (0)

Anonymous Coward | more than 5 years ago | (#26880221)

Thank you for reminding me of the episode, i had a good chuckle.
Funniest thing i have seen in a very long time.

The end was priceless. /off-topic

Re:Oblig. I.T. Crowd (0)

Anonymous Coward | more than 5 years ago | (#26880265)

Did someone search for "Google" in Google?

Oh, it's obvious, you know... (1)

jtara (133429) | more than 5 years ago | (#26879585)

If you can memorize this, you'll be the life of any cocktail party:

"We're seeing them from AS 48438, coming across to us as an Optional Transitive Attribute which our force-10s are not parsing (but cheerfully passing along to our clients, who are then flapping their peers because of it.)"

Uh-huh-huh-uh! They've been "flapping their peers".

Don't knock duct tape (3, Insightful)

fm6 (162816) | more than 5 years ago | (#26879643)

Well, do, you're right to be concerned. The thing is, our technology infrastructure has always been a nasty kludge. In 1965, some coincidental misconfigurations at two minor power plants took out the power grid for an area in the northeast U.S. and eastern Canada where 25 million people lived. It was 14 hours before the grid was fully restored. Our inability to keep our technical house in order is a very old problem.

Outage Cause: Old software (5, Informative)

Anonymous Coward | more than 5 years ago | (#26879649)

The AS 47868 decided that they wanted to prepend their ASN about 75 or so times to their BGP announcements. When this got re-populated throughout the rest of the world, a bug in older versions of Cisco IOS still in use on many ISP/NSP networks does not like paths this long. As soon as they saw the prefix with that long of a path, the software terminated the BGP session, resulting in the doorway being closed between the two networks -- So on and so forth throughout the rest of the web.

Make sure you are using cat 5 bailing wire. (4, Funny)

tlambert (566799) | more than 5 years ago | (#26879665)

Make sure you are using cat 5 bailing wire.

-- Terry

Re:Make sure you are using cat 5 bailing wire. (5, Funny)

egcagrac0 (1410377) | more than 5 years ago | (#26879871)

Can't. It's Monday. No cheezburgers.

It took out 9000 internets (4, Funny)

need4mospd (1146215) | more than 5 years ago | (#26879669)

In other words, a single mis-configured router is apparently able to cause a DOS for a huge chunk of the Net.

This means the router was able to take out over 9000 internets. Quite impressive.

Re:It took out 9000 internets (0)

Anonymous Coward | more than 5 years ago | (#26880109)

Where is Al Gore when we need him?

I'll bite... (3, Funny)

YodaToad (164273) | more than 5 years ago | (#26880247)

OVER 9000?!

meanwhile (0)

Anonymous Coward | more than 5 years ago | (#26879687)

and one wire-tapped router [theregister.co.uk] monitors half the internet...

It's not just the internet (1)

Weaselmancer (533834) | more than 5 years ago | (#26879689)

A lot of things, as it turns out, have these single points of failure that propogate.

I got to experience this one. [wikipedia.org]

Drove down Route 76/80 to NYC while it was happening. One city would be on, another off. No rhyme or reason to it at all.

ISC is your friend (0)

Anonymous Coward | more than 5 years ago | (#26879735)

http://isc.sans.org/diary.html?storyid=5872

I'm not sure I follow (2, Funny)

shaitand (626655) | more than 5 years ago | (#26879793)

If I'm understanding this 'router' thing correctly, its like a faucet connected to the series of tubes?

If not, exactly what role does this router thing play in tube interaction?

Duct-tape and bailing wire (1)

HTH NE1 (675604) | more than 5 years ago | (#26879829)

And people don't believe me when I tell them all this new-fangled technology is held together by duct-tape and bailing wire!

And chewing gum. Don't forget the chewing gum.

Similar story at MIT (1)

mpoon (1382749) | more than 5 years ago | (#26879839)

Funnily enough, something like this happened at MIT about a month ago. The whole MIT subnet went down for about 2 hours. Cause? Switch that was plugged into itself.

I wish... (2, Insightful)

egcagrac0 (1410377) | more than 5 years ago | (#26879855)

people don't believe me when I tell them all this new-fangled technology is held together by duct-tape and bailing wire

If only it were that reliable... my duct tape patches and bailing wire repairs typically hold for a decade.

You forgot to mention.... (1, Funny)

gearloos (816828) | more than 5 years ago | (#26879877)

You forgot to mention which Microsoft company the router belonged to....

No news here (0)

Anonymous Coward | more than 5 years ago | (#26879903)

There's no news here. It's been known for a long time that the entire Internet routing only works by agreement and fiat.

Remember a couple of years ago when one country managed to blackhole all Youtube for almost the whole 'net? Same type of thing here. As soon as any single entity decides, on purpose or by accident (via malfulction or human error) that it's not going to play by the rules, all the systems on the 'net get confused. They don't deal well with rules changes.

If it happened on South Park... (1)

HeLLFiRe1151 (743468) | more than 5 years ago | (#26879919)

it could happen anywhere.

Belkin = Broken (1)

pRtkL xLr8r (1264376) | more than 5 years ago | (#26879929)

So all terrorists have to do is buy a bunch of Belkin routers and set them up as per normal?

Duct tape, baling wire, chewing gum and... (0)

Anonymous Coward | more than 5 years ago | (#26879977)

a hot melt glue gun. I know its a bit high tech but I use mine all the time and my LAN runs great.

Someone dumping enormous amounts of material (1)

Beelzebud (1361137) | more than 5 years ago | (#26879983)

If I don't get that internet my staff sent me on Friday, I'm going to be pissed. Damn kids dumping enormous amounts of material in the tubes... IT'S NOT A DUMP TRUCK!

Ye olde versions of IOS (5, Informative)

DeadBeef (15) | more than 5 years ago | (#26880005)

This only broke BGP implementations that are getting pretty long in the tooth now, on a moderately recent version of IOS all we saw is:

Feb 17 05:25:03.731 nzdt: %BGP-6-ASPATH: Long AS path 10026 3356 29113 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 47868 received from xxx.xxx.xxx.xxx: More than configured MAXAS-LIMIT

It was definitely an insane path, our routers were configured to drop anything with an AS path longer than 75, old versions of IOS would often just drop the BGP session ( or even crash with some _really_ old versions ).

I'm sure there will be some red faced network engineers updating IOS or even doing forklift upgrades of old boxes at their edges in the near future.

BGP Misconfigurations (2, Informative)

Taikutusu (1479335) | more than 5 years ago | (#26880015)

Misconfigurations occur more than you would think, especially with regards to BGP; one estimate is around 300 per day. Most aren't going to knock our a substantial portion of the network (most of the time they'll either make paths longer or simply knock out the origin network), although occasionally you'll see a "black hole" effect like this. Again, these misconfigurations occur all the time, it's just that no-one really notices unless it manages to bring down any sizeable portion of the network, which is pretty rare.

Laugh all you want... (-1, Offtopic)

painehope (580569) | more than 5 years ago | (#26880035)

But this is seriously fucking up my day. First I discover that the idiots at my college (hey, it's a recession, I decided to finish my degree) deactivated the account I've had for ages on a system I need for class, and lost all records pertaining to it.

Then after I'm told it's fixed, I go to work on it today. Won't take my password. Can't access Gmail to send a goddamn e-mail to the one person my professor found that actually knows root on the system (it's an old alpha running Tru64) for hours. Then when I finally get a password, log in, tweak my settings, all that - I discover that I can't access the directory where my assignment is supposed to be.

At first I thought it was my usual ISP bullshit, reset my cable modem a few times. Problem kept happening. Can't access sites for shit, took me hours to get mails out to this "admin", and now it's after 5PM and I can't get access to this fucking directory because of this fucking bullshit!

Anyone want to pitch in on a plane ticket for me to go there and kill this fucking router admin? I promise I'll post the videos on the net for you. One "network administrator" force-fed a smashed-up router (probably a Cisco) and then garrotted in his network closet.

Where's the story? (1)

Eil (82413) | more than 5 years ago | (#26880063)

Okay. So I'm supposed to believe that one "smallish hosting provider" and three email messages are proof that half the Internet went down today?

WTF.

Have the submitter and kdawson both forgotten what an Internet is?

Only older versions of IOS with known bug (0)

Anonymous Coward | more than 5 years ago | (#26880073)

It only broke routers running old versions of IOS with a know bug related to AS-PATH length.

Anyone running patched IOS, non-cisco devices or who configured a max AS-PATH length on their ciscos was fine.

Tens of thousands of web sites were affected (2, Informative)

miller60 (554835) | more than 5 years ago | (#26880141)

This incident knocked several major hosting providers [datacenterknowledge.com] offline, including Media Temple in Los Angeles and Canada's iWeb.

Only some old versions of IOS broke (5, Informative)

lotaris (34307) | more than 5 years ago | (#26880175)

This only took down people running fairly old versions of IOS that didn't patch a known bug.

Did not affect non-cisco.
Did not affect modern versions of IOS
Did not affect old versions of IOS that set the knob to limit the max as-path.

Come again? (1)

digitalcowboy (142658) | more than 5 years ago | (#26880267)

You did what now, where and it caused which?

On your next visit be sure to ask your "counselor" about narcissistic tendencies.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>