Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

San Fran Hunts For Mystery Device On City Network

CmdrTaco posted about 6 years ago | from the they-just-can't-get-a-break dept.

Security 821

alphadogg writes "With costs related to a rogue network administrator's hijacking of the city's network now estimated at $1 million, city officials say they are searching for a mysterious networking device hidden somewhere on the network. The device, referred to as a 'terminal server' in court documents, appears to be a router that was installed to provide remote access to the city's Fiber WAN network, which connects municipal computer and telecommunication systems throughout the city. City officials haven't been able to log in to the device, however, because they do not have the username and password. In fact, the city's Department of Telecommunications and Information Services isn't even certain where the device is located, court filings state."

cancel ×

821 comments

Sorry! There are no comments related to the filter you selected.

Simple: (5, Funny)

SilentBob0727 (974090) | about 6 years ago | (#24962357)

Power cycle it with a city-wide EMP.

Re:Simple: (0, Redundant)

num42 (614006) | about 6 years ago | (#24962705)

hmm well he set up the other routers in a way that they loose their configuration upon reload/powercycle - maybe a city-wide EMP is not what city officials would want. I suppose they're still keen to gain access to the rest of the equipment. ;)

Re:Simple: (4, Funny)

Ethanol-fueled (1125189) | about 6 years ago | (#24962911)

All they have to do is look for the small black box with a lone, onerous blinking red LED.

Re:Simple: (5, Funny)

bratwiz (635601) | about 6 years ago | (#24962967)

All they have to do is look for the small black box with a lone, onerous blinking red LED.

Don't forget the obligatory RED and BLUE wires. Every small black box with lone onerous blinking red LED MUST have red and blue wires. Its a rule.

Re:Simple: (1, Insightful)

Anonymous Coward | about 6 years ago | (#24963165)

I think you guys mean "ominous", not "onerous".

Onerous (3, Funny)

Anonymous Coward | about 6 years ago | (#24963285)

All they have to do is look for the small black box with a lone, onerous blinking red LED.

I find it difficult to understand how a blinking red LED would constitute a heavy burden.

Re:Simple: (1)

Hognoxious (631665) | about 6 years ago | (#24963321)

But Boston's on the other coast.

Re:Simple: (2, Funny)

74nova (737399) | about 6 years ago | (#24962995)

I beg to differ. "Loose" devices are exactly what they're looking for

Re:Simple: (5, Funny)

elrous0 (869638) | about 6 years ago | (#24963283)

As someone who watches a lot of movies, I think I can help them find it. I suggest you look for the ominous looking computer with a single red eye. You'll know you're close when it activates some devious self-defense system (probably involving poisonous gas). Pay careful attention to the background music, as it will provide valuable cues on when to run.

Re:Simple: (5, Interesting)

iced_tea (588173) | about 6 years ago | (#24963333)

Could it be possible that the device is actually virtual? Like a Virtual Machine running under VMware or Virtual PC somewhere, with the software obfuscated or hidden? It would be a lot harder to track down that way.

The story keeps changing. (5, Insightful)

khasim (1285) | about 6 years ago | (#24962363)

From what I've read, his "hijacking" was limited to refusing to give the passwords to his boss whom he considered an idiot.

Given that they cannot hunt down a single device on the network, I'd have to agree with that assessment.

MAC address ... switch port ... it should be easy.

Re:The story keeps changing. (1)

flyingfsck (986395) | about 6 years ago | (#24962423)

Hmm, idiocy has no bounds.

Malice and stupidity. (5, Interesting)

twitter (104583) | about 6 years ago | (#24962741)

Why is Slashdot linking to stories that paint the network administrator as a bad guy when he's so obviously surrounded by morons? These are the same people who published all of their user names and passwords [slashdot.org] . That puts the cost of this "hijacking" into perspective. The cost of trusting their employee with the powers required to do the job was zero.

Mod Parent Up (5, Insightful)

mpapet (761907) | about 6 years ago | (#24962895)

I'd like to add that while the way he handled being surrounded by idiots was wrong, he was clearly surrounded by idiots.

No documentation?
No change control?
No diagrams?

What really rubs me the wrong way is how you haven't heard a single word from the admin and yet he is blamed for everything.

I worked one place where a guy with a great deal of responsibility died. (here today dead tomorrow kind of thing) His peers blamed *everything* on him simply because they could. This sounds like the same thing.

Re:Mod Parent Up (4, Insightful)

AioKits (1235070) | about 6 years ago | (#24963257)

What really rubs me the wrong way is how you haven't heard a single word from the admin and yet he is blamed for everything.

Well, every Stalin needs his Trotsky!

Re:Mod Parent Up (5, Insightful)

Sobrique (543255) | about 6 years ago | (#24963281)

Wait, you mean blame it all on the guy who left (be it through death or a cushy new job) isn't standard practice everywhere?

Re:Malice and stupidity. (-1, Troll)

Anonymous Coward | about 6 years ago | (#24962903)

Ohhh, I get it, you're completely right! Their idiocy completely excuses this man from behaving like a twat. It's all so simple, thanks for clearing that up.

Re:Malice and stupidity. (4, Funny)

bratwiz (635601) | about 6 years ago | (#24962913)

Why can't he be a bad guy AND be surrounded by morons-- you know, the old "bad guy surrounded by morons" routine...???

Re:Malice and stupidity. (5, Funny)

erroneus (253617) | about 6 years ago | (#24963091)

You mean like the VP of the United States? That has been done before.

Re:The story keeps changing. (4, Insightful)

DogDude (805747) | about 6 years ago | (#24962489)

1. Your boss is your boss. Unless there's the chance that somebody could be physically hurt, your employer's passwords are NOT yours, no matter how stupid you think your boss is.

2. Assuming that they have wireless on their network, there's no way to find wireless devices, since they can be put inside of locked buildings. Unless your name is "Superman", there's no real way to find exactly where wireless devices are, as far as I know.

Re:The story keeps changing. (4, Insightful)

goose-incarnated (1145029) | about 6 years ago | (#24962559)

... Unless your name is "Superman", there's no real way to find exactly where wireless devices are, as far as I know.

And exactly how would superman find it? Xray vision? How would he then know he found it?

Re:The story keeps changing. (2, Funny)

bratwiz (635601) | about 6 years ago | (#24962875)

... Unless your name is "Superman", there's no real way to find exactly where wireless devices are, as far as I know.

And exactly how would superman find it? Xray vision? How would he then know he found it?

Um, that's actually an easy one-- he'd zap it with his heat ray vision and then if it stopped, he found it... if not, well-- Ooops!

Re:The story keeps changing. (5, Insightful)

Crudely_Indecent (739699) | about 6 years ago | (#24963113)

If Superman had any IT skills, he'd perform a traceroute to determine the devices gateway. Once the gateway was determined, block the mac address from accessing the network. If the admin of that device is worth his salt, he'll change the mac address and continue. They could then specifically enable allowed devices and forbid all others.

Forget finding it, make the network inaccessible.

City of SF Admins, if this proves to be your resolution, you owe me $150 for 1 hour of my time. Sorry, I do not bill in lower increments.

Re:The story keeps changing. (3, Funny)

IntlHarvester (11985) | about 6 years ago | (#24963215)

City of SF Admins, if this proves to be your resolution, you owe me $150 for 1 hour of my time. Sorry, I do not bill in lower increments.

I know nobody RTFAs, but the city is spending $1 million on consultants to rebuild the network, so sorry a guy like is just too cheap for this project.

Re:The story keeps changing. (3, Insightful)

the_B0fh (208483) | about 6 years ago | (#24962657)

2) It's a freaking terminal server. How many wireless terminal servers have you seen?

Re:The story keeps changing. (1)

kent_eh (543303) | about 6 years ago | (#24962673)

there's no real way to find exactly where wireless devices are, as far as I know.

http://en.wikipedia.org/wiki/Radio_direction_finding [wikipedia.org] http://en.wikipedia.org/wiki/Transmitter_hunting [wikipedia.org]

As long as you know what radio device the rogue is communicating with (Shouldn't be too hard to identify what WAP it's using should it?) finding the mystery box should be fairly straight forward.

Re:The story keeps changing. (4, Funny)

Lumpy (12016) | about 6 years ago | (#24962699)

I CAN find a wireless device It's called Radio direction finding, with the right gear you can do it, and I have located 802.11g devices with it. It's not hard.

so you may start calling me SUPERMAN.

Re:The story keeps changing. (0)

Anonymous Coward | about 6 years ago | (#24963277)

Clark, is that you?

FoxHunt (5, Informative)

ka9dgx (72702) | about 6 years ago | (#24962717)

1> Yes.. people could be hurt because the network in question is used to save lives, so it's OK not to hand the keys to an idiot.

2> It's easy to find wireless devices... I've personally been doing it since the 1980's.. it's called a fox hunt [wikipedia.org] here in the Chicago area. We used to get 1 minute of transmission every 5... with WiFi you can just ping the dang thing... how easy is that?

--Mike--

Re:The story keeps changing. (2, Insightful)

chill (34294) | about 6 years ago | (#24962735)

The other end of that wireless device plugs into a wire, which has a MAC and then runs to a switch port.

You're an 1D10T (5, Informative)

Archangel Michael (180766) | about 6 years ago | (#24962757)

1) They were firing the guy, so he was no longer in the employ of the city, so his boss, was no longer his boss.

2) You don't know what you're talking about. Every IP address on the network should be known. Either through DHCP or static IP address map. A ping sweep should reveal any IP address in use, that shouldn't be. From the ping sweep, one can arp the unknown IPs to get a MAC address, and do a lookup on the Manufacturer code to know what KIND of device the MAC could be. one could use NMAP to try to discover type of device as well. Then you start going to every port on every switch with rogue IPs hanging off it, and manually looking at what is attached at the other end.

As for wireless access points, if you don't have control over them, you pull the freakin plug. Unsecured Access points and open access points should be VLANed off from administrative networked, including not allowing VPN tunnels from unsecured and open wireless access point.

If the boss allows crap like that on the network, he is an idiot, and shouldn't have the Passwords and access codes to anything.

Re:You're an 1D10T (5, Informative)

larry bagina (561269) | about 6 years ago | (#24962933)

Ping replies can be disabled. MACs can be faked. But everyone who supports more government ought to take a look at the incompetence here.

Re:You're an 1D10T (4, Insightful)

Archangel Michael (180766) | about 6 years ago | (#24963315)

Yes, both of those are true (Mac, Ping). Even NMAP responses can be spoofed. However the likelihood of all three being done is not likely. However NMAP will reveal a used IP, and a mac table somewhere will identify what port it is hanging on. Packets have to be routed to it somehow.

And I agree with your last point. I'm a Libertarian. ;)

Re:You're an 1D10T (4, Insightful)

denis-The-menace (471988) | about 6 years ago | (#24962999)

I wish I had mod point for you.

Chances are that internal policies prevent the use of "hacker" tools to secure the network.

Again, the PHBs are idiots!

not necessarily wrong... (5, Insightful)

damn_registrars (1103043) | about 6 years ago | (#24962785)

your employer's passwords are NOT yours, no matter how stupid you think your boss is.

Refusing to give out passwords to higher-ups is not always the wrong thing to do. If you are the network admin, and your job is to maintain security of the network, wouldn't it be reasonable to refuse to hand out passwords to people outside of the network administration roles?

Although I can say that an admin can make that choice at his or her own peril. After all, the higher-ups can always opt to fire the admin and replace him or her with someone who is willing to seek security of their job over security of the network they are paid to administer.

Re:not necessarily wrong... (4, Insightful)

Lonewolf666 (259450) | about 6 years ago | (#24963173)

Agreed.

If a boss I don't entirely trust demanded my password, I'd offer to upgrade his account to the same privileges at mine, but he'd NOT get MY password.

The reason is that if he does something stupid that will show up in logfiles, he can damn well do it on his account and get logged doing so ;-)

Admin code of ethics. (4, Insightful)

khasim (1285) | about 6 years ago | (#24963309)

What would you think of a doctor who, because some exec somewhere decided he should, pushed the WRONG medication / procedure to you?

Where does your ethical responsibility end and the boss's desires begin?

To me there isn't even a question. Fire me. Go ahead. I will get another job.

Re:The story keeps changing. (5, Informative)

LizardKing (5245) | about 6 years ago | (#24963297)

Your boss is your boss. Unless there's the chance that somebody could be physically hurt, your employer's passwords are NOT yours, no matter how stupid you think your boss is.

By the time his boss thought to ask for the password(s), he had already been fired. Any obligation he had to his boss had disappeared. The same goes for documentation and written procedures - I'm not going to document anything after I've been sacked. In this case the guy had been arguing for written procedures to be put in place, but no one in authority would sign them off as any failures would then be their ultimate responsibility. It should be the managers that are taking flack for this, as so often with IT cock ups.

Re:The story keeps changing. (4, Funny)

Fx.Dr (915071) | about 6 years ago | (#24962533)

...his boss whom he considered an idiot...I'd have to agree with that assessment

Second that motion. I'd say these guys are like the Marx Brothers of network administration, except they don't know the Secret Woid, so it looks like they're a couple notches down.

Mod Parent Funny (1)

mpapet (761907) | about 6 years ago | (#24962989)

Excellent Marx Brothers reference. Today is going to be a good day.

Re:The story keeps changing. (4, Insightful)

moderatorrater (1095745) | about 6 years ago | (#24962637)

Agreed. If they're still having problems at this point, they're incompetent jackasses. However, that's not an excuse for the employee to be a jackass too.

Re:The story keeps changing. (0)

Anonymous Coward | about 6 years ago | (#24963055)

Would any competent network engineer want to be employed by those morons? They cannot find a terminal server that provides out-of-band access to the network devices... Just trace the fricking console cables and reset the login password!!!

Re:The story keeps changing. (0)

Anonymous Coward | about 6 years ago | (#24963163)

If by jackass you mean offering his services to find and disable the device at going market rate for specialists then I have to disagree. How the hell else do you deal with idiots other than making them pay out the ass?

Re:The story keeps changing. (0)

Anonymous Coward | about 6 years ago | (#24963247)

Well,

This employee is being kept in county jail while the city complains. (the story made it sound like about a month now). I'd say that I'd probably turn into a bit more of a jackass at that point.

Re:The story keeps changing. (1)

FireStormZ (1315639) | about 6 years ago | (#24962649)

"From what I've read, his "hijacking" was limited to refusing to give the passwords to his boss whom he considered an idiot."

Which is hijacking... Most problems entities will have in regards to security come from within. If my boss were the crown prince of idiots and asked me for a password he should not have I would send him an email CC'd to his boss politely explaining that the password is tightly kept for a reason and if he wanted to break standard security practice and get the password anyway he should just email me back with the request.

As my father always said "The boss ain't always right but he is always the boss" This is a CYA moment..

Re:The story keeps changing. (0)

Anonymous Coward | about 6 years ago | (#24962807)

Man, they must truly have some fucking idiots in charge out there in order for this to have been listed in court documents.

They should replace the whole damn department...

Re:The story keeps changing. (0)

Anonymous Coward | about 6 years ago | (#24962837)

What makes you think that the mystery device is attached to a port on an ethernet switch? Childs was wasn't responsible for the Ethernet/IP router side of things...

Re:The story keeps changing. (1)

wtfispcloadletter (1303253) | about 6 years ago | (#24963017)

Not only that, but once you know which port on which switch something is connected to, it truly is just a matter of following the wires to find the device. A pain in the ass, yes, but it really is that simple.

Unfortunately, it appears that with all the tech talent in SF, not a single person with an ounce of talent or know how has been hired by the city.

MAC search (5, Informative)

jeffy210 (214759) | about 6 years ago | (#24962381)

Um, do what any network admin does with a rouge device. Search out what port its MAC address is connected to and then start tracing the cable?

I'm fairly certain most all current managed switches allow for this. Even with unmanaged ones you can hunt down which unmanaged switch it is connected to and snoop from there.

Re:MAC search (2, Interesting)

Lumpy (12016) | about 6 years ago | (#24962437)

Exactly, hell I can sit down with my laptop and tell you what switch it's connected to in 20 minutes. Bet you $50.00 the community strings on all their network gear is still set to public and private :)

Are the IT people they hire completely dysfunctional? Or do they do what most cities do and not actually hire IT people or networking admins because they command a real salary instead of the $12.00 an hour that someone handy with computers get's...

Re:MAC search (5, Informative)

the_B0fh (208483) | about 6 years ago | (#24962697)

Apparently this was why he refused to give out the admin passwords - he thought, and so far, it appears that he is correct, that they are all morons.

Re:MAC search (1)

onecheapgeek (964280) | about 6 years ago | (#24962973)

Or perhaps, because they don't have the passwords they can't do this? From what I recall (and I have been mistaken before) He has all the equipment set to require a reload of the config data if there is a power cycle so they do not have access to the running config.

How, then, can they use the management functions of the equipment if they can't get to it?

Oh, I get it. They are supposed to pull the plugs (or hard reset) to regain control of the system and then try to set it back up without his help.

Re:MAC search (5, Insightful)

Archangel Michael (180766) | about 6 years ago | (#24962899)

I learned early on, that most people don't see the difference between a $12 hour high school geek and a $75 hr network administrator. All most people see is that both do roughly the same job and there is $63 hour difference.

Most of the time, the $12 hr guy is doing most of the same work as the $75 hour guy. The big difference is when crap like this comes up, the $12 hour guy can spend years trying to figure out what the $75 hr guy can figure out in 5 minutes.

Even when the $12 hr guy screws up, the response is "But he was cheaper". It is cheaper to keep a $12 hr guy trying to keep crapware off a computer, rather than a $75 hour guy who doesn't allow crapware in the first place.

The point I'm making, is that a $75 hr guy is worth it, but only to people where time has real value. People who place no value on TIME, don't care about anything other than $ per HR

Re:MAC search (5, Funny)

Yvan256 (722131) | about 6 years ago | (#24962453)

I'd think that a red device would be easy to spot in a server room.

Re:MAC search (2, Informative)

Soruk (225361) | about 6 years ago | (#24962523)

It's probably hidden in a wall cavity somewhere, a bit like that Netware server in the news a few years back.

The City of SF is undermining its case! (3, Insightful)

StandardCell (589682) | about 6 years ago | (#24962755)

If the city can't even complete one of the most basic network administration tasks of finding a physical device on a network, I think they have absolutely no right to accuse anyone of "hijacking" their network. I hope the defense attorney for Terry Childs brings this up.

Re:MAC search (3, Interesting)

d_ron_218 (1343245) | about 6 years ago | (#24962955)

I worked for a company where they cheaped out on the switch infrastructure and bought low-end Dell switches for the entire network. The kind that don't let you see the MAC address table.

Some guy decided to bring in his Linksys router from home so he could use his laptop and his desktop at the same time (instead of, you know, asking IT to add a second port at his desk). Problem was he left DHCP running on the thing, which obviously led to some confusion. Took forever to find it.

Then again it sounds like the city of 'cisco bough nothing but Cisco gear, so who knows what's really going on here...

Re:MAC search (0)

Anonymous Coward | about 6 years ago | (#24963077)

Um, do what any network admin does with a rouge device. Search out what port its MAC address is connected to and then start tracing the cable?

I'm fairly certain most all current managed switches allow for this. Even with unmanaged ones you can hunt down which unmanaged switch it is connected to and snoop from there.

Uhhh ... Block traffic to and from the MAC address of the device ... at least incapacitate it while you search for it.

One million? (1)

alexborges (313924) | about 6 years ago | (#24962439)

The guy costed the city one million?

How much does it cost for San Fran to have an incredibly stupid IT manager that cannot keep his best talent on the job?

Fuck that: im with the rogue guy.

Um, Traceroute? (1)

linuxwebadmin (694411) | about 6 years ago | (#24962465)

I'd suggest using traceroute if they know the IP address.

to quote bash.org... (4, Funny)

SomeGuyFromCA (197979) | about 6 years ago | (#24962467)

<erno> hm. I've lost a machine.. literally _lost_. it responds to ping, it works completely, I just can't figure out where in my apartment it is.

Re:to quote bash.org... (4, Insightful)

FireStormZ (1315639) | about 6 years ago | (#24962819)

The admin might not be stupid he might be an ass

1) He placed a rouge device (his personal property) on the SF network
2) He set all the network devices on the network to lose all info on a reboot
3) He will hand over the passwords (after jail) to all the devices except the rogue

You can make equipment hard to find ( mac masquerading comes to mind )... I'm only adequate in terms of networking but I am pretty sure someone who is really good can play a mean game of hide and seek. Who knows *what* he was doing with that device? and were I the network admin I would have to *on principle alone* rebuild everything after this guy left..

Re:to quote bash.org... (0)

Anonymous Coward | about 6 years ago | (#24963033)

Will you people please learn how to spell rogue correctly?

Re:to quote bash.org... (5, Funny)

alnya (513364) | about 6 years ago | (#24963043)

He placed a rouge device (his personal property) on the SF network

My guess is it'll be next to his guyliner

Re:to quote bash.org... (3, Funny)

Chris Mattern (191822) | about 6 years ago | (#24963099)

What is this fascination with red devices? Should I start painting my network gear red?

Re:to quote bash.org... (3, Funny)

Yvan256 (722131) | about 6 years ago | (#24963101)

I still don't understand why everyone keeps saying the rogue device is red.

Re:to quote bash.org... (4, Insightful)

russotto (537200) | about 6 years ago | (#24963131)

2) He set all the network devices on the network to lose all info on a reboot

I wonder if this one is just a complete misunderstanding. One article says that they were set to lose configuration files on "reset". That's pretty typical -- if you have some device you don't have the password to, you can do a full factory reset and get it back to the default password, but that also wipes the configuration files. He might have told his incompetent bosses that, and they thought he meant they'd lose the files on a reboot instead.

Anyway, if this guy is what they're making him out to be, they need to completely wipe and reconfigure the network anyway; it's the only way to be sure he didn't leave a few presents for them.

Re:to quote bash.org... (1)

FireStormZ (1315639) | about 6 years ago | (#24963217)

"One article says that they were set to lose configuration files on "reset". That's pretty typical..."

Umm no, its really not typical, while its true I have seen that in places in general you would like to be able to bounce a device and restart it.

"if you have some device you don't have the password to, you can do a full factory reset and get it back to the default password, but that also wipes the configuration files."

Maybe you're right, I was reading this as a power restart because as cisco is involved in this there are many way to subvert the admin password (resetting resigters) to reset the password that *don't* involve loss of configuration information. That makes me suspect if they could do this (all of 10 minutes per device) they would.

"Anyway, if this guy is what they're making him out to be, they need to completely wipe and reconfigure the network anyway; it's the only way to be sure he didn't leave a few presents for them."

Agreed but you might want to take a glance at what he has done first.

Re:to quote bash.org... (0)

Anonymous Coward | about 6 years ago | (#24962997)

Bash.org RIP

This is a job for nmap (4, Interesting)

Jeremiah Cornelius (137) | about 6 years ago | (#24962477)

Hey! Fyodor! They need your number! [insecure.org]

Fyodor spent much of this summer scanning tens of millions of IPs on the Internet (plus collecting data contributed by some enterprises) to determine the most commonly open ports. Nmap now uses that empirical data to scan more effectively.
Zenmap Topology and Aggregation features were added, as discussed in the next news item.
Hundreds of OS detection signatures were added, bringing the total to 1,503.
Seven new Nmap Scripting Engine (NSE) scripts were added. These automate routing AS number lookups, "Kaminsky" DNS bug vulnerability checking, brute force POP3 authentication cracking, SNMP querying and brute forcing, and whois lookups against target IP space. Many valuable libraries were added as well.
Many performance improvements and bug fixes were implemented. In particular, Nmap now works again on Windows 2000.

With just nmap, my old buddies at Farm9 could have sussed this out in a few hours. I think they are still around - as Red Siren / Getronics. [getronics.com]

Ahh. I miss running netcat at 3 AM!

those Cylons are crafty (1)

wardk (3037) | about 6 years ago | (#24962513)

check the reservoir, they like to mess up water supplies

Ghost in the Machine (1)

thbigr (514105) | about 6 years ago | (#24962519)

Let Cyber punk rule!

Please - It's San Francisco or simply "The City" (1, Informative)

Registered Coward v2 (447531) | about 6 years ago | (#24962549)

Tourists...

Re:Please - It's San Francisco or simply "The City (2, Funny)

Anonymous Coward | about 6 years ago | (#24962675)

Hey, at least they didn't say "Frisco".

Re:Please - It's San Francisco or simply "The City (0)

Anonymous Coward | about 6 years ago | (#24962787)

There's only one "The City" that matters on 9/11. - Rudy G.

Re:Please - It's San Francisco or simply "The City (2, Informative)

Sobrique (543255) | about 6 years ago | (#24962789)

No no. "The City" is quite clearly "The City of London". And no where near San Francisco. (I wonder if they use Cisco hardware though, which might make the San Fran - Cisco more apt)

Re:Please - It's San Francisco or simply "The City (3, Funny)

Registered Coward v2 (447531) | about 6 years ago | (#24962937)

No no. "The City" is quite clearly "The City of London". And no where near San Francisco. (I wonder if they use Cisco hardware though, which might make the San Fran - Cisco more apt)

Huh? London is only about 142 miles SE from San Francisco and with a population of about 2000 people barely qualifies as a city, let alone "The City" moniker.

Re:Please - It's San Francisco or simply "The City (3, Interesting)

Sobrique (543255) | about 6 years ago | (#24963111)

Your London may be inferior. Ours definitely warrants a 'City' moniker. Especially when The City of London is distinct from the conurbation that is known as London. And the City of London is actually fairly small - almost exactly a square mile - but ... well, you know what they say. It's not the size, it's how you use it.

Re:Please - It's San Francisco or simply "The City (0)

Anonymous Coward | about 6 years ago | (#24963005)

San Fransisco is not "The City" to anyone not living in the area in any other way then the residents of Oklahoma consider Oklahoma City "The City".

Siding with the network guy (5, Insightful)

John Jamieson (890438) | about 6 years ago | (#24962583)

Man, the more I read about this story, the more inclined I am to believe the network admin.

He may be incredibly bull-headed and lacking social self preservation techniques, but he may have been technically right.

Re:Siding with the network guy (1)

SecurityGuy (217807) | about 6 years ago | (#24963177)

I don't know what part of this you think he's technically right on, other than that he worked for incompetents, which seems to be true. The cardinal sin in this whole mess is that he apparently had exclusive access to a lot of stuff, and nobody was clued enough to say "Hey, that's a bad idea." This is a massive failure of IT management, who should not have let this guy build a house of cards that he can knock down at will, and only he can rebuild.

That said, his actions are still beyond reprehensible.

The scene when they find the server (5, Funny)

UnknowingFool (672806) | about 6 years ago | (#24962593)

I'm sure the scene will be like this:

As Indy deciphered the symbols, he found the correct sequence of tiles to push. The huge stone door slowly opened. Indy grabbed a torch and headed inside. At the end of the long room, there it was on the throne: A massive server. It was archaic, and it appeared to be attached to a punch card reader. Along the sides of the room, there were two rows statutes of archers pointed at the center. Indy made his way slowly to the monitor and keyboard of the server. He brushed away the dust and hit the spacebar. The screen turned on slowly and it displayed:

SCO Server 1.0

Your license has expired. You owe use $699.
>_

Suddenly the archers rotated positions and were aimed at Indy.

"Oh boy."

I've Changed my mind. (2, Insightful)

misterjava66 (1265146) | about 6 years ago | (#24962605)

When I first heard what the rogue-SF-admin had done, I was very negative on his actions.

Now, that once again, and now at least for the third time, I hear of absolute stupidity and ineptness on the group at sf, I am certain the so called rogue was right on the ball from the beginning.

Re:I've Changed my mind. (2, Insightful)

Anonymous Coward | about 6 years ago | (#24962817)

Oh yeah, let's give him a break. Oops, he's been by hit by a bus. Where's his disaster recovery plan? That's right, there isn't one. He fscked his employer with his trumped up little admin attitude. Like most admins, he's on a power trip because he has root access on a network. The shit should have been fired, then sued him into oblivion for illegally locking up infrastructure that doesn't belong to him. Give him a few months jail time to top it off, he'll never get a decent job again.

Sparcstation In The Wall (5, Funny)

gentimjs (930934) | about 6 years ago | (#24962661)

I recall hearing a story about a Sun Sparcstation 2 at my old college that had accidentilly got sealed inside a wall by construction folks when re-working the building the CS lab was in to eliminate a few closets for structural support reasons.. nobody could find it (shock!), but kept using it as a DNS server for another six years. It was found about 2 years after it stopped responding to ping when some component (nvram?) let out, and it started beeping after a power flicker.

Re:Sparcstation In The Wall (1)

Sobrique (543255) | about 6 years ago | (#24962877)

I have heard many variants on this particular story.

Most seem to include a server getting hidden by construction work, but exact details of how it was discovered and what OS/job it was doing are variable.

Sadly it seems this is one urban legend that snopes couldn't lay to rest for me.

Although I did at one time, find a box of sun kit, in our 'goods out'. We'd sent them back to sun, and had to fight to claim our discount (they still gave them the discount in the end) but it turns out they'd never been sent, and had been sat in our goods out for nearly 8 years.

Re:Sparcstation In The Wall (1)

gentimjs (930934) | about 6 years ago | (#24963035)

Yeah, I'm somewhat skeptical of the accuracy of the story, but it was mentioned by campus IT staff more then once, so I'm inclined to give it -some- credibility.

Re:Sparcstation In The Wall (1)

Sobrique (543255) | about 6 years ago | (#24963167)

Oh I'm fairly sure it happened to someone at some point in some form. I'm just curious as to how many of the incidences I've heard are actually the case, or more an 'adopt an urban legend' cases.

I mean, Sun has the 'Server 54' reference from 2001: http://www.sun.com/smi/Press/sunflash/2001-05/sunflash.20010521.3.xml [sun.com]

Just remember. (4, Interesting)

AltGrendel (175092) | about 6 years ago | (#24962725)

These are the guys that the "rogue" admin said were too stupid to run the thing in the first place.

You think they've learned anything about the gear since then? No wonder they're having problems.

Where to look... (4, Funny)

s0litaire (1205168) | about 6 years ago | (#24962867)

Did they try the Rouge Admin's office. It's probably that beige box under his desk... Either that or he made up the device and it does not exist, he's laughing at them ripping the place apart trying to find it :D

Re:Where to look... (1)

Yvan256 (722131) | about 6 years ago | (#24963127)

What's a red admin?

Modern Government (1)

TheNinjaroach (878876) | about 6 years ago | (#24962929)

As usual, our modern government continues to bungle their day to day operations with complete ineptitude.

The only reason this is getting any attention is because the city of San Francisco chose to make the initial debacle a very public affair, and now people are watching.

Hmm.. (1)

drewsup (990717) | about 6 years ago | (#24963059)

If a device pings on a network and no one is there to see it, does it exist? (bring on the tree in the forest metaphors!)

Simple co-dependency (3, Insightful)

Anonymous Coward | about 6 years ago | (#24963069)

If you find that you are "holding the place together", IT-wise, you are likely part of the co-dependency and are part of the problem.

IT and the other management have both agreed to ignore each other, literally or otherwise, allowing each (and the individual personalities) to do things "their way"; damn the best practices, good management, logical, financial, or even legal issues.

Except when things go wrong.

Like a breakup, they can get ugly. And, as the IT guy, you will always lose for it is not your Business, but theirs. You are simply hired help.

Re:Simple co-dependency (2, Insightful)

Sobrique (543255) | about 6 years ago | (#24963223)

It's grossly unprofessional to 'ransom' you're employer in such a way. The job of a _good_ sysadmin, is essentially to make himself redundant. I consider it professionally acceptable to leave it 'almost' redundant, in the 'can read slashdot all day, and provide a good service' kind of sense.

When you're fighting fires, you're failing.

cisco command "show cdp neighbors" (2, Informative)

jamcc (792681) | about 6 years ago | (#24963183)

Will track down where any MAC address is connected. If they have the IP, they can get the MAC. If they have the MAC, they can get what port it's plugged into. Find the switch, find the cable, and air-gap it. I know this, and I'm not even a network guy.

More Evidence (0)

Anonymous Coward | about 6 years ago | (#24963195)

This is just more evidence that the Government of San Francisco is full of a bunch of Morons.

No power outage in the Terry Childs case? (5, Informative)

Joe The Dragon (967727) | about 6 years ago | (#24963255)

http://weblog.infoworld.com/venezia/archives/018376.html [infoworld.com]

An insider claims that the power outage that Terry Childs was accused of using to sabotage the San Francisco network was not a planned outage.

TAGS: Problems, San Francisco's FiberWAN, Terry Childs

If you've been following the Terry Childs case to any degree, you probably know that one of the key allegations keeping him in prison on $5 million bail is that he had willfully planned to cause the network to fail during a planned power outage at the DTIS One Market Plaza Datacenter on July 19th. According to credible information I've recently received, that power outage was only going to affect the cubes and offices in that building, but not the datacenter itself.

Thus, there never was a plan to power down the network core. Thus, there's no way that Childs could have tried to engineer the failure of the network during this planned power outage, since the network core would not have lost power.

[ Follow the Terry Childs saga with InfoWorld special report: Terry Childs: Admin gone rogue. ]

The evidence supporting this claim comes from someone certainly in a position to know: Ramon Pabros, the DTIS Datacenter Supervisor himself. Pabros has been employed by San Francisco's DTIS for a surprising 41 years. He's been the Datacenter Supervisor since 1984. He's been running datacenters for the City of San Francisco since Ronald Reagan's first term, the introduction of the Macintosh, and the second season of The A-Team. It's probably safe to say that he knows what he's doing.

According to my source, he will testify to the fact that he discussed the power outage with Childs several weeks before the outage, and at least 10 days before Childs' arrest. He will also state that Childs specifically asked for confirmation that the datacenter itself would not be affected, and was reassured that it would not lose power.

With this statement, the City's allegations that Childs planned to cause the failure of the FiberWAN basically collapse.

Now, I'm admittedly a stranger to San Francisco politics, and am certainly not a lawyer, but if the DA was going to make these accusations against Childs, shouldn't they have talked to Pabros? If the OMP Datacenter was not going to lose power on that date, then this charge against Childs is essentially the same as charging someone with planning to burgle a store that doesn't exist.

But then again, this is the same DA's office that placed valid group usernames and passwords into the public record, and an IT department that ran public, unprotected websites containing internal emails, core network details, as well as usernames and passwords.

I suppose I really shouldn't be surprised at all.

UPDATE: It appears that Pabros has just announced he will be retiring, effective next Wednesday. I can't help but wonder if one event has anything to do with the other. I do know that there have been a number of odd layoffs from San Francisco's DTIS in the past two weeks.

Posted by Paul Venezia on September 8, 2008 08:48 AM

Road trip (5, Funny)

Oriumpor (446718) | about 6 years ago | (#24963261)

There are now dozens of cars packed full of cheetos cheap laptops and foul smelling individuals travelling near, or perhaps at the speed limit, towards san francisco. They're full of people thinking the same thing, "Shit if they can't find a wired device, they sure as hell can't find a wireless one!"

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?