Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Feds Investigating Water Utility Pump Failure As Possible Cyberattack

Soulskill posted more than 2 years ago | from the water-the-chances dept.

Government 136

SpuriousLogic writes with this quote from CNN: "Federal officials confirmed they are investigating whether a cyber attack may have been responsible for the failure of a water pump at a public water district in Illinois last week. But they cautioned that no conclusions had been reached, and they disputed one cyber security expert's statements that other utilities are vulnerable to a similar attack. Joe Weiss, a noted cyber security expert, disclosed the possible cyber attack on his blog Thursday. Weiss said he had obtained a state government report, dated Nov. 10 and titled 'Public Water District Cyber Intrusion,' which gave details of the alleged cyber attack culminating in the 'burn out of a water pump.' According to Weiss, the report says water district workers noted 'glitches' in the systems for about two months. On Nov. 8, a water district employee noticed problems with the industrial control systems, and a computer repair company checked logs and determined that the computer had been hacked. Weiss said the report says the cyber attacker hacked into the water utility using passwords stolen from a control system vendor and that he had stolen other user names and passwords."

cancel ×

136 comments

Sorry! There are no comments related to the filter you selected.

Darned commies (5, Funny)

Anonymous Coward | more than 2 years ago | (#38101898)

Tryin to interfere with America's precious bodily fluids

Re:Darned commies (0)

Anonymous Coward | more than 2 years ago | (#38102170)

Because America is a giant Zerg hive?

Re:Darned commies (0)

Anonymous Coward | more than 2 years ago | (#38102320)

I only drink rainwater.

Re:Darned commies (0)

Anonymous Coward | more than 2 years ago | (#38102790)

and pure grain alcohol, yuo insetsitvie cold!

No Reason (5, Insightful)

sycodon (149926) | more than 2 years ago | (#38102454)

I can think of no reason facilities such as this should be accessible via a public network. You should have to be physically present to access these control systems.

Re:No Reason (3, Informative)

Sarten-X (1102295) | more than 2 years ago | (#38102604)

Unless something goes catastrophically wrong, such as a fire in the control building, in which case the pumps (which must still operate) will need to be controlled remotely. Even during routine operation, the control system is likely connected to a monitoring network of some kind, to make sure things run smoothly.

That means either wiring up a physically-isolated network (and constantly checking it for unauthorized alterations), which is ridiculously expensive, or connecting to the public network physically, and relying on software to keep it secure. Given that this system is probably a few decades old, and probably installed by the lowest bidder, you can make some reasonably-depressing assumptions about how secure that software is.

Re:No Reason:AMEN!!! (0)

Anonymous Coward | more than 2 years ago | (#38102810)

Yes, yes, yes!!! If your system integrator needs remote access, then when he's done, UNPLUG THE %$^^&^ cable!!

'Been in the water/SCADA industry for 10 years... (5, Insightful)

kackle (910159) | more than 2 years ago | (#38103522)

I've been in the water SCADA industry for 10 years. What I'm seeing lately are water operators, IT people, and system integrators who are overzealous when it comes to connectivity and all the "neat" things that can be done remotely via technology. It's the standard human foible when it comes to technology, writ dangerous: they consider what can be done versus whether it should be. The water industry isn't that exciting, so when flashy tech. comes along, and the taxpayer is footing the bill, I can see where they say "Yes!" And who is the salesperson to refuse this order?

I'm all for automation, and crying out when a system is in trouble. But I haven't yet seen where humanized remote control is critical. Hackers aside, it's probably better if it's not.

Re:No Reason (1)

AHuxley (892839) | more than 2 years ago | (#38103598)

The days of paying 2-3 smart people and their helpers to sit around waiting for a problem or looking over a plant daily are over.
You can connect any large scale plant with networked sensors and have one very expensive person sitting at home over looking a wide area of a state.
If a code flashes, expert contractors are sent in to help the small group of very cheap staff on site.
No more teams doing maintenance unless a VIP tour enters or cable tv science show offers free PR.
Count the system down to just before fall apart point and swap out critical parts as needed.

Re:No Reason (2)

mcgrew (92797) | more than 2 years ago | (#38103618)

An update: I just discovered that it's my own city, Illinois' capital, Cartoon City. From the State Journal-Register:

CWLP denies reports it was victim of cyber attack

By DEANA STROISCH (deana.stroisch@sj-r.com)
The State Journal-Register
Posted Nov 18, 2011 @ 11:05 AM
Last update Nov 18, 2011 @ 11:31 AM

City Water, Light and Power officials are denying reports that the utility was a victim of a cyber attack that may have been responsible for the failure of a water pump.

âoeCWLP has not had any breach of its Water or Electric Department Supervisory Control and Data Acquisition (SCADA) systems,â the utility said in a statement issued this morning.

SCADA is the computer control network that operates various systems at the utility.

In a story that has since been picked up by CNN, Joe Weiss, a cyber security expert, says he obtained a state government report dated Nov. 10, which allegedly gave details of a computer hacking that led to the âoeburn out of a water pump.â

The Department of Homeland Security identified the water system as being located in Springfield, Ill.

"DHS and the FBI are gathering facts surrounding the report of a water pump failure in Springfield Illinois, said Peter Boogaard, DHS spokesman. âoeAt this time there is no credible corroborated data that indicates a risk to critical infrastructure entities or a threat to public safety.

âoeIf DHS ICS-CERT identifies any information about possible impacts to additional entities, it will disseminate timely mitigation information as it becomes available."

Amber Sabin, CWLPâ(TM)s public information officer, said there have not been any water pump failures of any kind in the last month.

Links to the reports:

http://www.cnn.com/2011/11/18/us/cyber-attack-investigation/index.html [cnn.com]

http://www.wired.com/threatlevel/2011/11/hackers-destroy-water-pump/?utm_source=co2hog [wired.com]

Copyright 2011 The State Journal-Register. Some rights reserved

 

Re:Darned commies (2)

bobcat7677 (561727) | more than 2 years ago | (#38102700)

That's why I only drink distilled water and pure grain alcohol!

Cyber security expert... (1)

Anonymous Coward | more than 2 years ago | (#38101968)

...thinks innocuous event is a cyber security attack. News at 11.

SCADA vulns (5, Interesting)

sl4shd0rk (755837) | more than 2 years ago | (#38102036)

SCADA systems were sold en masse under the presumption that they were "secure" because they were not connected to public networks. It will be interesting to see which entities did, or did not, follow their policies. Stuxnet was a USB infection but it was still able to route over the internet to phone home. I'm going to bet that a lot of SCADA networks are implemented to allow egress packets. It will be interesting to see how many SCADA systems are actually "isolated".

Re:SCADA vulns (3, Interesting)

Anonymous Coward | more than 2 years ago | (#38102134)

I worked for a Utility in the early 2000's..I was on the post-9/11 security team that had to investigate and close loopholes for that utility. Many sites had interconnected the SCADA systems with the corporate network for GIS information. We were hard pressed to find adequate solutions that would meet the requirements that the federal government set at that time; as the engineering staff didn't want to give up the real-time GIS information they got from the SCADA systems.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38102232)

Without an "air gap" separating electronic infrastructure control systems from any system with a NIC card/USB port, that system is not secure.

Hell, without auditing the contents of any volume traversing that air gap, the system is not secure. If you do not audit the contents of a CD, DVD or even Tape or a swapable or external HDD, the system is not secure.

Any unchecked, unaudited bits being loaded into RAM (even if hand-keyed) create a potential vulnerability to be exploited.

Honestly, this should be obvious. ...and IS obvious to technical people. It's just that laziness and time concerns (read: impatience) prevail.

Re:SCADA vulns (5, Interesting)

mlts (1038732) | more than 2 years ago | (#38102398)

The ironic thing, there is a secure way to get GIS info out, although it isn't the fastest method. I did this on a lab network that needed to be air-gapped from everything else:

1: Build two machines, each has a NIC, and both have a serial card ($60 from NewEgg for a PCIe to Serial.)

2: Build a custom cable with the RX wires cut, so data only goes one way. I did this so an intruder has no chance of being able to send anything to the box on the secure network, much less create a SLIP or PPP connection.

3: Configure one box on the secure network. It scrapes input from the embedded boxes, formats it (so stuff from one device is marked as such so it can be told apart from a different one and to help keep both machines in sync), then pushes it over the serial device.

4: The other box is configured to passively take what comes over the serial port, un-format it (so stuff from one device goes to one web server, stuff from another device gets E-mailed to an admin, alerts get set if something is wrong, etc.)

The result of this is being able to get reports from the embedded boxes on a real-time basis, but without any way of a remote intruder ever getting on the network. Since the physical serial cable cannot send any data to the machine on the embedded network, it would take a physical attack in order to compromise the boxes.

I'm sure there are faster ways to get data across a cable one-way, but this was ideal, as the data obtained was not much, and the latency of the multiple steps to shoot it to a box, stuff it across a serial pipe, then on the other side, send it where it needs to go was just fine.

Re:SCADA vulns (1)

chill (34294) | more than 2 years ago | (#38102484)

Any reason you just don't do the same with a second Ethernet port?

Cross-connect the two boxes via direct connection. Feel free to disconnect the one set of transmit wires, ala a network tap. Use different IP scheme and don't route. Use blind-drop FTP, or SCP (or nntp if you want to be a super-geek).

Gigabit crossover cables beat old serial connections by several orders of magnitude.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38102556)

On the hardware layer, Ethernet needs to have the ability to communicate both ways for error checking/correcting in the frames. Without this, the NIC won't even bother transmitting in a lot of cases.

Re:SCADA vulns (1)

MobyDisk (75490) | more than 2 years ago | (#38102814)

Caveat: I've never tried this, but it got me thinking...

Ethernet hardware doesn't have error correcting, except to retransmit if there are collisions. Error checking would work since CRCs would still be sent. The problem is there would be no mechanism for retransmission. But the serial solution has that same problem. You can send CRCs with your packets, but you can't request a retransmission. But for Ethernet, that is up to the higher layers like TCP. A bigger problem is that the Ethernet speed autonegotiation won't work. You might be able to disable that on the card.

The error handling problem arises with serial too. You would have to send a CRC on each packet, but you would have no way to retransmit if there was an error.

Re:SCADA vulns (1)

greed (112493) | more than 2 years ago | (#38103146)

The usual way of dealing with errors in this sort of situation is to send the data multiple times. That means each message needs a sequence number. (And don't forget to include the sequence number in the checksum; so you drop the whole message if the checksum fails.)

The first step in reducing the error rate, though, is reducing the speed. RS-232 (and -422 and -423) are well-understood and quite robust--in part because they don't use really high speeds, and their data clock is much slower than their sampling clock--a nature of the asynchronous beast.

If you were really going to do this, say for modern boxes with USB, you can even get USB-to-RS-232 cables that end in pigtails or sockets and use TTL voltages; you don't need to bother with the +/-12V level conversion. Just wire RX to TX, GND to GND, and you're ready. (Bonus points for hooking DTR to DCD or something like that for detecting "remote live, no transmission".)

And if I was doing this "for real", I'd use optoisolators so the two boxes really aren't electrically connected: they don't even need to share a ground. Like the way MIDI serial works; it's an electrical current-loop like RS-232, but the receiver is an optoisolator so you don't get ground loops between your instruments and sequencer. (Once you get past that, it's basically RS-232 with an unusual clock oscillator; you can interface to it with standard UART ASICs.)

Errr. So basically what I'd do is use a communication system that is exactly like MIDI electrically (PHY, layer 1) but put my on protocol on it.

Right, where's another wheel I can reinvent?

Re:SCADA vulns (1)

sl4shd0rk (755837) | more than 2 years ago | (#38102640)

> Since the physical serial cable cannot send any data to the machine on the embedded network, it would take a physical attack in order to compromise the boxes.

Very clever. Eliminates network component completely. Is there no way to cause a remote buffer exploit at the downstream end?

Re:SCADA vulns (1)

mlts (1038732) | more than 2 years ago | (#38102848)

The downstream end can be completely rooted, but it won't affect the upstream in any way, other than perhaps putting up fake alerts to try duping people on site to do something, or trashing/modifying the data as it comes down the serial connection. There is no connection other than signal ground that is common and can push data from downstream to upstream.

If the upstream end is compromised; game over.

Re:SCADA vulns (1)

greed (112493) | more than 2 years ago | (#38103290)

To further clarify the mlts' response, you hook up ONLY transmit lines on the transmitter's side. You leave out all of the handshake lines going the other way, so no RTS/CTS handshake; definitely no XON/XOFF.

If the transmitter is too fast for the receiver, the receiver will buffer-overrun and corrupt the data it sees; the UART hardware SHOULD set a status flag when it overruns. But there MUST NOT be any way for the receiver to tell the transmitter to slow down.

It is acceptable to have lines from the transmitter that tell the receiver it is "online", like hooking DTR to DSR. But it can't be done the other way; the transmitter just sends blind.

Re:SCADA vulns (1)

onepoint (301486) | more than 2 years ago | (#38102716)

would someone please mode this up, it's a clean, affordable solution that makes perfect sense without causing any real nightmares.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38103032)

Either you lopped off the control lines too (in which case, props to you for getting the thing to work without those lines), or someone could assert/deassert the control lines back to the system you want protected. If he is lucky, he may find a race condition and crash your "secure" network.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38103376)

Yeah, I thought of that too. A fancy (60 bucks!) serial card might actually have some decent clock recovery circuitry, so it would be OK.

Otherwise, Itd be pretty straightforward to make a small circuit using an MCU (like that Arduino board that thats all the rage amongst kids these days) to implement a phase-locked loop & other signal munging to "track" the voltage and bit timings passively.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38103470)

A plain old serial port isn't a smart protocol. It isn't like USB where it can send packets upstream and downstream. It is as brain-dead as you can get. You dump data to the device, and the other end gets the data. No error correction, no frames, just dumb bits on wires. There is no clock timing, mainly it is bits per second, how many stop bits, parity (which is useless), and async or sync mode.

On the basic level, you have three pins: Tx, Rx, and signal ground. The others are important, but for a well shielded wire, all you really need is the ground, and the wire for transmitting.

No, this isn't fancy, and it is slow, but it is as secure as you are going to get unless you have one side print the bits out on a dot matrix printer, only to fall into a scanner's bin on the other side.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38103696)

It's fairly trivial to change the serial driver to use the RX lines as TX lines. They're the exact same circuits on the serial chip, it's the software that makes them work as RX or TX lines.

So like most security "professionals" you don't understand what you're up against.

Re:SCADA vulns (1)

LostOne (51301) | more than 2 years ago | (#38103962)

Even if this is possible (and it is on some of the hardware I've used over the years), the change must be made on *both* ends or no data can be passed through. It makes no difference if the downstream box switches its transmit and receive lines. You still have no way of making the upstream box do the same unless you have physical access to it, in which case there's no point doing the serial link anyway. So as long as the upstream box has not been tampered with physically, there is no way to send anything *that it is going to receive* because it's not listening for it.

For those who haven't seen it yet, let me repeat: changing one end to transmit on the receive line does not magically mean the other end is going to receive on its transmit line.

Re:SCADA vulns (0)

Anonymous Coward | more than 2 years ago | (#38104004)

Sure, if you had access to the upstream box, that is trivial. But without that physical access, flipping TX to RX means the box just won't transmit data. It won't know that it should be receiving data.

In no case, can the downstream box affect the upstream box... unless someone puts 120VAC down the RX/ground link and fries it.

AWESOME (4, Interesting)

WindBourne (631190) | more than 2 years ago | (#38102038)

That is possibly just a kid playing, however, it could be somebody learning. The nice thing is that it has now been detected. Perhaps it is time to push not just security, but to insist that the parts be western or better yet, American made. Seriously, this is infrastructure that should be local to friendly nations. China is hard at work to make sure that they have the ability to import zero food as well as all of their equipment is from local sources. In doing that, they claim national security. Makes sense. But we should be doing the same.

Re:AWESOME (1)

flyingsquid (813711) | more than 2 years ago | (#38102902)

Kids playing around on the internet tend to pick more interesting targets. Large multinational corporations, government organizations, that kind of thing. Try to imagine a kid saying, "Woo! I stuck it to the man! I struck a blow for freedom! I'm living dangerously! I'm such a badass because I just shut down a WATER PUMP IN ILLINOIS! FUCK YOU, ILLINOIS!"

What I can imagine is a military official talking to the commander of a cyberwarfare unit in China, and saying, "So you say you could in theory launch cyberattacks against critical U.S. infrastructure similar to the way the Americans and Israelis attacked Iran with Stuxnet. And that you could do so in such a way that the attack is untraceable? That sounds very interesting... in theory. It seems to me, however, the only way to know if it would actually work is to test it out against a real target."

Re:AWESOME (1)

epic93 (1863952) | more than 2 years ago | (#38102980)

Or... perhaps it's a cautious kid who knows that such high profile targets would be stupid to attack and wants to learn on something the mass population isn't as concerned about.

Re:AWESOME (1)

WindBourne (631190) | more than 2 years ago | (#38103162)

That is pretty much what I was thinking. In fact, another play on that, is by burning out the equipment early, then they get more sales. However, I suspect that it was what you were saying. That is why I think that finding this is actually an opportunity to fix things.

Re:AWESOME (1)

Grelfod (1222108) | more than 2 years ago | (#38102986)

We should have done that long long ago. Now we are behind the curve and have been infiltrated at every level with outsourced goods and services - that just screams screwed royal... meanwhile dumbass government will spent 10 more years debating it :/

Re:AWESOME (0)

Anonymous Coward | more than 2 years ago | (#38103442)

hat is possibly just a kid playing, however, it could be somebody learning. The nice thing is that it has now been detected.

It wasn't detected. Notice the title says *Possible* Cyberattack. It's not probably some kid, it's probable someone screwed up and in a week or two we'll get a story saying someone typed in the wrong number some place. *If* we get any followup, because non-sexy followup stories don't get published.

having solved all other (2)

nimbius (983462) | more than 2 years ago | (#38102054)

major federal crimes such as the collapse of the united states economy at the hands of wall-street, human trafficking between south america and north america, net neutrality compliance that is largely being ignored by major carriers, civil rights abuses in united states prisons, and protestor police brutality in major metropolitan cities, federal officials target their laser-like scrutiny upon the teeming cesspool of violent crime and evildoings that is Springfield Illinois. their objective? prove a small and unsubstantial water pump in a city of 116,000 people has been nefariously compromised and destroyed by cyber (attackers/hackers/criminals) from (china/iran/north korea/syria) in order to deprive american citizens of their shitty and unaccountably safe drinking water for an evening while the district manager oversees a few dozen pipefitters and welders as they replace a pump on a blustery november weekend.

Re:having solved all other (1, Offtopic)

couchslug (175151) | more than 2 years ago | (#38102352)

They are ignored because the folks who post regarding them have inoperable or intermittent Caps Lock keys.

This particular keyboard malfunction mimics the way the way paranoid nutjobs once typed when conspiracy theorists only had mimeographed newsletters with which to practice "samizdat".

e.e. cummings was cool,

http://en.wikipedia.org/wiki/E._E._Cummings [wikipedia.org]

but he's fucking DEAD and anyone imitating that stylistic affectation (especially at length) needs to be.

The only cure for "crazy" is to do a Hemingway (Ernest or Margaux depending on how much mess you mind leaving) and check out.

Re:having solved all other (1)

Sarten-X (1102295) | more than 2 years ago | (#38102494)

The FBI (or rather, a group of people from it) is investigating a small problem, because it looks like the kind of small problem that can become a big problem later. Perhaps it's now a local water pump in Illinois, but next time it will be a coolant pump at a power plant. Logs from this incident may provide more information about an attack that the "real deal", if this is a practice intrusion.

Given that the investigators knowledgeable about water control systems aren't likely to be the investigators knowledgeable about risk-management accounting, human trafficking, civil rights politics, or the latest tactics for successfully negotiating with irrational group-thinking mobs, I think it's perfectly reasonable that they spend their time doing what they know. The federal officials aren't universally-adaptable masters of all things investigative. Each person has a particular set of skills, so it makes sense that they be split up doing many different things.

Re:having solved all other (0)

Anonymous Coward | more than 2 years ago | (#38102766)

*Smack*, right upside the head.

Now go back to your room.

Find and kill the hackers (-1)

Anonymous Coward | more than 2 years ago | (#38102094)

No judge, no jury.

Possibly, or... (1)

MiniMike (234881) | more than 2 years ago | (#38102118)

Maybe one of the local citizens was just upset about low water pressure and decided to take matters into their own cyber-hands?

It would be interesting if the system hacked was similar to the ones used for the hacked Iranian centerfuges, as both attacks involve spinning a motor too fast.

Re:Possibly, or... (0)

Anonymous Coward | more than 2 years ago | (#38102582)

Water pumps don't normally affect water pressure. They pump water to the top of a water tower for storage until gravity pushes it out.

You are misinformed (1)

_0xd0ad (1974778) | more than 2 years ago | (#38103788)

Water pumps don't normally affect water pressure. They pump water to the top of a water tower for storage until gravity pushes it out.

That is not how they "normally" work. It is only one way that a water system can be designed to work. It can also be designed with pumps that pump directly into the system.

Most water systems of any decent size have a combination of both elevated storage and pumps. Some parts of the water system's distribution are may be pressurized by elevated storage tanks, while other parts of the distribution area are pressurized by pumps.

Obligatory.... (1)

sudden.zero (981475) | more than 2 years ago | (#38102120)

Live Free or Die Hard Reference. I'm the good guy here. I told them this could happen if they didn't prepare. Did I get a "Thank you"? They have been warned now we are all going to pay for their ignorance!

Perhaps Not All Remote Management Worth The Risk (5, Insightful)

stating_the_obvious (1340413) | more than 2 years ago | (#38102122)

Perhaps it's time to start we stop believing that everything in the world needs to be connected to external networks.

In the battle of the sword and the shield, the sword eventually wins, but it takes a hell of a lot longer when the sword and shield are separated by the moat and a thick stone wall...

Re:Perhaps Not All Remote Management Worth The Ris (1)

ColdWetDog (752185) | more than 2 years ago | (#38102272)

Perhaps it's time to start we stop believing that everything in the world needs to be connected to external networks.

Perhaps it's time to stop believing that everything in the world that goes wrong is due to a 'cyberattack'.

Re:Perhaps Not All Remote Management Worth The Ris (1)

idontgno (624372) | more than 2 years ago | (#38102510)

Perhaps it's time to realize both statements are true and completely orthogonal to each other.

Leaping to the conclusion that pump failure in a SCADA-controlled utility is cyberwar is foolish.

Believing that anything remotely important should be connected to a publicly-accessible network is also foolish.

Both skepticism, and air-gapped networks, are very good ideas.

Re:Perhaps Not All Remote Management Worth The Ris (1)

Pope (17780) | more than 2 years ago | (#38102338)

I for one welcome our new stone wall making overlords!

Re:Perhaps Not All Remote Management Worth The Ris (4, Interesting)

Mr. Freeman (933986) | more than 2 years ago | (#38102458)

Perhaps it's time that people realize that a lot of things do need to be connected to external networks and that "air gap them" is simply a cop out response equivalent to saying "use a typewriter".

Yes, some things should be air-gaped, nuclear gas centrifuges come to mind. However, many industrial control systems need to report information over the internet. Remote pumping stations, unmanned power distribution centers, etc. Having a lot of data is not simply a convenience. This data allows engineers to troubleshoot failures, predict future failures, and adjust systems for optimum efficiency.

What's really necessary is for some kind of device that will communicate the data to remote places, but refuse to pass any messages from the outside onto the control system. I don't know how difficult this is, but it's certainly harder than "air gap it". On the other hand, this solution actually addresses the problem.

Re:Perhaps Not All Remote Management Worth The Ris (3, Informative)

idontgno (624372) | more than 2 years ago | (#38102590)

What's really necessary is for some kind of device that will communicate the data to remote places, but refuse to pass any messages from the outside onto the control system. I don't know how difficult this is, but it's certainly harder than "air gap it". On the other hand, this solution actually addresses the problem.

So, what you're saying is, if a utility is too cheap to lay in dedicated network assets and buy their own blacknet (which is not hard to do if you want to), it's ok to just connect the the Internet?

That said, the thing you're looking for is called a unidirectional network [wikipedia.org] . Back in my military network operations days, the colloquial name was "data diode". Data goes one way but nothing (no data, no handshakes, no signaling at all) goes the other way. In that environment, they were used to promote data from a lower-level security environment (say, Secret-only) to a higher-level one with no risk of leak-back.

Yeah. They exist. They're considerably lower-bandwidth than your average gigabit Ethernet switch, but if you're just talking SCADA telemetry, they should suffice.

Re:Perhaps Not All Remote Management Worth The Ris (2)

Obfuscant (592200) | more than 2 years ago | (#38103406)

However, many industrial control systems need to report information over the internet.

Maybe over AN internet, but not over THE Internet. "Report information" is not the same as "allow incoming control or information."

This can be as simple as a Lantronix XPort (or equivalent) tied to a serial port TX line on a secure machine, allowing telnet connections to read the serial data coming out but not send anything back. Or any terminal server with the RX lines cut.

What you need to be careful of in the planning of this system is that the information coming out of the secure system isn't being fed back into the system as the result of an external control. I.e., "Water level low in reactor 5" as outbound information cannot cause an "increase water flow to reactor 5" command from outside.

China is taking US fresh water by the billions of (1)

Anonymous Coward | more than 2 years ago | (#38102128)

gallons, towing it to China in huge bladders and hacking our cities' pumps?

Uh...hackers did it. (0)

Anonymous Coward | more than 2 years ago | (#38102164)

According to Weiss, the report says water district workers noted 'glitches' in the systems for about two months. [...]
and a computer repair company checked logs and determined that the computer had been hacked.

It's not really a stretch to say this event was the result of lax maintenance.

Maybe it was more innocent... (0)

Anonymous Coward | more than 2 years ago | (#38102184)

"Oh...the water pump runs on 480 VAC? I thought it ran on 600! That explains the smoke..."

false flag (0)

Anonymous Coward | more than 2 years ago | (#38102186)

guess we need to setup the great firewall of USA

Makes sense.... (2)

TheCarp (96830) | more than 2 years ago | (#38102188)

Lets face it, when they are putting out advisories actually advertising that one of the FBI's "Most Wanted" is some dude who blew up a package at a building, in the middle of the night, injuring noone, just so he could make some statement about "Animal Liberation".... you really have to wonder what the hell these people actually do for a living anyway.

I mean.... if that dude is one of the top 10 threats out there.... then I think we can all relax.

Quick, somebody find a tenuous link to terrorism so we can look relevant!

Re:Makes sense.... (0)

Anonymous Coward | more than 2 years ago | (#38102400)

Yes, because blowing up buildings is okay if they are empty.

Also, none of the FBI's Ten Most Wanted are bombers such as you describe.

Re:Makes sense.... (1)

TheCarp (96830) | more than 2 years ago | (#38102588)

Well I never said it was ok, just that, if thats the biggest danger out there, then we are paying way too much for protection from it.

But.... they were just sending out notices here in MA that one of their "most wanted" was believed to be in the area and... thats exactly how he was described. Some animal rights bozo.

There.... this guy....

http://en.wikipedia.org/wiki/Daniel_Andreas_San_Diego [wikipedia.org]

Seems to fit my description pretty well.... ok he planted 2 bombs... one incident.... no casualties.... in 2003.

If thats what it gets to be "Most Wanted" and even featured 6 times on "Americas Most Wanted" then...seriously.... I feel pretty fucking safe knowing that this dude is amongst the worst out there.

Not saying hes good, or a hero or anything, just... as far as bad guys go... I am pretty unimpressed.

Re:Makes sense.... (1)

TheCarp (96830) | more than 2 years ago | (#38102648)

In fact, it would be odd for me to say he is good or a hero given that I am one of his legitimate targets, as someone who works for a company that does animal testing. Hell, I have been in the room while they were doing necropsy procedures on mice. I am a fucking monster by his standards.... still.... not so worried.

Re:Makes sense.... (0)

Anonymous Coward | more than 2 years ago | (#38102712)

You are asking me to believe that the FBI maintains dozens of most wanted lists, and this guy is on one of them.

Just to prove you wrong, I did some digging. And, um, it turns out you are right. He is on a list (though not on ten most wanted) and there seem to be a boatload of such most wanted lists.

If you excuse me, I need to see if my name is on the list of most wanted parking violators.

Re:Makes sense.... (1)

TheCarp (96830) | more than 2 years ago | (#38102922)

See, they were making a big deal about it because they thought this guy was here...thats how I fond out.

I didn't realize how many lists they have but...it makes sense. Violent crime in general has been on the downswing since the 90s. Their most recent big op was Whitey Bulger... which was a big deal to some people around here (given that we live less than a mile from where his gang was active... and my wife is from southie)

but... he was a geriatric old man, who stopped being truely criminally active 20 years ago.... well... unless you count coming back to Boston armed a couple of times looking to settle some scores... which... kind of makes me chuckle the thought of a white haired old man in his 70s rolling heavy around boston.

Re:Makes sense.... (0)

Anonymous Coward | more than 2 years ago | (#38103252)

Hmm, supposedly the incidents involved two bombs each with the second bombs on a delay (presumably to injure responders). Assuming this isn't just propaganda from the FBI, this dude sounds like he was ready to be a full on, murder-level terrorist.

Re:Makes sense.... (0)

Anonymous Coward | more than 2 years ago | (#38103390)

Clearly you haven't seen 28 days later...

Re:Makes sense.... (1)

Obfuscant (592200) | more than 2 years ago | (#38103466)

Lets face it, when they are putting out advisories actually advertising that one of the FBI's "Most Wanted" is some dude who blew up a package at a building, in the middle of the night, injuring noone, just so he could make some statement about "Animal Liberation".... you really have to wonder what the hell these people actually do for a living anyway.

You don't think that someone who would go to that extent to make that kind of statement is dangerous to the rest of us?

People who plant bombs and blow things up are dangerous. Period. The fact that he managed not to kill anyone the first time he tried doesn't mean he won't the next. Even if he's not trying to blow people up, it happens. He can't know that an anaimal caretaker isn't visiting a sick animal that night, or doing some late night cleanup, for example. Maybe he screws up the timer and it goes off at 10 AM instead of 10 PM. The bomb doesn't know "I'm not supposed to kill anyone I explode close to".

I'd say that someone who has already gone that far over the line is much more dangerous than your typical bank robber who uses a rubber gun to rob a bank. Not more dangerous than a robber who blows up a bank (like here in Oregon), but they've been caught so they won't be on the Most Wanted list. They're on the "waiting to be executed" list.

Could be something incredibly simple (5, Interesting)

slewfo0t (679988) | more than 2 years ago | (#38102206)

As a controls engineer, I program these type of systems all the time. A simple incorrect setting for when the pumps turn on and off (Lead,Lag) could cause this type of problem. It could literally be a new operator that fat fingered a parameter in the SCADA system. To hack these systems requires specific knowledge of exactly what kind of control architecture is in place at the facility and then having the appropriate software to gain access to the control system. Not that this type of hack cannot be done, but it does require specific knowledge. This really sounds like operator error to me.

Re:Could be something incredibly simple (3, Funny)

ColdWetDog (752185) | more than 2 years ago | (#38102350)

This really sounds like operator error to me.

From TFA:

But in its statement, the DHS said the water system was located in Springfield, Illinois.

Springfield....

Operator error....

Something in the back [wikipedia.org] of my mind....

Re:Could be something incredibly simple (0)

Anonymous Coward | more than 2 years ago | (#38103372)

Please, everyone knows that The Simpsons takes place in Ohio.

Re:Could be something incredibly simple (4, Informative)

Anonymous Coward | more than 2 years ago | (#38102480)

Sort of. To program or configure the specific SCADA system requires specific knowledge of the device, installation architecture, firmware, and version supplied by the system operating manual. Until you get to the S part of SCADA and it all goes into some sort of aggregation platform with a big old GUI on a windows 2000 or windows XP box hooked into a cable modem.

Well, to program them correctly requires that knowledge.

These manuals are often trade secrets for the manufacturer, but are 'openly' passed around by maintenance technicians and field installers, and probably controls engineers such as yourself--although I never had the pleasure to work with one.

Depending upon the organization, such manuals are often shipped to other third party contractors with a "legitimate need" as determined by an engineer or manager.

When you tell them you have a corporate filter on PDFs, they will send to a personal email address if they would send it to start with. If they won't send it directly to you, their client will find a way to get their hands on it and forward it to you.

These manuals contain relatively complex documentation--including ports, encoding types, bit masks, register sizes and addresses that may be remotely configured by a couple of pretty common protocols which tend to be "extended" by the vendor in odd ways.

Sure, every bigwig in the industry has their own special program for everything that talks some proprietary clusterfuck. But mostly, they all have legacy support and some sort of shitty standard that will do basics.

Admittedly, any piece of hardware may implement complicated control processes specific to the device at hand, but all of which (that I've seen) generally fall into about three different "protocol families" for control purposes once you're down to a sensor or switch. Maybe you can't calibrate the device over your basic serial port, but you can throw a relay with it.

All of which I once wrote software for to control via plaintext text message at the demands of a former employer. Who insisted on static vendor passwords, and no encryption or even authorized whitelists to make our controllers easier and faster to install for subcontractors. Plug and Play. Or Pray. Or Plug and hacker prey. Whatever.

Now, you can say it's operator error to use that device. But the bottom line is even in your wealthy industries that do readonly monitoring over encrypted VPN--sooner or later somebody insists on remote control in order to cut maintenance costs. The moment that happens, they're hooked up to hardware that might be 25 years old. And then they're gonna hire somebody with a cheap solution to plug into it.

Re:Could be something incredibly simple (2)

onepoint (301486) | more than 2 years ago | (#38102782)

that's the same shit they said back in 1985ish when those hackers were moving satellites around. nothing is unhackable, sometimes it just takes more time to figure out.

Since when is this hacking? (2)

rudy_wayne (414635) | more than 2 years ago | (#38102268)

the cyber attacker hacked into the water utility using passwords stolen from a control system vendor

WTF?

It's not hacking if you know the password.

Re:Since when is this hacking? (2)

Kikuchi (1709032) | more than 2 years ago | (#38102376)

It can't be anything else than hacking, not when the word cyber appears seven times in the summary.

Re:Since when is this hacking? (1)

gl4ss (559668) | more than 2 years ago | (#38102422)

well the city worker said he didn't do it.

Re:Since when is this hacking? (0)

Anonymous Coward | more than 2 years ago | (#38104152)

Maybe they should ask somebody that was recently let go. At least that would be high on the list of suspects if the password was known by whoever was accessing the system in an unauthorized manner.

At least turning off the pump wasn't as bad as doing something that would make the pump cavitate or cause the packing seals to burn out. Something like remotely operated service valves being shut while a pump keeps operating could have gotten expensive quick.

The sad part is... (0)

Anonymous Coward | more than 2 years ago | (#38102330)

...if a 'security researcher' (aka whitehat, ethical hacker, etc.) had informed them of the potential for this occurring before, they could be sued into oblivion.

And the real threat - if a malicious actor did indeed do this - would walk away, laughing at how idiotic the whole scenario is.

The moral of the story is... (2)

Gyorg_Lavode (520114) | more than 2 years ago | (#38102344)

...a hacked pump at a water station DOES NOT DESTROY THE COUNTRY.

Re:The moral of the story is... (1)

onepoint (301486) | more than 2 years ago | (#38102906)

You are correct that a hacked pump does not destroy the country, but if it's a proof of concept, then 30000 hacked pumps around the country get ton's of people mad/upset/scared ...

the best way to win a war is to starve the opponent into submission, getting rid of the water supply is a solid first step, getting rid of the food can turn it very violent quickly. For some reason people think lack of food = death quickly, which the real truth is lack of water = death quickly. so people will panic if there are food shortages but water shortages won't get people reacting quickly. ( yep someone would get nuked for sure if they stopped food, but water problems, nah )

Think Of The Children!!! (0)

Anonymous Coward | more than 2 years ago | (#38103382)

Why do you hate our freedom?!?!?

Real Cause of Failure (3, Insightful)

fsckmnky (2505008) | more than 2 years ago | (#38102464)

Connecting your water pumps to the public internet.

Der der der.

Re:Real Cause of Failure (2)

dev236 (2509380) | more than 2 years ago | (#38103176)

But internet is a series of tubes? Isn't it?

Re:Real Cause of Failure (1)

fsckmnky (2505008) | more than 2 years ago | (#38103364)

Oh yeah ... *thats* why we hooked the pump to the internet.

You should run for mayor. You gotz da answers w00t ! ;)

F CYBER (1)

Fishbulb (32296) | more than 2 years ago | (#38102550)

I am soooo damn tired of word 'cyber' now. Used to be kind of a neat word, way back when it actually meant something.

Re:F CYBER (0)

Anonymous Coward | more than 2 years ago | (#38102818)

I am soooo damn tired of word 'cyber' now. Used to be kind of a neat word, way back when it actually meant something.

I thought it still meant something? Doesn't it mean that the person who wrote it is a know-nothing douchebag?

Why in the hell (1)

Roachie (2180772) | more than 2 years ago | (#38102602)

Is all this crap attached to the intertubes?

OK, now we don't even have to come into the office to change the position of the control rods and avert a meltdown, we can do it from home, or heck... Kazan, Russia if you really wanted to.

Shitty Risk Management (1)

Anonymous Coward | more than 2 years ago | (#38102668)

Weiss said the report says the cyber attacker hacked into the water utility using passwords stolen from a control system vendor and that he had stolen other user names and passwords."

In other words, people are not capable of understanding the situation they are in. Computers are mysterious, magical creatures, with pink tails and fluffy hair from which you can hold on when riding on the waves of the cyberspace holding a pink bunny, a packet of noodles and wearing the everlasting Viking helmet.

WOIP? (0)

Anonymous Coward | more than 2 years ago | (#38102706)

Do we have Water Over IP already?

Retards, stop trying to connect everything to the Internet.

Re:WOIP? (0)

Anonymous Coward | more than 2 years ago | (#38103078)

I have taken that one step further. I transfer water around with no pipes or wires. A humidifier in one room and a dehumidifier in another. Works great and no bulky pipes or wires in the way. The system is not secure and in theory others could intercept some of my water while it is transferring over. I'm working on that but I haven't seen any rouge or unauthorized dehumidifiers running or open boxes of baking soda in the area so security through obscurity is working so far. I did see a small desiccant packet on the floor near the humidifier the other day. That investigation is still on going but I believe it came from the box that my shoes came in and is probably not an indication of a hack attempt.

We should post the instruction (1)

Roachie (2180772) | more than 2 years ago | (#38102754)

manuals for all this equipment on the the internets in conspicuous locations in the control network with special attention given to tolerances and acceptable operating parameters. So that hacker wont accidentally damage the critical infrastructure component they are the playing with.

Heck, we may want to make more foolproof by publishing user dashboards, with very strict input checking, at playwithcriticalinfrastructure.com.

It the responsible thing ...and knowing is half the battle.

FIRESAIL!!! (1)

m1ndcrash (2158084) | more than 2 years ago | (#38102844)

o_0

Oh dear god .... (0)

Anonymous Coward | more than 2 years ago | (#38102866)

"Joe Weiss, a noted cyber security expert ..."

"... cyber attacker ... "

Seriously! Who fucking talks like that? Cyber cyber cyber .....

Will 'digital outlaw' ever catch on?

Lack of people (0)

Anonymous Coward | more than 2 years ago | (#38102914)

Folks, let's be honest about this. You only attach crap to the intertubes so you can send people home and not have people on hand to cover their jobs. "The water system is out of whack again? Call Johnson at home, he can get in through his PC." Cheaper than hiring a new person and the Republicans get to claim a big, fat score on the reducing the gub'ment goal.

Easy, two-part solution here.

1) Disconnect all of our stuff from the net
2) Hire some people who know about the systems to work the other shifts

Problem solved and not a single right was harmed in this solution.

computer repair company? (0)

Anonymous Coward | more than 2 years ago | (#38103166)

Who did they use a IT consulting firm? well some use remote admin / monitoring and for something like this I hope that some one would at least say something about how unsafe it was.
Did they use a outsourced IT firm that may do stuff like have call centers out side of the usa. They may have on site desktop guys but at some they have to fix / what the clueless call center go wrong.

Now I hope that a system hooked to the water pump was not manged the same way that all other desktops are. Now some firms may do that and the water pump system ends getting software pushed that it does not need and opens it up from some to use that software with a hole to hack in.

But a computer repair company said it was hacked? Did use the geek suard or some one like them if so then the techs can be very hit or miss. And some may say they where hacked when it was some other fault.

No on site IT stuff? (1)

Joe_Dragon (2206452) | more than 2 years ago | (#38103240)

This what out souring and useing vender systems get's you people who are not there and or see you as just a other client you want use to come out a look at a system out of it's window that's a added change.

Can someone explain to me (1)

koan (80826) | more than 2 years ago | (#38103426)

Why such systems are online and accessible via the Internet? Is this a cost cutting measure? Why aren't critical passwords changed every week? Why isn't database information stored in encrypted containers or hard drives? Why does this happen again and again and again?

Several stories online of laptops containing massive DB's get stolen, in fact a previous employed of mine (major chip manufacture) got one of their HR laptops stolen out of a car at Starbucks, I was sent a letter by said company giving the excuse "The laptop hard drive could did not support full drive encryption" which is complete bullshit, full drive encryption has been around for a long time, as have encrypted containers.

Why was that DB allowed on a laptop? Why was it left in the car, but the best question is why wasn't the entire drive encrypted, or at the very least the DB put into an encrypted container?

Re:Can someone explain to me (0)

Anonymous Coward | more than 2 years ago | (#38103550)

Why such systems are online and accessible via the Internet?

They aren't accessible from the internet. It's just that newpeople are too stupid to know that or do reseatch and just believe everything DHS, aka the Department of Hypocrites and Sacmmers, tell them.

I call bullshit. (4, Informative)

Lumpy (12016) | more than 2 years ago | (#38103436)

I have worked with SCADA and water filtration plant pumps, big ass pumps, like 650hp pumps that run on 7200volts.

You cant set it to "burn out". you can adjust the speed of the pump from 10% to 100% the only way to kill a pump is to drop power to it without dropping power to it's valve so it will not close. wait for the pump to start spinning backwards from the water running back downhill through the pump and then slamming the power back on at 100% after the pump was free wheeling in reverse at full speed.

Then they don't burn out, they freaking explode.

This happened when we lost power plant wide and a hydraulic failure kept the valve from auto closing. (not electronic, it's a mechanical/hydraulic thing, a blockage in the pressure line)

Unless the plant was designed by a utter moron and made it so a programming error could blow up parts of the plant.

Re:I call bullshit. (0)

Anonymous Coward | more than 2 years ago | (#38104118)

Not true. Take a vfd and crank up the carrier frequency and most motors begin to have problems. Even inverter rated motors with sequentially wound stators begin to have problems above 4k. A randomly wound stator would begin to develop hot spots way below that. It is not that hard to burn one up.

Are DHS employees morons? (0)

Anonymous Coward | more than 2 years ago | (#38103452)

Weiss said the report says the cyber attacker hacked into the water utility using passwords stolen from a control system vendor and that he had stolen other user names and passwords.

So is that idiot/moron/incompetent-high-school-dropout saying that the so called attacker hacked through the internet into a physical facility (that requires a person to physically push buttons from the control room) ?? Did they just gloss over the fact that a person has to control things from the control room and hope people just ignore that part?

default passwords? (2)

blivit42 (980582) | more than 2 years ago | (#38104076)

From TFA (and the summary):

"Weiss said the report says the cyber attacker hacked into the water utility using passwords stolen from a control system vendor and that he had stolen other user names and passwords."

How likely is it that a control system vendor would have the usernames and passwords of their client, used in the actual production system? Maybe they actually do, as part of some sort of remote support agreement, but if this is the case, that's already a bad security practice.

It seems more likely to me that the vendor has a list of default usernames and passwords, and THIS is what was obtained. Perhaps what Weiss *really* meant to say would be be something like: "Someone got ahold of the default usernames and passwords that our vendor uses. Since we never changed them from the default values, it's our own damn fault."

After seeing SO many stories like this, it's usually a case of not changing default passwords. Given that Weiss's statement *could* be read as I have read it, this seems the most likely scenario to me. I'm going to write this one up as stupidly bad security policies until I have sufficient evidence contradicting this assumption.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>