Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Code Spaces Hosting Shutting Down After Attacker Deletes All Data

Unknown Lamer posted about 3 months ago | from the hackers-without-manners dept.

Security 387

An anonymous reader writes Code Spaces [a code hosting service] has been under DDOS attacks since the beginning of the week, but a few hours ago, the attacker managed to delete all their hosted customer data and most of the backups. They have announced that they are shutting down business. From the announcement: An unauthorized person who at this point who is still unknown (All we can say is that we have no reason to think its anyone who is or was employed with Code Spaces) had gained access to our Amazon EC2 control panel and had left a number of messages for us to contact them using a Hotmail address. Reaching out to the address started a chain of events that revolved around the person trying to extort a large fee in order to resolve the DDOS.

At this point we took action to take control back of our panel by changing passwords, however the intruder had prepared for this and had already created a number of backup logins to the panel and upon seeing us make the attempted recovery of the account he proceeded to randomly delete artifacts from the panel.

cancel ×

387 comments

Sorry! There are no comments related to the filter you selected.

The cloud (5, Insightful)

Anonymous Coward | about 3 months ago | (#47263163)

Good thing people hosted their stuff on the cloud...

Re:The cloud (4, Interesting)

SQLGuru (980662) | about 3 months ago | (#47263203)

Single account to rule them all......the best approach is the separation of concerns (user management, server management, backup / restore, etc.) so that it is a lot harder to compromise everything.

Re:The cloud (5, Insightful)

i kan reed (749298) | about 3 months ago | (#47263243)

But that would have cost the company a little more money.

Re:The cloud (4, Insightful)

NatasRevol (731260) | about 3 months ago | (#47263531)

More likely, actual planning would have to be involved.

Re:The cloud (1, Interesting)

ArmoredDragon (3450605) | about 3 months ago | (#47263573)

I don't think that was a money thing, rather it was an oversight of risk management. Hindsight is always 20/20.

(Besides, where does this "blame the victim" attitude always come from? It's ridiculous. This is equal to saying that wearing scantily clad clothing means a woman deserves to get raped.)

Re:The cloud (-1)

Anonymous Coward | about 3 months ago | (#47263629)

Except that it's more like a scantily clad woman going into a prison yard unattended and complaining when bad stuff happens. The internet is not a safe place, you need to plan accordingly.

Re:The cloud (4, Insightful)

Mister_Stoopid (1222674) | about 3 months ago | (#47263659)

Having an offline backup isn't 20/20 hindsight, it's the absolute basics of the basics.

This is equal to saying that wearing scantily clad clothing means a woman deserves to get raped.

It's more like saying that a guy who dies in a car accident because he was street racing while drunk, high, and not wearing a seatbelt got what he deserved.

Re:The cloud (1, Insightful)

pla (258480) | about 3 months ago | (#47263747)

Besides, where does this "blame the victim" attitude always come from? It's ridiculous.

Bad people exist. Plan accordingly, or don't come crying when you get hacked.

Otherwise, I agree with you, this looks more like an oversight of risk management: When wandering around the park at 2am in a mini-dress... don't.

Re:The cloud (0)

Anonymous Coward | about 3 months ago | (#47263351)

This cloud is certainly full of meatballs.

.

The cloud (3, Funny)

Anonymous Coward | about 3 months ago | (#47263387)

Normally things form clouds AFTER going up in smoke. With the 'new technology' it is the opposite.

Re:The cloud (4, Interesting)

Dishevel (1105119) | about 3 months ago | (#47263389)

The real problem was that they still had access to their stuff and never bothered to look at the number of accounts on the system before changing the password.

The concept was good but the people in charge were in way over their heads and it became suddenly clear to them that they had no business securing other peoples data. Good for them. At least they know what they suck at.

Re:The cloud (0)

AGMW (594303) | about 3 months ago | (#47263445)

Off Topic ... but I like your sig. I've been leaning that way myself for some time now. Much like the US president can only run for two terms, wouldn't it be grand if there was something similar for the politicians lower down the tree! Politicians _should_ be people who've been out in the real World. They should _not_ be people who go to university with the desire to be politicians.

Re:The cloud (1, Insightful)

Dishevel (1105119) | about 3 months ago | (#47263579)

More of us are becoming aware all the time. There is a need for people to fill political offices. There is no need for politicians.

Re:The cloud (4, Informative)

Kagato (116051) | about 3 months ago | (#47263595)

AWS has one of the best security systems out there. IF you decide to enable the features. The production AWS configs I've used have mandated multi factor auth (using the number generator on the phone) as well as network source network restrictions. You can also setup a large number of ACLs to restrict things like the ability to create additional accounts.

It's hard for me to feel bad for these guys.

Re:The cloud (1)

LWATCDR (28044) | about 3 months ago | (#47263713)

Isn't the real problem the criminals that made the attack?

Re:The cloud (0)

Anonymous Coward | about 3 months ago | (#47263777)

If they'd used MFA I doubt that the attacker would have gained access in the first place.

Secure. Responsive. 24/7/365. the Cloud. (1)

swschrad (312009) | about 3 months ago | (#47263401)

and our admin password is "letmein"

Re:The cloud (2)

roc97007 (608802) | about 3 months ago | (#47263435)

Good thing people hosted their stuff on the cloud...

No kidding. Their backups also, apparently.

Re:The cloud (2)

nullchar (446050) | about 3 months ago | (#47263647)

You should always have an offline backup (even if slightly out of date).

In this case, they could have used a separate "cloud" provider just for backups.

Cloud or not, everything under one umbrella was the problem.

Re:The cloud (5, Interesting)

Penguinisto (415985) | about 3 months ago | (#47263441)

Good thing people hosted their stuff on the cloud...

I don't think their problem is necessarily because it was "on the cloud" - the same thing could have happened if someone penetrated a corporate network and got hold of a VM farm. A bigger obstacle to be sure, but if your corporation has partner/vendor access and a not-so-sharp security guy...

One question I have though - instead of changing a password, why couldn't they have called Amazon, had the thing universally locked out for that company, replaced all root-level access with a new account, and sent the new username and p/w by phone back to the company?

Also, why didn't they have an offline (think: off-cloud) backup of the stuff? Sure it costs time/money/skull-sweat to do that, but it's worth the time and trouble in the end. After all, if your family jewels are hanging out there, it always pays to have a DR plan for 'em...

If nothing else, they could have set up a separate and distinct AWS account/rigging as a "DR" of sorts, with DB replication and the works feeding it as a warm DR site. That way if some jackass compromises the first, you only need to stop DB replication, turn on the rest of the DR servers, do a quick test, and shift your DNS to the backup site - 15 mintues later, you can delete the objects yourself in the original site if you want (while you set up yet a different site and build a new backup site to replace the one you just put into production.)

We have a sizable AWS setup where I work, and first/foremost we back that shit up (the DB contents) to machinery that we control. We also have a means of re-deploying/rebuilding if necessary; sure it takes time, but it's better to have it and not need it...

Re:The cloud (5, Insightful)

vux984 (928602) | about 3 months ago | (#47263625)

I don't think their problem is necessarily because it was "on the cloud"

No. The cloud was a key part of the problem. They had as much access and control over the system as the hacker did with no physical fall back.

A VM farm on an onsite rack or even a colo rack? You knock out the hacker by unplugging it from the router to the internet, and then audit and reset security to your hearts content.

Re:The cloud (5, Informative)

Anonymous Coward | about 3 months ago | (#47263683)

With Amazon's service you can contact them and have all access blocked until there is time to sort things out, and authenticate the real admin with billing information or the root SSH key you're given, etc.

Re:The cloud (0)

Anonymous Coward | about 3 months ago | (#47263717)

Also, why didn't they have an offline (think: off-cloud) backup of the stuff? Sure it costs time/money/skull-sweat to do that, but it's worth the time and trouble in the end.

It costs money that some customers don't want to pay? I'm not familiar with their site and business, and if their prices and advertised services suggests one should expect reasonable backup policy. But there are customers that want to go as cheap as possible at the expense of reasonable policy, and as a result services that provide what they want. The problem is most such customers will still blame the company even if they explicitly avoided security and backup practices to save money.

Re:The cloud (3, Insightful)

rwven (663186) | about 3 months ago | (#47263473)

It has nothing to do with the cloud. It could have been any un-managed hosting.

The fact that they went with un-managed hosting in the first place is what really screwed them. If they had a real support team they could turn to, steps could have been taken to keep this from happening as soon as the DDOS started, and they would have had "offsite" or at least "offline" backups.

This happened because it appears that code spaces had some knee-jerk reactions and didn't think through how they were handling this (like changing the password before making sure there weren't other methods of access already established). They should have straight-up called amazon, explained what was going on, and paid for support for amazon put access to their account and instances on lockdown until the situation was resolved. Shoulda, woulda, coulda though...

Re:The cloud (0)

Anonymous Coward | about 3 months ago | (#47263587)

The comment was more in reference to their customers. Must be nice for them all to wake up and find their code base gone because the cloud service they were using got hacked.

Re:The cloud (0)

Anonymous Coward | about 3 months ago | (#47263757)

Good thing people hosted their stuff on the cloud...

"heisted" in the cloud? or from management perspective "hoisted" in the cloud?

Backing up your cloud in your cloud... (4, Insightful)

QilessQi (2044624) | about 3 months ago | (#47263199)

...doesn't seem to work so well.

Re:Backing up your cloud in your cloud... (5, Funny)

gstoddart (321705) | about 3 months ago | (#47263223)

Yo dawg, I hear you like clouds.

Re:Backing up your cloud in your cloud... (1, Funny)

Anonymous Coward | about 3 months ago | (#47263567)

It's clouds all the way down.

Just unplug your server from the internet... (5, Funny)

Anonymous Coward | about 3 months ago | (#47263205)

So you just unplug your server's network connection from the internet while you fix the damage... oh. cloud stuff needs constant internet connection? hm. well I guess that's it then. It was an honor to serve with you. BOOM!

Re: Just unplug your server from the internet... (2, Insightful)

Anonymous Coward | about 3 months ago | (#47263329)

Well, sounds like they first attempted to fix it themselves using ther mad 1337 skills. Amazon cloud is run by adults, and they must have a large staff of top notch security experts. This might sound like monday morning quarterbacking, but if they really feared this threat, they should have called amazon so that not only could they put their instance on ice, they might have gotten some help in hunting down the creep.

Re: Just unplug your server from the internet... (1)

BUL2294 (1081735) | about 3 months ago | (#47263415)

Who do you "call" with most cloud vendors? After all, sounds like whoever was doing the DDOS to extort Code Spaces could have also "called" Amazon to do any number of things, as whoever it was had the passwords, other accounts, etc.

Unless you're one of Amazon EC3's largest customers (e.g. Netflix), you're one of thousands of low-paying customers with rudimentary authentication. Amazon should have an "oh shit" master key that relies on old-school technology, like a RSA number keyfob that the client's president keeps in a locked drawer. That would be the nuclear option. But if something like that were available, it might have cost the client an extra $10/month...

Re: Just unplug your server from the internet... (5, Informative)

Penguinisto (415985) | about 3 months ago | (#47263497)

Who do you "call" with most cloud vendors? After all, sounds like whoever was doing the DDOS to extort Code Spaces could have also "called" Amazon to do any number of things, as whoever it was had the passwords, other accounts, etc..

I've actually worked with them once - sure someone could impersonate them, but you could just as easily call up, explain the situation, and then prove you're the rightful owner of the account (using info that most script kiddies aren't going to think of gathering in the first place, let alone spoof the original contact phone #.)

To their credit, Amazon is actually fairly intelligent and responsive, even to small accounts.

BTW - if you use/handle it right, each instance comes pre-made with a specific SSH auth keyset for root, and you're the only one with the private key (even Amazon doesn't have it) - store/use that as your proof by logging into an instance with one (it's something the script kiddie definitely won't have).

Re: Just unplug your server from the internet... (0)

Anonymous Coward | about 3 months ago | (#47263577)

Well, if i was just joe plummer goofing about on the cloud, i wouldnt have a number to call, but if i was a compqny with paying customers, and all of the companys working capital stored in amazon cloud, i would at least want to have certainly have offline backups, plus a callback number that could be reached in case of a breakin. The same way people secure their houses with commercial security companies.

Picard and Dathon at El-Adrel (1)

Thud457 (234763) | about 3 months ago | (#47263369)

I can't think of a better argument... (5, Insightful)

Lab Rat Jason (2495638) | about 3 months ago | (#47263207)

for air gapped backups.

Re:I can't think of a better argument... (5, Insightful)

Russ1642 (1087959) | about 3 months ago | (#47263241)

If your backups are sitting right next to your active files they aren't backups. They're just copies sitting there.

Re:I can't think of a better argument... (4, Insightful)

DoofusOfDeath (636671) | about 3 months ago | (#47263569)

If your backups are sitting right next to your active files they aren't backups. They're just copies sitting there.

I think that's an oversimplification. They're still backups. They're just not backups against some failure modes that people would have expected.

Re:I can't think of a better argument... (0)

Anonymous Coward | about 3 months ago | (#47263753)

They're backups that would protect you against the most common sources of data loss. Fat fingers and disk failures. It's clearly not a back up against theft or the building burning down, but to say that they aren't backups is misleading as they cover you for the most common cases.

Re:I can't think of a better argument... (1)

Richy_T (111409) | about 3 months ago | (#47263315)

There may be better ones but this is sufficient all on its own. As the poster above me says, if it's not offline, it's not a backup.

Re:I can't think of a better argument... (1)

CAIMLAS (41445) | about 3 months ago | (#47263333)

Or for in-house networks.

Pretty trivial to just pull the cable when your kit has been compromised and you're facing extortion.

Re:I can't think of a better argument... (2)

gsslay (807818) | about 3 months ago | (#47263397)

Why isn't this standard procedure for all data repositories?

Doesn't matter how efficient and secure you are, if one person can wipe absolutely everything from one control panel then you have a risk that is not being addressed. And one that isn't even difficult to address.

Re:I can't think of a better argument... (4, Interesting)

Charliemopps (1157495) | about 3 months ago | (#47263475)

for air gapped backups.

It has to be more than that. We had a policy of air gapped backups that everyone followed. But we had several different sites with several different admins. There was a large hurricane and we found some flaws in the system to say the least.

In several cases, the backups were kept IN the drive... they were gone.
In others, they removed the backups, put them on top of the server or in a desk draw.... gone as well.

In others, they actually removed the tapes from the site, but often they were taken home by the admin or other staff... in those cases we faired slightly better because both the site and the staffs house would have to be under water. Hurricanes are big however, so we had about a 50% success rate there.

In some cases they had a safe on site. This proved marginally better... the tapes were safe in most cases. In one instance we had a rather brave Admin fly across the country, take a cab out to the site and the literally SWIM to get the tape. But in a lot of cases the tape was OK, but the safe was under water. So we weren't able to retrieve it for days.

The sites where local admins stored the tapes at local banks faired the best. So now that's our policy. Backups get stored off-site, in a vault. Technology is better now so we also do remote backups across the net now as well in case the bank is under water as well. But no matter what, we can always head to the bank vault. Ok, I guess a meteor would ruin our day, but you cant plan for everything.

Re:I can't think of a better argument... (2)

nine-times (778537) | about 3 months ago | (#47263741)

There was a large hurricane and we found some flaws in the system to say the least.

That's why you have backups in different geographical areas.

The sites where local admins stored the tapes at local banks faired the best.

Have you considered a service like Iron Mountain? They'll send out a truck to pick up your backups every day, if you like, and store it in a very safe location.

Whoever pulled this off (4, Funny)

Anonymous Coward | about 3 months ago | (#47263213)

would you mind going into ebay.com & deleting my account?

Ebay refuses to close it.

Re:Whoever pulled this off (2)

sexconker (1179573) | about 3 months ago | (#47263371)

would you mind going into ebay.com & deleting my account?

Ebay refuses to close it.

Move to Europe and sue them under your new right to be forgotten.

Well that escalated quickly (2)

ACK!! (10229) | about 3 months ago | (#47263215)

At least they had backups of their cloud data in a safe place where no random asshat could just go in and waste the data. That is a code hosting company you can trust with your stuff that is for sure!

Well.... (0)

Anonymous Coward | about 3 months ago | (#47263217)

A back-up that can be deleted so easily is no back-up at all.

That's a disadvantage of the cloud (0)

Anonymous Coward | about 3 months ago | (#47263219)

Definitely a strong reminder to have at least some off cloud presence, unreal to think a hacker could ruin your business by stealing your thunder (cloud).

No offsite backups? (1)

cHALiTO (101461) | about 3 months ago | (#47263225)

They didn't have offline backups? tapes? I'm not familiar with codespaces service, but how come the backups could be deleted remotely?

Re:No offsite backups? (5, Insightful)

gstoddart (321705) | about 3 months ago | (#47263269)

No, because it was all in Amazon. Who needs tape when you have the cloud, right?

So the stuff they had backed up from Amazon to Amazon, was still controlled by the same logins (or the ones the hacker had created).

So when he/she/they started deleting stuff, the backups also got deleted.

Sounds like a brilliant strategy, and an epic demonstration of what can go wrong with the cloud.

If you host your own stuff, you do your own backups. If you backup your cloud data to the cloud using the same stuff as the rest of it ... well, your backups are hardly secure, are they.

So unless Amazon has offsite tape backups (which I highly doubt) ... they're pretty much screwed.

I think this is about the same as backing up your hard drive to itself so you have a spare copy.

Re:No offsite backups? (2)

Bengie (1121981) | about 3 months ago | (#47263373)

No, because it was all in Amazon. Who needs tape when you have the cloud, right?

A rule of thumb that I've heard was "It's not backed up until on at least 2 different media types, at least 2 different file systems, and stored in at least 2 different physical locations".

Re:No offsite backups? (1)

Anne Thwacks (531696) | about 3 months ago | (#47263541)

You have been short-changed.

If its worth money:

Hve three copies, on three media types in three locations.

Not so sure about file systems. If you have proprietry backup software, then you will never get the data when you really need it. tar loves you!

Re:No offsite backups? (0)

Anonymous Coward | about 3 months ago | (#47263773)

I'd add another requirement: to have checked that backup with an independent system. Because if your computer is encrypted with the Cryptolocker malware, then your backup has been encrypted too, as you were making it. And it would look perfectly normal until Cryptolocker deletes the private key on your machine to start the ransom process. And only then would you find your backup was unreadable. So you have to check your backup with a different machine entirely.

Re:No offsite backups? (1)

Anonymous Coward | about 3 months ago | (#47263403)

>

I think this is about the same as backing up your hard drive to itself so you have a spare copy.

This is the crux of the matter... they had backups meant for accidental delete events (like copying a file you edit over to file.orig just in case you fuck it up) but that is of absoultely zero use in a malicius delete event.

Re:No offsite backups? (4, Insightful)

Jeff Flanagan (2981883) | about 3 months ago | (#47263453)

>Sounds like a brilliant strategy, and an epic demonstration of what can go wrong with the cloud.

No, it's just an example of what can happen to incompetent people. There's no reason to believe that these people would not have also failed to have offline backup with local servers. There was nothing to prevent them from keeping backups locally or on another cloud.

Blaming cloud computing for this is completely idiotic, and about what I expect on the dumbed down Slashdot these days.

Re:No offsite backups? (0)

Anonymous Coward | about 3 months ago | (#47263639)

The reason the cloud is to blame is because it causes this sort of complacency. The cloud cures 99% (maybe even 99.9%) of problems at some fraction of the cost of achieving the same 99% yourself. The problem is if you actually account for the other 1% of problems the cloud does not fix, you generally are back at or above the cost of doing everything in house.

Financial cost benefit analysis of the cloud is being done on an apples to oranges basis and this does help demonstrate that. This is an example of what additional costs need to be considered when computing how much you will save by switching to the cloud. This case study demonstrates that you need frequent, air gapped, backups that are in your physical control.

Re:No offsite backups? (1)

anolisporcatus (969211) | about 3 months ago | (#47263667)

I agree, there always need to be multiple backups in multiple locations, especially if it is someone elses information.

Re:No offsite backups? (1)

nine-times (778537) | about 3 months ago | (#47263775)

I don't think you necessarily need to backup to tapes yourself. If you backed up your Amazon stuff to Rackspace, for example, you would be protected both against someone gaining access to your Amazon account, as well as a systemic problem with Amazon. Just so long as there's nothing in your Amazon account that would allow an attacker to access your Rackspace account, that should be a pretty good solution.

No solution is perfect. You're just looking for one that's extremely unlikely to break.

Re:No offsite backups? (1)

Threni (635302) | about 3 months ago | (#47263327)

You mean if you copy "file.txt" to "file-copy.txt" in the same folder you've not performed a backup? Wow! I learned something today!

I hope their customers get their money back! Or did the attackers copy "all our bank details.txt" as well?

MS (0)

Anonymous Coward | about 3 months ago | (#47263229)

ah a hotmail account

Like MS will not give up the IPs that accessed that account...

Someone is going to jail...

Re:MS (0)

Anonymous Coward | about 3 months ago | (#47263235)

You mean the IP of the pawned access point? It's probably your mother's.

Re:MS (1)

lucm (889690) | about 3 months ago | (#47263263)

That jail must be very crowded with all the nigerian scammers and fake craigslist landlords who use hotmail to scam people.

MS (0)

Anonymous Coward | about 3 months ago | (#47263275)

yes, because using a proxy to access hotmail is the most difficult thing ever...

Re:MS (1)

CheezburgerBrown . (3417019) | about 3 months ago | (#47263291)

Any self respecting bassmint dweller would not have used their home network to do this.

Re:MS (1)

sexconker (1179573) | about 3 months ago | (#47263433)

Any self respecting bassmint dweller would not have used their home network to do this.

Basement dwellers don't leave the basement.
Basement dwellers are self-loathing, not self-respecting.
Basement dwellers use their own network to connect to proxies, which just makes it more of a pain in the ass to trace back.
Extreme basement dwellers will use other means of accessing a separate network - a cantenna pointed at a neighbors house, a spliced line, whatever. This just means the cops track down the victim, figure out they're not computer literate, and ask "Any people who could have done this?" and learn about the freak in the neighbor's basement.

Hackers don't get caught because law enforcement doesn't care.
When the cops, the government, or a corporation cares, hackers get caught or disappeared.

Re:MS (0)

Anonymous Coward | about 3 months ago | (#47263655)

You would think that, but I've seen a surprising number of DoS type attacks that come from a single IP address, once that IP address gets blocked as a simple fix, they repeat the DoS from some proxy and/or try something more advanced. In a couple cases where the original IP address was from a university campus, their IT department is not too happy and quickly finds both original DoS and a connection to the proxies coming from a single dorm room...

Re:MS (0)

Anonymous Coward | about 3 months ago | (#47263305)

ah a hotmail account

Like MS will not give up the IPs that accessed that account...

Someone is going to jail...

Unfortunately the person going to jail won't be the attacker because the police will not make the case a priority. There is no child exploitation, terrorism claim, or national security interest at stake. By lo-and-behold the police will will drop a bomb on anyone trying to release pay-walled research papers. The cloud company should request the IP address of the person's email account and initiate a bloodbath civil litigation case which seeks not only restitution but will ensure the attacker never touches anything again after the court orders his hands and eyes removed. Justice frontier style.

Re: MS (0)

Anonymous Coward | about 3 months ago | (#47263405)

I doubt very much the ip adress hotmail has on file, or those present in mail headers means anyting. Unless the greedy skript kiddie was completely kluless, she would have accessed hotmail from a proxy, via a compromized innocent third party, and after that wiped the system of that third party. Of course they need to look at it, but the likelyhood of finding the pasty faced juut is small.

EC2 (0)

Anonymous Coward | about 3 months ago | (#47263271)

I still don't get the logic of running your business in the cloud as a company. Sure it makes sense when your small and it gives you global presence with no investment, but people don't realize how much security / freedom they give up doing this. and EC2 ensures it will cost a lot to move out of their space. Good luck with the founders of code space. Glad I didn't put my repo there.

Re:EC2 (1)

Richy_T (111409) | about 3 months ago | (#47263365)

The trade-offs can be really good even for a large company. It has to be done right though and many companies don't even do their local IT properly.

If your operation is compromised, shut it down (4, Insightful)

Anonymous Coward | about 3 months ago | (#47263281)

The guys behind Code Spaces should be issued a citation for Operating While Pwned. If you know admin access is compromised, shut it down out-of-band.

A Cloud backup of Cloud data is not a backup (0)

Anonymous Coward | about 3 months ago | (#47263297)

I assume that this is probably becoming a relatively common practice, but, to me, if it is not reliably written on offline physical media of which I have control, it is not a backup.

So what to do about it (1)

bugs2squash (1132591) | about 3 months ago | (#47263307)

Presumably when they realized that the attacker had access to their control panel they shoulda coulda (yes I know I hate that too) called Amazon and had them shut everything down until order could be restored.

backups deleted? (0)

Anonymous Coward | about 3 months ago | (#47263313)

Yeah, that is what I was wondering. Did the hosting company have offline backups? Too expensive to implement? Too time consuming to copy gigabytes of data to an off-line storage disk? Just asking.

The dog ate my homework. (3, Insightful)

Thanshin (1188877) | about 3 months ago | (#47263321)

I must be a cynic but my first reaction is to think:

1 - Create cloud based system.
2 - Sell subscriptions for hundreds of $.
3 - Announce hacker attack!
4 - Profit.

Re:The dog ate my homework. (0)

Anonymous Coward | about 3 months ago | (#47263447)

I must be a cynic but my first reaction is to think:

1 - Create cloud based system.
2 - Sell subscriptions for hundreds of $.
3 - Announce hacker attack!
4 - Profit.

Yeah but you are forgetting

5 - The law firm of Lawyer, Lawyer, and Lawyer.

Not a Great Response (5, Insightful)

Edrick (590522) | about 3 months ago | (#47263337)

If you're a hosted site with important data and your site is compromised, the first & best move is to cut the cord immediately. Contact Amazon (or whomever is hosting the data) and get all access shut down instantly and immediately, thereby ending the attacker's ability to do anything further. This will cause an outage, but at least everything is safe.

Working with Amazon, they can create a new account, give it a strong password, and begin cleaning up the mess with the new account (which the hacker will be unaware of). Now they can, at their own leisure, change passwords, administer accounts, delete crap created by the hacker, etc...Trying to outpace a professional hacker at their own game is a gamble that isn't worth it---especially if no offsite backups exist!!!

Lastly, they should be forwarding all of the email/attacker info to Amazon, Microsoft (Hotmail), and to the authorities. Whether they can be caught or not is up in the air, but odds are almost certain that this attacker has hit other sites and would eventually have different cases correlated to each other.

Safety & security of data is #1, fixing damage caused is #2, and accountability is #3. Securing the site against future attacks is part of #3---there's no reason to put the site up (or leave it up) and risk further attacks, thereby risking data loss or a security breach.

Re:Not a Great Response (2)

jader3rd (2222716) | about 3 months ago | (#47263539)

Contact Amazon (or whomever is hosting the data) and get all access shut down instantly and immediately, thereby ending the attacker's ability to do anything further.

But what if the attacker is the one contacting Amazon to shutdown everything? Do you want your business shut down by random teenagers calling Amazon, telling them to shut everything down?

Re:Not a Great Response (4, Insightful)

Nemyst (1383049) | about 3 months ago | (#47263779)

If the attacker has access to the financial details used by the company to pay for the hosting, which is generally how you can authenticate people safely, you have much bigger problems.

Re:Not a Great Response (0)

Anonymous Coward | about 3 months ago | (#47263705)

If you're a hosted site with important data and your site is compromised, the first & best move is to cut the cord immediately. Contact Amazon (or whomever is hosting the data) and get all access shut down instantly and immediately, thereby ending the attacker's ability to do anything further.

How do you propose Amazon distinguishes between the owners and the hacker impersonating them, once the hacker has obtained their logins and passwords?

Re:Not a Great Response (0)

Anonymous Coward | about 3 months ago | (#47263793)

Hotmail doesn't give two shits what people did with their service. Neither does google. That's why people use them for throwaway email accounts to harass people on the internet.

shut down immediately and lock up (1)

stenvar (2789879) | about 3 months ago | (#47263341)

If someone has penetrated your system so that they have root or admin privileges over all your machine, you shut down immediately. In the physical world, you pull the plug. On Amazon, you immediately tell Amazon to lock things down, disable all passwords and administrative control, and then work back up to fixing things.

Re:shut down immediately and lock up (1)

tlhIngan (30335) | about 3 months ago | (#47263673)

If someone has penetrated your system so that they have root or admin privileges over all your machine, you shut down immediately. In the physical world, you pull the plug. On Amazon, you immediately tell Amazon to lock things down, disable all passwords and administrative control, and then work back up to fixing things.

But that's so 20th century! I mean, in the 21st century, if you can't do everything yourself without having to deal with another human being, then it's broken! Interacting with other humans is so... icky.

Ahahahahaha. (0)

Anonymous Coward | about 3 months ago | (#47263377)

I arrived on the Internet in 1994 and took a part in developing a couple well-known web sites. I loved the idea of a network empowering individuals to both control and share their data rather than relying on some big mainframe company. "Peer-to-peer" wasn't just a technical achievement, but a social achievement.

As "the cloud" emerged, I decided to pack my shit up and move to another career. I don't even begin to understand how billions of dollars are invested into this retrograde leap.

Facking Idiots (4, Interesting)

l0ungeb0y (442022) | about 3 months ago | (#47263393)

Not providing for your own OFFLINE BACKUPS is a reckless oversight of such magnitude that I am entirely incapable of having sympathy for these asshats. We need a few examples such as these to serve as cautionary tales for those who think the Cloud is the answer to everything.

Re:Facking Idiots (2)

iggymanz (596061) | about 3 months ago | (#47263499)

nothing to do with being cloud based or not, just proper attention to good systems operations practices was lacking.

even not doing the obvious and blocking all newly created accounts after certain time is just incredibly irresponsible.

Re:Facking Idiots (1)

locotx (559059) | about 3 months ago | (#47263653)

Well there is something wrong when people believe "the cloud" is the solution. It's a misinterpretation of a concept applied. I think the marketing push for "cloud" services being sold as a end all solution for backups, security and data storage gives off the feeling from the early 2000's where websites were being sold for all the things they could deliver, which they didn't. So to say it has nothing to do with "cloud based", I agree from the technical side, but i disagree from the "cloud" concept and marketing pitch side.

Git (5, Interesting)

blackiner (2787381) | about 3 months ago | (#47263429)

This is why git is such an effective code hosting solution. Everyone who has cloned the repository is a potential backup copy.

wrong order? (1)

roc97007 (608802) | about 3 months ago | (#47263503)

Someone else mentioned having offline backups, so I won't belabor that. But once they knew they were compromised, perhaps a smarter thing to do would have been to contact the service provider and take countermeasures (ask for a snapshot of the site as it was, examine and disable accounts, change admin passwords, perhaps contact authorities) before reaching out to the perp. I'm not sure reaching out to the perp was a good idea in any case.

For awhile I hosted a number of websites from a rental space, and I did get compromised once. (security hole in a popular web admin tool) As soon as I detected it, I drove to the physical site, unplugged the server from the internet, and worked from the console. It occurs to me that this might be a difficult strategy to implement with cloud services.

backup training (1)

tommyatomic (924744) | about 3 months ago | (#47263527)

So these guys apparently had no training on proper backup policies and procedures.

This is definitely a training issue. Clearly no one taught them how to do proper backups or even what a proper backup policy should look like.

I feel bad for them, but at the point that they have done nothing to protect themselves I cannot bring myself to feel too bad.

Why does no one take their backups offsite anymore or backup to a NAS device that backs itself up to something that can be taken offsite?

Backups Backups BACKUPS!!!

wtf... stuff evaporated! (0)

Anonymous Coward | about 3 months ago | (#47263529)

looks like someone got fired and was pissed!

i bet they didn't read http://docs.aws.amazon.com/IAM/latest/UserGuide/IAMBestPractices.html when creating their accounts, so pissed ppl still had access ....

Now things have "gone away" .... go figure ....

ofc, this may or may not have been the problem.

RTFM (0)

Anonymous Coward | about 3 months ago | (#47263565)

http://docs.aws.amazon.com/IAM/latest/UserGuide/IAMBestPractices.html#lock-away-credentials

Contact Amazon to lock entire account (0)

Anonymous Coward | about 3 months ago | (#47263583)

Why didn't they first contact Amazon to lock everything down and reset all the passwords?

Also, STOP USING THE SAME PASSWORD FOR EVERY WEBSITE

No offline backups? (1)

krelvin (771644) | about 3 months ago | (#47263589)

Seriously.... no offline backups? Not a real business in that case.

Could Amazon have handled it better? (1)

Max Threshold (540114) | about 3 months ago | (#47263611)

Instead of trying to take back control themselves, shouldn't they have contacted Amazon and let them handle it? Perhaps they could have frozen the entire account, locking out both the rightful owner and the attacker, until things were sorted.

We have Bunnie. Gather one million dollars... (1)

steak (145650) | about 3 months ago | (#47263675)

This is a bummer, man.

Additional Evidence (0)

Anonymous Coward | about 3 months ago | (#47263691)

Is there additional evidence that this is what happened or is the only narrative the one on Code Space's homepage?

Use Git (1)

stewsters (1406737) | about 3 months ago | (#47263697)

This is why distributed version control is important (git/mercurial), even if you think SVN is easier. Sometimes your remote server will disappear, whether its hackers, fires, or someone forgot to pay the bill.

Competitors? (0)

Anonymous Coward | about 3 months ago | (#47263701)

What other services can former customers go to for SVN and issue tracking services?

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?