Predicting Malicious Web Attacks 82
KentuckyFC writes "Recommendation systems attempt to guess what books, movies, or news people are likely to be interested in. Companies such as Amazon, Google, and Netflix have developed algorithms to mine vast databases looking for correlations that they then use to recommend new items. Now a team of computer scientists has used some of the same filtering techniques to predict the origin of malicious Web attacks so that they can be blacklisted in advance. The team mined a database of hundreds of millions of security logs looking for correlations between victims. The correlations were then used to produce a predictive blacklist of potential attackers. The team says its algorithm is up to 70 per cent more successful at predicting the origin of attacks than current state-of-the-art predictive blacklisting."
What we really need. (Score:1, Offtopic)
Hiro Protagonist. And his sword. And his undefeatedness-nous.
No doubt useful (Score:4, Insightful)
But this is still treating the symptom as opposed to the core problem, which is poor security in OS and app design.
Microsoft is starting to come around on this to an extent (not running as administrator), but shouldn't we be more concerned about true security?
Re:No doubt useful (Score:4, Insightful)
Re: (Score:2)
Perhaps I should included my title in my post. No doubt this useful given the current situation. But we wouldn't be in this position so much if we had well designed systems in place from day 1.
I do think this is interesting how we can use massive data sets to predict and map trends so much quicker. But I'd rather not have to worry about them in the first place.
Re: (Score:3, Insightful)
Re: (Score:2)
Unix was designed from day 1 with the notion that it is a multi-user system that needs serious integrated security. Windows was designed for a home PC with a single user. It wasn't designed with the notion that it would be on the internet, or need much in the way of security.
It isn't to say that we couldn't have forseen security concerns to design it correctly in the first place. Most *nix systems were always designed this way. Windows opted not to follow that model.
Choosing Security vs. Dancing Pigs vs. Unix (Score:2)
Bruce Schneier [schneierfacts.com] says that give a choice between security and dancing pigs on your computer, people will take the dancing pigs every time.
When Windows came out, it was perfectly secure - there's only one user in the universe, and she's allowed to do whatever she wants. ("Format C: "? Sure!).
Unfortunately, while Unix was designed from the beginning for security, it didn't always _stay_ designed for security, and some of the things that were done for security had serious tradeoffs. Networking was usually th
Re: (Score:2)
In truth, all you need to do is read the Art of War, and you'll know that implementing proper Windows permissions couldn't possibly the the answer to security. You'll also realize that collaborative filtering couldn't possibly the the answer to security either.
The only answer is to be one step ahead of the attackers, and to think up what they're going to throw at you next.
(That's not to say that proper Windows permissions don't help, and that collaborative filtering doesn't help, but security is war, and th
Re:No doubt useful (Score:5, Insightful)
but shouldn't we be more concerned about true security?
What is "true security" against the main threat of the modern era: social engineering? How does your operating system protect you from from responding to that e-mail you've just received from your long lost uncle in Nigeria? How do you protect a user that will click on the user account control pop-up as many times as is required to install that cool "weather forecasting" program that sits in his task tray?
Or were you referring to "true security" in the context of firearms, expendable redshirts and moats filled with laser wielding sharks? ;)
Re: (Score:2)
Don't underestimate sharks with friggen laser beams!
I agree that Social Engineering is likely the number one threat in many cases.
UAC is security theater in that people are trained to simply click allow, absolving Microsoft of responsibility.
What I mean by true security is sandboxing and accountability. Look at Chrome's design, in that a browser window (process) has limited access to data on your HDD.
Users in an enterprise environment frankly shouldn't have access to install software at all. And the more I
Re: (Score:2)
Users in an enterprise environment frankly shouldn't have access to install software at all.
Which leads us to the true security question/issue. The only truly secure system is one users don't have access to. In any other environment, where people are trying to get work done. a completely locked down environment can impede the business. The end goal, whether the security types like it or not, isn't a secure environment. It's to make money or reach some other objective. Security is relevant in that it sup
Re: (Score:3, Interesting)
Re: (Score:2)
As long as I can get the user to visit my site and load up my malware I can spew spam, I can DDOS, etc.
Not true. If neither the user nor app have admin/root access, and you're using a secure browser (say, Chrome) then your malicious web site can't do squat. The biggest hole here right now would be that plugins aren't fully sandboxed, and Acrobat has a serious vulnerability every other week. But that is partially why I keep recommending to businesses to use Foxit as opposed to Acrobat.
Re: (Score:2)
Re: (Score:2)
Every browser uses javascript and plugins. Saying that Chrome isn't secure because of these things is silly.
Chrome places each process in a jail and prohibits access to the HDD to make changes to your system.
Will there be Acrobat exploits that can also be accessed via Chome? Yes, until Chrome figures out how to fully sandbox plugins, but Google said they are working with plugin vendors to make them play nice within Chrome's security concepts. Chrome is still more secure than IE and Firefox, not just because
Re: (Score:2)
sers in an enterprise environment frankly shouldn't have access to install software at all.
Unfortunately it's rarely that simple. I've worked in two "enterprise" environments in my IT career. One (my current job) makes this fairly easy to implement -- most of our operations run around web based database apps and Office. Very easy to lock users into restricted accounts.
The other enterprise I worked for was an insurance agency. The insurance industry has so much legacy software that restricting users to non-admin accounts is not possible unless you are willing to sacrifice needed functionality
Re: (Score:2)
Both of the two enterprise environments I've worked in have used proprietary legacy apps that "need" admin rights.
Most of the time, all the app really needs is write access to a certain folder. However, in the rare instance that the process truly does need administrator access, I make the app/process into a Windows service that starts automatically at login with System level access. The user doesn't have admin access, and other apps don't. That one app is elevated.
Re: (Score:3, Interesting)
What is "true security" against the main threat of the modern era: social engineering?
Social engineering will always be a problem but there is a simple fix. Restrict the user on damage they can do on their own given the worse case circumstances and you will also end up with the same prevention of malware in the process.
Speaking of which... Why does a web page ever need to communicate with the OS to make file changes to the OS? Why?! Why I ask?!
This is a flawed premise and will solve 99% of the problems we f
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I think there is a company willing to do that. They're in Redmond, Washington.
Re: (Score:2)
Re: (Score:2)
" "Well I want a pony!" you reply "But you'll just have to deal with a limited OS because we can't have nice things because they keep installing viruses on their machine!" "
Thank you, you've just made Jonathan Zittrain's point exactly [futureoftheinternet.org].
Except he thinks this is a bad development and can still be changed.
Re: (Score:1)
Require the "weather forecasting" app to submit an approval to a central repository like the iPhone.
See where I am leading you...
Yes, and it is a bad idea. Secure the OS by securing the OS, not by adding in a random trusted third-party that will probably make mistakes anyway (maybe we should call that "security by authority"?). Sandbox applications so they only have access to the files and services they need, perhaps with permissions like "safe" network access which is capped or can only access one server or port or has to display the bandwidth used on screen and be advertized as a possibly dangerous high-network usage application (e
What the military means by "Secure a computer" (Score:2)
I used to work with an ex-Navy guy - our lab became much neater once he joined us, and more secure as well. But different organizations have much different concepts of what it means to "secure a computer" -
Re:No doubt useful (Score:4, Insightful)
"True security" is a fantasy. No such thing exists, nor will it ever.
We should be concerned with balancing risk reduction with its cost. We should not be concerned with your silly fantasy.
Re: (Score:2)
Designing the OS to be secure as opposed to chasing people attacking vulnerabilities left by design in the OS is silly?
Re: (Score:2)
I truly agree, bad OS design, some lack of security based on too much money it would cost, and not enough people really taking security seriously, there will always be that one person with a password equal to a dogs name or their birthday!!!
Minority Report (Score:2, Interesting)
What about false positives? Can they be held responsible for blacklisting an innocent site?
Re: (Score:2)
You can't be held responsible for blacklisting sites right now, what makes this any different from any other Blacklist?
If you want to get traffic to/from the site then Whitelist it.
Now they say their approach is 70% better than existing Predictive Blacklist technology, well how good is that, 70% better than horrible false positives and annoyed customers is not enough. Throwing darts at the DNS listings is also not optimal, so how good is this new technique.
BTW Amazon and Netflix recommend crap to me I don't
Re: (Score:1)
Most people don't understand the internet. I'm not sure how this blacklisting thing would be deployed, but your average person is just going to accept it and move on. Now, what if it blacklists something like Amazon or eBay? Would Amazon be able to sue someone over lost revenue because all the Joe I. Pod's out there stopped visiting their site all of a sudden? Especially if it was just a false positive.
As an afterthought, is there really a distinction between a false positi
Re: (Score:1)
Hello, I am Anh Le, the second author of the work.
First, investigating the false positive is not the main focus of our work. We did our analysis on the log entries generated by the intrusion detection systems (IDS) deployed at various sites. Granted that there are false positives in the dataset, these false positives, however, are from the IDSs because of, for example, bad signatures and configuration errors. This is itself an area of active research.
Furthermore, the entries included in the blacklist appear
Finally a use for this technology (Score:3, Funny)
There's finally a use for this collaborative filtering technology.
Re: (Score:2)
Well, according to TFA it's not quite ready for prime time.
Re: (Score:1)
Re: (Score:2)
If it doesn't, they won't have much of a website left after today.
Re: (Score:2)
You may joke about it, but I wrote a "slashdotted" snort rule for a web development and hosting company.
Did I read this right... (Score:5, Interesting)
recommendation systems may soon be providing you not only with books and movie tips but a happier surfing experience too
I am a little weary of making my surfing experience happier by allowing the system to do my thinking for me. Just think, "clippy" for the browser.
Re: (Score:1)
weary (tired of) or wary (nervous about)?
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
sarcasm on
Yeah, and I'm pissed that I can't get Clippy working on Ubuntu!! The little dog in the search box too!! What, when you start to go geeky you can't have pets anymore? It's just WRONG, I tell you!! I'm going to send some hate mail to Canonical, and find out what the deal is. This just pisses me off!!
sarcasm off
Seriously - all those user agents and stuff should have been a tipoff. A corporation that offers cartoonish characters as part of a "serious" operating system can't be trusted with secur
Re: (Score:2)
recommendation systems may soon be providing you not only with books and movie tips but a happier surfing experience too
I am a little weary of making my surfing experience happier by allowing the system to do my thinking for me. Just think, "clippy" for the browser.
The article doesn't seem to say how it will be implemented, but I would assume it would be some server-side app that generates firewall (and possibly spam filter) rules.
Hoory Preemptive Blacklisting! (Score:1)
Umm... (Score:2, Funny)
Re: (Score:2)
That, or go Skynet. The ideal way to stop all web attacks would be to bring down the internet itself. I so hope these guys did their homework [xkcd.com].
the new 404 (Score:3, Funny)
"People..." (Score:5, Funny)
"People who attacked this site ALSO attacked..."
Re: (Score:3, Funny)
"Was this review helpful? Yes or no"
Re: (Score:1)
Amazon should patent "1-click attacking"
Re: (Score:3, Informative)
Amazon should patent "1-click attacking"
Ptech [google.com] already has it patented!
Re: (Score:2)
We used to get targetted many, many moons ago by people searching Google for "phpBB version x.y.z". If you want to predict web attacks, Google says:
Results 1 - 20 of about 80,600,000 for "phpbb version x.y.z"
Re: (Score:2)
a "coorelationisnotcausation" tag
Thanks, I knew that there was a perfect tag for this story! Marking it as such allows two benefits which I can easily define:
The Article is obviously a fake (Score:4, Insightful)
Or greatly exaggerated...
"The team mined a database of hundreds of millions of security logs"
Nobody actually keeps security logs, certainly not hundreds of millions of somebodies.
The kind of people that DO keep security logs probably wouldn't hand them over either.
I call shenanigans
Re: (Score:2)
Obviously they have developed hacking technology to break open all these systems to get at their logs to determain if they have been hacked. Well they will be blacklisting themselves later this afternoon.
Re: (Score:3, Insightful)
Fixed:
Now it makes more sense, and is quite believable, no?
Re: (Score:1)
False positives (Score:2)
False positives, here we come...
Meatware needed (Score:5, Insightful)
This sounds great, but only if it requires human intervention to implement the block. I used to work in a NOC, and we would have loved to throw up a warning on the big screens that an attack is 80% likely from the following netblocks in the next N hours. That way we would have a strategy developed for defending before it even started and would be able to minimize downtime.
On the other hand, if you make this automatic you're going to piss off a lot of people very quickly because it's going to be wrong more often than you want.
Re: (Score:2, Insightful)
Exactly. Because even if it's true, and it's 70% more accurate... I've yet to see a predictive system that's even remotely accurate. It may predict say... 50% of the sources of an ongoing attack (assuming a collaborative effort to determine when attacks are happening, and that you're not the first one hit), but that's far from enough to prevent a DDoS attack. And if you "accidentally" block... Say Canada (which I've seen before), then that's a LOT of costumers you just pissed off, but hey... Doesn't matter,
False positives (Score:2)
Re: (Score:1)
Hello, I am Anh Le, the second author of the work.
I responded to the concern about false positives in one of the replies above. In brief, investigating the false positives is not the main focus of our work, and it is an area of active research in the intrusion detection system community.
Link to our paper: http://arxiv.org/abs/0908.2007 [arxiv.org] [arxiv.org]
The Minority Report (Score:1)
Okay if your going to do this anyway, here let me gaze into my crystal ball. Blacklist China, North Korea, and major parts of Russia.
Predictorator (Score:2)
Calculate the annoyance factor
If site is shitty, + .1
If site has a "clever" name, such as bit.ly, +.1
If site's name has become widely used as a verb or other part of speech, +.1
+ unique users in the last 24 hours / 100,000,000
Calculate the monetary factor
If site sells something, +.05
If site makes revenue through ads, +.05
If site is partnered or associated with a megacorp like a bank, ms/google, etc., +.1
+ dollars lost per minute of downtime (based on the last 24 hours) / 1,000,000
Calculate the brought it up
Never assume anything (Score:1)
Sounds vageule familiar... (Score:2)
They *guess* that you may be guilty before it happens and blacklist you.
Does it also predict its own false positives? (Score:2)
Great, so it can "predict" IP or site origins of malicious attacks, but can it also predict its own inevitable false positives? If so, how is it better than a DNSBL or other blacklist, except that it can make money for its owners without requiring constant updating and the requisite human labor?
I'd hate to use an IP or own a site that it happened to incorrectly "predict" as the source of an impending-but-as-yet-not-real attack. They might as well compile a Minority Report against me. How would that be an
sidreporter? (Score:2)