×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

NSA To Use Cloud Model For Intelligence Analysis

Soulskill posted more than 4 years ago | from the have-fun-securing-that dept.

Government 41

Hugh Pickens writes "Information Week reports that the National Security Agency is taking a cloud computing approach in developing a new collaborative intelligence gathering system that will link disparate intelligence databases geographically distributed in data centers around the country. The system will house streaming data, unstructured text, large files, and other forms of intelligence data, and analysts will be able to add metadata and tags that, among other things, designate how securely information is to be handled and how widely it gets disseminated. For end users, the system will come with search, discovery, collaboration, correlation, and analysis tools. The intelligence agency is using the Hadoop file system, an implementation of Google's MapReduce parallel processing system, to make it easier to 'rapidly reconfigure data' and for Hadoop's ability to scale. The NSA's decision to use cloud computing technologies isn't about cutting costs or seeking innovation for innovation's sake; rather, cloud computing is seen as a way to enable new scenarios and unprecedented scalability. 'The object is to do things that were essentially impossible before,' says Randy Garrett, director of technology for NSA's integrated intelligence program."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

41 comments

8==C=O=C=K==S=L=A=P==D ~~-_ (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#28782653)

thunk.

Money well spent? (1)

siloko (1133863) | more than 4 years ago | (#28782711)

When will people realise than having more data often makes it more difficult to find the needle in the haystack. I am all up for utilising raw power in distributed networks to gain insights into patterns previously hidden but given the sheer breadth of proposed data formats to be mined this effort seems doomed to producing no tangible results at all. Except the ongoing expenditure of tax dollars.

Re:Money well spent? (1)

vertinox (846076) | more than 4 years ago | (#28783103)

When will people realise than having more data often makes it more difficult to find the needle in the haystack.

The goal of government data collection isn't to find a needle in the haystack.

The goal is to turn more hay into needles.

Now how many crimes have you uknowlingly broke today, citizen?

Re:Money well spent? (1)

MindKata (957167) | more than 4 years ago | (#28783635)

"Now how many crimes have you unknowingly broke today, citizen"

Well you've just committed one crime, you've shown decent, citizen, by joking about Big Brother ;). So we need to add you to more of our lists. (After all, we need to know who to round up first, when we next want to distract political opponents).

"more data often makes it more difficult to find the needle in the haystack"

The more haystacks they look through the more needles they can find. It'll just cost a lot more to build such a big system. Luckily money isn't such a worry for them. (For them that is, as they don't end up paying for it! ;) ... Ironic really, they use our money to spy on all of us.

As for tangible results, that's a very interesting question. Who exactly is this huge and ever growing distributed system going to data mine? ... I'm sure the idea of it is sold to everyone on the basis its all part of the war on fear, I mean war on terrorists (hiding on every street corner), but that's the point. Is this directed internally as well as externally?

We all know knowledge is power but with a system of such growing knowledge, what safe guards are in place to prevent greedy people in power exploiting this much growing knowledge (and power) for their own gain?

Because each time some more liberty is taken from us all, someone else (i.e. the people in power and their friends with the money) ends up profiting and gaining personally from our loss of yet more liberty.

In which case, Benjamin Franklin, must be spinning in his grave by now, i.e. "Those who would give up Essential Liberty to purchase a little Temporary Safety, deserve neither Liberty nor Safety." ... (Its amazing he said this over 200 years ago, yet still we make the same mistakes).

Re:Money well spent? (1)

WindowlessView (703773) | more than 4 years ago | (#28785247)

Needles. Haystacks. How is it that it every government endeavor except intelligence agencies someone asks "Hey, exactly what is the cost per needle found in those haystacks?"

Why is there no commission that meets twice a year and announces to the public: we found 8 terrorists, killed 3. It cost 160 billion dollars. The commission should be composed of people the public knows and trusts. They can have their backgrounds examined by the agencies. They should give out as much information as possible w/o putting people or procedures at risk. But give the people who are paying for all some fucking idea of the efficacy of the operation. Or perhaps that is what they are really afraid of?

Re:Money well spent? (1)

salimma (115327) | more than 4 years ago | (#28783221)

You could say the same thing about search engines back in the mid-90s, before Google's PageRank. What's the point of indexing more and more information if it's impossible to find the relevant page? Necessity is the mother of all inventions. The only regret here is that whatever they come up with, probably won't see daylight until someone outside reimplements it.

Re:Money well spent? (2, Insightful)

siloko (1133863) | more than 4 years ago | (#28783689)

You could say the same thing about search engines back in the mid-90s, before Google's PageRank.

I couldn't agree more however the ramifications of inaccurate or misleading results are minimal - to the user if not Google's bottom line. What is at issue here is collating a resource from disparate sources, including vastly different formats, which would enable relevant agencies to better sort the wheat from the chaff. Having been part of a team aiming to standardise similar data sets to provide a search resource I speak from experience when I say this is no simple undertaking. Slashdot has recently [slashdot.org] posted an article on the difficulties of trawling datasets with tools which inevitably produce false positives, and the ramifications on false positives for the people in question are onerous to say the least. And this is without factoring in the liklihood that no more personnel will be available to follow up flagged content (people) so the net result is either an unused collection of positives or an accross the board policy change to haul in outliers under the pretense that they have been highlighted by 'powerful new technology'.

Re:Money well spent? (1)

salimma (115327) | more than 4 years ago | (#28785249)

I guess it really depends on the details. The two bottlenecks are certainly worrisome: 1) the need for analysts familiar with both the system, and domain experts, to classify the data; 2) that the data is made available to a a wide range of users [wikipedia.org] .

Let's hope the trial is realistic enough to bring up potential problems before real people get pulled in because of an overreliance on technology...

Re:Money well spent? (1)

Bakkster (1529253) | more than 4 years ago | (#28783223)

"The system will house streaming data, unstructured text, large files, and other forms of intelligence data, and analysts will be able to add metadata and tags that, among other things, designate how securely information is to be handled and how widely it gets disseminated. For end users, the system will come with search, discovery, collaboration, correlation, and analysis tools."

More data is only bad if the signal to noise ratio doesn't also improve an equivalent amount. It doesn't matter if you've got 4x more hay to search through if you also can search the hay 4x (or more) faster. You might even find 4x more needles for the same effort.

Re:Money well spent? (0)

Anonymous Coward | more than 4 years ago | (#28783555)

Siloko - It has produced tangible results in its original incarnation with a single data type - the question/bet Mr. Garrett is making is whether it will scale out and be able to handle much larger data files than Hadoop was originally built for. I hope he is hedging his bet.

Centralized computing/storage (1)

Dareth (47614) | more than 4 years ago | (#28782743)

Do the "Cloud Vendors" sell big cloud stickers to put over the data center portion of the design diagrams to hide the details from management?

This sounds like centralized computing and storage on dedicated servers. They are not going to buy a slice of some public cloud computing infrastructure.

   

Distributed == cloud? (4, Insightful)

Red Flayer (890720) | more than 4 years ago | (#28782793)

If in the cloud
The data you keep
The NSA
May get a peek.

Burma shave.

But seriously... Distributed data is now "the cloud"? Is my dirty laundry in "the cloud" because it is scattered in my bedroom?

Tags For Cheney Intercepts: +1, PatRIOTic (0)

Anonymous Coward | more than 4 years ago | (#28782855)

from his Naval Observatory spider-hole [harryshearer.com] .

Yours In Espionage,
K. Trout

Cloud this cloud that (4, Insightful)

croddy (659025) | more than 4 years ago | (#28782867)

Listen, shut up about the "clouds" already. Just because you don't understand the architecture doesn't make it a "cloud". That word doesn't mean anything. I'm sick to death of hearing it already.

Re:Cloud this cloud that (1)

Em Emalb (452530) | more than 4 years ago | (#28783041)

who are you talking to? Hate to tell you, Mr Preacherman, but the congregation is thataway. We're the choir.

Re:Cloud this cloud that (1)

thebian (1218280) | more than 4 years ago | (#28783105)

+1

And while we're at it, let's bury map-reduce, too.

Re:Cloud this cloud that (0)

Anonymous Coward | more than 4 years ago | (#28783769)

+1

And while we're at it, let's bury map-reduce, too.

Map-reduce actually has a specific meaning.

Re:Cloud this cloud that (0)

Anonymous Coward | more than 4 years ago | (#28787315)

Isn't that called something along "divide et impera" ?

Re:Cloud this cloud that (1)

El Torico (732160) | more than 4 years ago | (#28783123)

Don't worry, if the Government wants to do it, then it's no longer hip. I expect the phrase "cloud computing" to go the way of "paradigm shift" and "mindshare" soon.
Too bad the NSA will piss away a few billion before they realize they aren't cool.

Re:Cloud this cloud that (0)

Anonymous Coward | more than 4 years ago | (#28783369)

Everything are clouds these days. Slashdot is also about cloud computing. I had just defined it and hence it's true.

Obligatory (0)

Anonymous Coward | more than 4 years ago | (#28783397)

"Old Man Yells at Cloud"

http://i274.photobucket.com/albums/jj254/dear_die_hard/grandpa_simpson_yelling_at_cloud.jpg

Re:Cloud this cloud that (0)

Anonymous Coward | more than 4 years ago | (#28785359)

I like how you think the rest of the world gives a shit what you're tired of. I'm tired of morons but you're still breathing.

The Stasi, revisited . . . (1, Informative)

PolygamousRanchKid (1290638) | more than 4 years ago | (#28782875)

. . . the Stasi http://en.wikipedia.org/wiki/Stasi [wikipedia.org] , the former East Germany's secret police, ended up collecting so much information on folks, that they couldn't process it all.

From the Wikipedia article: "When informants were included, the Stasi had one spy per 66 citizens of East Germany.[8] When part-time informer adults were included, the figures reach approximately one spy per 6.5 citizens."

Yo.

I guess the NSA thinks that they can do better than the Stasi with brute force computing power.

Re:The Stasi, revisited . . . (1)

gknoy (899301) | more than 4 years ago | (#28783671)

Brute force computing power has certainly advanced a lot since the Stasi. Look at what Google does with a large volume of cheap hardware.

Re:The Stasi, revisited . . . (1)

DNS-and-BIND (461968) | more than 4 years ago | (#28783877)

I swear to God, there has to be some extension of Godwin's Law for every time someone brings up the Stasi. What is the ratio of NSA informers to the American population?

Re:The Stasi, revisited . . . (1)

Bakkster (1529253) | more than 4 years ago | (#28784243)

I swear to God, there has to be some extension of Godwin's Law for every time someone brings up the Stasi. What is the ratio of NSA informers to the American population?

Exactly. Especially since the NSA is not (legally) allowed to analyze "signals" from US citizens, and the vast majority of their employees care about following that rule.

Re:The Stasi, revisited . . . (0)

Anonymous Coward | more than 4 years ago | (#28785171)

Cold comfort when an NSA imp who doesn't care about that rule has targeted you.

Re:The Stasi, revisited . . . (0)

Anonymous Coward | more than 4 years ago | (#28785449)

What is the ratio of NSA informers to the American population?

I don't know (obviously), as a German, I'll note that the reason the Stasi was evil was not that it had a lot of informers.

Rather, the reason it was evil that it did, well, evil things. The large number of informers merely allowed them to do that, and the GP's point is actually that the NSA does NOT need as many informers anymore now to do the same things the Stasi did.

Whether that means the NSA and the Stasi are comparable is another question entirely, though.

Re:The Stasi, revisited . . . (0)

Anonymous Coward | more than 4 years ago | (#28785775)

I swear to God, there has to be some extension of Godwin's Law for every time someone brings up the Stasi. What is the ratio of NSA informers to the American population?

Measured in humans, infinitesimal.

Measured in megabytes of data and megahertz of processing power available, maybe not so small. There are only 300 megapeople to monitor, and we're talking about an organization that measures the sizes of its server rooms in acres.

Analysed Use of Intelligent Models on a Cloud NSFW (1)

srussia (884021) | more than 4 years ago | (#28782955)

Geez, don't they teach editors to come up with snappy headlines anymore?

When too much is in the cloud it rains! (0)

Anonymous Coward | more than 4 years ago | (#28783115)

terrorist assassinate csystems JUWTF Waco, Texas kilo class Telex fissionable benelux Ruby Ridge Operation Iraqi Freedom anarchy World Trade Center data haven covert video ASO anthrax United Nations Zachawi Plame cypherpunk jihad Capricorn Mole FTS2000 unclassified NSA ammunition NASA SAPO strategic Janet Reno COSCO jihad Zachawi Abu Ghraib JFK JPL Bletchley Park Craig Livingstone AMW spy illuminati terrorism radar

Information overload makes for a leaky cloud. Have fun weeding through the false positives assholes spook motherfuckers!

ammunition rs9512c Al Jazeera military encryption AIEWS AIMSX pipeline New World Order ANC Perl-RSA mindwar global Ruby Ridge Rand Corporation

Neat software (1)

Demonantis (1340557) | more than 4 years ago | (#28783155)

This would be really helpful for law enforcement with their need to be able to access data from other police counties. There is tonnes of cases where cases cross several jurisdictions and the police have trouble integrating the investigations.

Intelligent Intelligence? (0)

Anonymous Coward | more than 4 years ago | (#28783411)

More geographic distribution = more points for hacker interception.
Intelligence has nothing to do with it.
Intelligent Intelligence = off-the-wall, off-the-air .
(Oh, but encryption will save us - yeah, right!)
Just a QA perspective

Re:Intelligent Intelligence? (1)

jamstar7 (694492) | more than 4 years ago | (#28784887)

No, this just means the data will 'disappear' into the cloud when some citizen comes walking up with a FOIA request to see the data. "Hey, all that shit got lost someplace on the net, we ain't got it!!"

why not google docs? (1)

fluffernutter (1411889) | more than 4 years ago | (#28784057)

Since the average shashdotter seems to think third party server clusters a.k.a. "The Cloud" are good and secure and useful for all us human beings and the intentions of said third parties are all well and good; I'm surprised that no one has suggested that the NSA should just use Google Docs or Facebook and get over the supreme silliness and unneccesary cost of private servers. Oo. Or what about Google Wave. I'm sure there's something in that for the NSA, it is the new cool thing after all.

Hey You! (1)

metaforest (685350) | more than 4 years ago | (#28800625)

I store my data in a VM on the 99th U of the Rack
And I sit ~/ look at the windows
Imaging the world cash crops
Then in flies a guy who's all dressed up like a NSA Spook
And says, I've won five pounds if I have his kind of data packet

I said, Hey! You! Log off of my cloud
Hey! You! Log out of my cloud
Hey! You! Log off of my cloud
Don't login here two UIDs is a crowd
On my cloud, Baby

The VOIP is signaling
I say, "Hi it's me. Who is it there in the stream?"
A voice says, "Hi, hello, how are you"
Well I guess I'm doin' fine
He says, "It's three a.m. watch the SNR boy!
Don't you ever halt that simulation?
Just cause it runs so good do you have
to fill the drives to the end?

I say, Hey! You! Log off of my cloud
Hey! You! Log out of my cloud
Hey! You! Log off of my cloud
Don't hack around 'cause two UIDs is a crowd
On my cloud baby

I was hacking python, and fed up with spaces
And decided to connect to slashdot
It was so very quiet and peaceful
With all of the ACs banned no new postings around
I hibernated the laptop, I was so lagged I started to scream
In the morning the parked heads were just like
A write-flag stuck on my windows machine

I say, Hey! You! Log off of my cloud
Hey! You! Log out of my cloud
Hey! You! Log off of my cloud
Don't hack around 'cause two UIDs is a crowd
On my cloud baby

Hey! You! Log out of my cloud
Hey! You! Log off of my cloud
Hey! You! Log out of my cloud
Don't hang my process, baby two UID's a crowd!

-----

With apologies to Mick and the boys.

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...