DARPA Seeks the Holy Grail of Search Engines 78
coondoggie writes "The scientists at DARPA say the current methods of searching the Internet for all manner of information just won't cut it in the future. Today the agency announced a program that would aim to totally revamp Internet search and 'revolutionize the discovery, organization and presentation of search results.' Specifically, the goal of DARPA's Memex program is to develop software that will enable domain-specific indexing of public web content and domain-specific search capabilities. According to the agency the technologies developed in the program will also provide the mechanisms for content discovery, information extraction, information retrieval, user collaboration, and other areas needed to address distributed aggregation, analysis, and presentation of web content."
Re:Join the slashdot farewell: (Score:5, Insightful)
Gee, I can't wait to participate in an echo chamber made up exclusively of nerds that got pissed to high hell when a UI redesign was _proposed_ on a site thats totally fucking free to them and will still work the same way in the future with regards to content (gripes about content aside)... What a joy it will be:
[dream twinkles]
"The scientists at DARPA say the current methods of searching the Internet for all manner of information just won't cut it in the future" What the fuck are they thinking? If they can't just use GREP to find what they need they must be some serious federal fuckups! Get Ron Paul down here to stop this!!!!!!!!!11
[/dream twinkles]
Ah what a lovely place, indeed.
Re: (Score:3, Insightful)
You have put a name to my pain, sir/madam.
I realized that the "Beta sucks!" chorus was bothering me, not just because it was a huge overreaction to, as you pointed out, a _proposed_ change to a _free_ site.
The thing that bothers me the most is that there seems to be almost a competition to show who can be the most obnoxious in registering their objections... from people who went through "gamer shaming" in high school, insults about nerdiness through college, and what not. It seems that these are people who
Re:Join the slashdot farewell: (Score:4, Informative)
You're a fucking idiot. Slashdot isn't about the articles and it isn't about the fucking pathetic summaries. Slashdot is about the user-generated content. Slashdot is the comments.
Now, you think all that time spent by knowledgable members posting all that good stuff comes free? People invested their time, care and attention here. Slashdot isn't a fucking TV. Slashdot is a community.
The reason the response was negative and loud is because the beta makes it harder to follow threads. It makes it harder to follow discussion context. It makes it harder to comment within the discussion context.The beta redesign ruins the community experience. Dice is trying to turn Slashdot into another Digg or Reddit. Soon, you'll be flooded with ads you can't turn off, looking for the useful, informative and interesting comments that used to be posted here. Fuck that shit.
TL;DR: FUCK BETA!
Re: (Score:2)
Re: (Score:3)
You're a fucking idiot. Slashdot isn't about the articles and it isn't about the fucking pathetic summaries. Slashdot is about the user-generated content. Slashdot is the comments.
Now, you think all that time spent by knowledgable members posting all that good stuff comes free? People invested their time, care and attention here. Slashdot isn't a fucking TV. Slashdot is a community.
The reason the response was negative and loud is because the beta makes it harder to follow threads. It makes it harder to follow discussion context. It makes it harder to comment within the discussion context.The beta redesign ruins the community experience. Dice is trying to turn Slashdot into another Digg or Reddit. Soon, you'll be flooded with ads you can't turn off, looking for the useful, informative and interesting comments that used to be posted here. Fuck that shit.
TL;DR: FUCK BETA!
Now, at least some reasoned, if seemingly over-wrought, debate.
You argue that the content, being user-provided, will go away if people are unable to see the content organized in a sensible fashion. Okay, I can buy that. I also hadn't considered that since the content is provided by users, they might have a reason to be a bit more passionate about how their thoughts are presented.
What I don't get is the need for insult to anybody who disagrees, or the call to leave Slashdot in droves even after they acknowle
Re: (Score:2)
2 things:
A First, on the bitter vituperation, as one who quite unabashedly shared my thoughts about how and why the new design was/is bad: much of the speech was strongly voiced, but it was simply the truth. If Dice kills the site layout and function, I will stop coming here. That's not a threat so much as a fact that I feel dice should be privy to. I don't want them to kill the site, a
Re: (Score:2)
In this instance, by failing to read the wind, Dice bared this truth to all of us; reminded us that this community isn't really ours so much as it is theirs. Absolutely that was a hurt to the community.
Now that's something that I can take to the bank.
You see, I lost that aspect of it when I was seeing people posting line after line of "Fuck Beta", often anonymously, and often seeming to offer insults to anybody who felt differently.
You, on the other hand, have answered my questions without questioning my intelligence, or heaping scorn and abuse on my head for daring to have a different opinion, or generally resorting to name-calling, so I don't think you qualify for "bitter vituperation". <grin> Rat
Re: (Score:2)
What I don't get is the need for insult to anybody who disagrees
You take the good [wikipedia.org] with the bad. [penny-arcade.com]
Re: (Score:1)
Slashdot is the comments.
Not anymore...
Re: (Score:2)
It's only a "free" site if you consider that the time and effort that the participants put into it is valueless. And "Beta" is not a term generally used to mean "trial balloon". It usually means, Next Week In Production (unless you're Google, in which case "Next Week" can be removed).
The advertisers pay for the hosting services and infrastructure, but without content, the product is nothing. So users have a definite interest in keeping the site usable and comfortable.
I haven't actually seen the Beta yet, bu
Re: (Score:2)
It's only a "free" site if you consider that the time and effort that the participants put into it is valueless. And "Beta" is not a term generally used to mean "trial balloon". It usually means, Next Week In Production (unless you're Google, in which case "Next Week" can be removed).
The advertisers pay for the hosting services and infrastructure, but without content, the product is nothing. So users have a definite interest in keeping the site usable and comfortable.
I haven't actually seen the Beta yet, but if it's truly keeping with recent trends in other products towards removal of commonly-used features and insertion of empty glitz, as some anti-Beta whingers have asserted, I can see why there is cause for a major uproar.
We have a lot of self-professed Libertarians here, they chant the mantra claiming that if you don't like service pick a competitor. Consider Digg. That's basically the route that Digg went down, its users fled and Digg dug its own grave. There aren't so many competitors for Slashdot that its users want it too to be ruined, so they're trying to ensure that it doesn't go that route.
I don't often counsel this in the IT world, but if it ain't broke...
If it ain't broke, it must run at least as well as Friendster.... Right? Or maybe it runs as well as Altavista? Geocities? Need some more examples of services that failed to change at the right time? No one said "f--- this beta friendster s---, I'm going to facebook because I dont like this any more", did they? Nope, they said "well hell, 10 of my friends just joined facebook, what am I doing on friendster?" and the rest is history.
Re: (Score:2)
It's only a "free" site if you consider that the time and effort that the participants put into it is valueless. And "Beta" is not a term generally used to mean "trial balloon". It usually means, Next Week In Production (unless you're Google, in which case "Next Week" can be removed).
The advertisers pay for the hosting services and infrastructure, but without content, the product is nothing. So users have a definite interest in keeping the site usable and comfortable.
I haven't actually seen the Beta yet, but if it's truly keeping with recent trends in other products towards removal of commonly-used features and insertion of empty glitz, as some anti-Beta whingers have asserted, I can see why there is cause for a major uproar.
We have a lot of self-professed Libertarians here, they chant the mantra claiming that if you don't like service pick a competitor. Consider Digg. That's basically the route that Digg went down, its users fled and Digg dug its own grave. There aren't so many competitors for Slashdot that its users want it too to be ruined, so they're trying to ensure that it doesn't go that route.
I don't often counsel this in the IT world, but if it ain't broke...
If it ain't broke, it must run at least as well as Friendster.... Right? Or maybe it runs as well as Altavista? Geocities? Need some more examples of services that failed to change at the right time? No one said "f--- this beta friendster s---, I'm going to facebook because I dont like this any more", did they? Nope, they said "well hell, 10 of my friends just joined facebook, what am I doing on friendster?" and the rest is history.
There's one critical difference, though. One group lost membership by failing to adapt when competitors started sucking their members away. The other group, however, chased members away. Slashdot seems to be falling into that second category.
Re: (Score:1)
Unicorns... you forgot the Unicorns.
Re: (Score:3)
Why invest so much money in this... (Score:1)
When they can just google it?
Re: (Score:2)
They aren't spying and censoring the information.
They are just making a search engine and present the information which fit your profile.
Re: Why invest so much money in this... (Score:2, Insightful)
Have you tried Google lately for a complex search? Or a search for a specific topic which uses words that are common to other, more popular topics?
It's a disgrace, and getting worse every year.
Re: (Score:3)
Have you tried Google lately for a complex search?
I believe that the presence of the words "information extraction" and "information retrieval" in the summary means that the search engine should be able to answer questions of the type "which kings were assassinated less then two years after their coronation?" by going through the facts available on the web, extracting them, and figuring stuff out. For any given random question, it's unlikely that someone already has it answered on the web, but the facts are all there anyway.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Agree with that. The other concern i have is that i want results only after 2011, for example chasing up statistics/surveys and looking for relevant ones, I constant find results from, say 2005. But because the site it is on refreshes the container for the content, it looks like new content. Note an easy thing to solve but perhaps google can hash the content, excluding the framework, and work out the true age of the content.
This issue was not a problem when everything googled was new, but after ten years
Re: (Score:1)
Re: (Score:2)
Re: (Score:1)
Google is not the answer.
Want proof? Try these two searches:
"Thor"
"atm"
How is Google supposed to know what to do with that? Do you want norse mythology, a comic book, or something else? Do you want packet switching information, bank information, or "Acrylic Tank Manufacturing" -- that's a new one.
About a decade ago, "Cow9" -- that was the name of the alta vista search engine -- had a wonderful solution to this, that required loading a java applet into your browser as part of the search. I loved it, and was
why? because NSA (Score:3)
FBI M-O-U-S-E. next best thing to surfing along looking over your shoulder, taking notes, texting the US Marshals Service what they need to subpoena before driving over and clapping on the leg irons.
Re: (Score:1)
Google really is an exceptionally good search engine if it wasn't for a career known as "Internet Marketing". Scam and ad sites are the #1 reason why it's so damn hard to find relevant pages in a search. I should not have to construct a search query that spans the entire width of a 1680 monitor just to filter out all the criminally insane "Internet Marketing" guys grab for cash.
Re: (Score:1)
Actually there is more truth in this answer than most people would think. With wolfram alpha and knowledge graphs Google did recently included features that actually do waht DARPA has in mind: make sense of crawled data, draw conclusions and display accordingly.
The difference with Google is: They do introduce the changes step by step. But it is defininately out there: I just recently noticed that the first search results are more and more spread around different possible meanings of my search terms, when in
Re: (Score:1)
Re: (Score:1)
citizens (Score:1)
FTFY
Re: (Score:2)
I guess they decided... (Score:3)
They might want to be careful. (Score:5, Interesting)
Second - we (common netizens) may welcome the sort of information availability DARPA is seeking - sort of like the scifi future where you just ask the nearest terminal whatever you want to know and magically get the answer you need - but there are lots of bad people still running around on this planet (scamsters, governments, jilted ex-lovers, religious extremists, etc.). The problem isn't the technology, the problem is our ability to handle it.
I very much suspect DARPA may be onto something. I wonder if it will be as beneficial as the WWW has been.
TFS reads like marketing babble (Score:2)
Domain-specific huh? (Score:2)
Google already does domain-specific indexing, certain sites get indexed faster or deeper than others based on a number of secret rules.
For site specific search prefix your query with "site:foo.com"
Re: (Score:2)
Addendum: It just hit me that they might've meant *topic* specific.
Google has something like this as well but it can be hard to notice. I remember one time I was searching for something automotive-related and my query had a word in it that meant something in the fashion world. It took hell to get out of "fashion mode" and get automotive results, IIRC I had to completely rephrase the question.
Re: (Score:2)
DARPA Seeks the Holy Grail of Search Engines (Score:1)
So do I... One that is robust and can't be censored by anybody, ever.
Use Google to search the Internet for it.. (Score:2)
Just search in Google for "holy grail of search engines". Job done.
Reads like one of those RFPs... (Score:1)
Sometimes, as you read the RFP document you can tell it's been written with a specific proposer in mind, and everybody else gets a short window to come up with an idea + jump through endless list of government hoops.
one of the smartest guys I know uses... (Score:2)
metasearch
*yawn* (Score:1)
Google, Deepmind, etc.
The Shannon Limit is the Big Bottleneck Problem (Score:1)
Today unstructured information search findability is limited by the Shannon Limit, this is a fundamental physical limit since all pattern search engines are statistical decoders. Google does a little better than the Shannon limit by looking at which search results are selected, this a communal intelligence technique based on how we "vote" for the right result. Unfortunately this only works well for high volume searches, that's why Google work's best if you know exactly what you're looking for or you're look
No Holy Grail, Buuuuuuuuttttt... (Score:2)
If they can't find the Holy Grail of search engines, I can direct them to this alternative that is just a Grail Shaped Beacon of a search tool instead...
OWL and RDF (Score:1)
One easy improvement... (Score:2)
Have one without "sponsored ads".
Google has produce less useful hits over the last five years, as its advertising income skyrocketed. Search for something, and add -"photo", and you'll still get photographer. Try mens riding boots -women -womens -women's, and you'll probably get a "sponsored ad" for women's boots, with the letters "men" bolded.
This isn't even counting Target, which will claim it has anything you're looking for, but if you follow the link, oh, no, sorry... but we have !!!!!
Add the way librar
Re: (Score:1)
... not just with AND and OR, but also "within so many words"...
Try Google's firstterm AROUND(3) secondterm.
That's it...I'm out'a here - farewell to Slashdot (Score:1)