Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Watson Wins Jeopardy Contest

samzenpus posted more than 3 years ago | from the john-henry-hangs-up-his-hammer dept.

AI 674

NicknamesAreStupid writes "The word is in, Watson beats the two best Jeopardy players. Sure, it cost IBM four years and millions of dollars and requires a room full of hardware. In thirty years it will all fit in your pocket and cost $19.99. Resistance is futile; you will be trivialized."

cancel ×

674 comments

Sorry! There are no comments related to the filter you selected.

AI Winter (2)

MarriedGeek (634736) | more than 3 years ago | (#35228494)

I think it is safe to say the AI Winter is over.

Re:AI Winter (4, Insightful)

Jane Q. Public (1010737) | more than 3 years ago | (#35228558)

Why? I don't see anything more special or "AI" in this than in Deep Blue's wins at chess so long ago.

Yes, the natural language processing is impressive. But it takes a really huge computer, and it's really nothing more than a bunch of clever software along with a database of trivia.

Watson showed very clearly how deeply it did not "understand" anything about what it was doing, via the nature of the blunders that it did make.

Re:AI Winter (1)

The End Of Days (1243248) | more than 3 years ago | (#35228580)

The main thing Watson showed by its mistakes was that it had no hardware monitoring the studio.

Re:AI Winter (2)

TheLink (130905) | more than 3 years ago | (#35228768)

The other thing Watson showed by its mistakes is that its AI still lacks understanding and intelligence.

Often it's your wrong answers that show how much you really understand.

http://www.csmonitor.com/Innovation/Latest-News-Wires/2011/0215/On-Jeopardy-Watson-s-mistakes-reveal-its-genius [csmonitor.com]

Clue: It was this anatomical oddity of US gymnast George Eyser.
Ken Jennings' answer: Missing a hand (wrong)
Watson's answer: leg (wrong)
Correct answer: Missing a leg

And the "Toronto":
http://www.theglobeandmail.com/life/the-hot-button/watson-computer-makes-elementary-error-on-jeopardy/article1909685/?cmpid=rss1 [theglobeandmail.com]

Once the AI's wrong answers start to look intelligent, the next level of understanding would be when the AI can actually teach someone what it knows.

If you can teach difficult new stuff to stupider people then you're getting to Feynmann material :).

Re:AI Winter (2)

Totenglocke (1291680) | more than 3 years ago | (#35228836)

Watson's answer was correct - he just didn't phrase it the way they want. Human contestants get hit with that rather frequently on Jeopardy for forgetting to say "What / Who / Where is....?". Hell, he actually had more correct answers than he buzzed in because he's programmed to not buzz in if he's not above a certain confidence level for the answer (I forget what it was in the video....under 90% and he wouldn't buzz in, I believe).

Re:AI Winter (0)

Jane Q. Public (1010737) | more than 3 years ago | (#35228900)

"... he just didn't phrase it the way they want..."

And on Jeopardy that's still called a "wrong answer".

Re:AI Winter (2)

petteyg359 (1847514) | more than 3 years ago | (#35228614)

Your brain is nothing more than a bunch of clever software with a database of trivia, and you do not understand many things about what you do. The template for Brain - being released many thousands of years ago - having longer to get fixes and new features, and your specific instance of Brain having some years to add content and optimize its database indices doesn't make this event any less significant.

Re:AI Winter (2)

Jane Q. Public (1010737) | more than 3 years ago | (#35228872)

"Your brain is nothing more than a bunch of clever software with a database of trivia..."

Not only do we not know this, there is a good bit of evidence to the contrary. The human brain is not anything like the computers we build today, except to say that they can both "compute", to varying degrees.

But even if you did believe that they worked in fundamentally the same way, THIS article [arstechnica.com] from just the other day claims that the best estimate so far is that all the computing power in the world today, including Watson, Deep Blue, all the Crays, desktops, and all the way down to cell phones, added together... is equivalent to approximately the computing and storage power of a single human brain.

So... it's going to be a very long while before we can make a single computer anywhere near that powerful. Much less make it fit in a pocket.

Watson is a good demonstration of what modern software -- with the aid of an outrageous shitload of processing power, even by today's standards -- can do in the area of natural language processing. Nothing more. And even that has a long way to go.

Re:AI Winter (2)

MoonBuggy (611105) | more than 3 years ago | (#35228790)

Why? I don't see anything more special or "AI" in this than in Deep Blue's wins at chess so long ago.

Chess can be brute forced in a way that language can't. Not to say that Deep Blue was all brute force, and I don't think Watson is true AI, but I do think it's a moderately significant step, not to mention more practically applicable. If nothing else, it (in combination with some decent speech recognition software) brings us one step closer to a Star Trek style voice interface, and that's a damn worthwhile cause in my book!

These debates always remind me of something I read once, too. I forget the book, but the gist was that coders will be the last people to accept true AI if we ever do create it; we tend to believe that we have a full understanding of the way computers work, and what they are and are not capable of. While this is true to an extent, it becomes less so as systems become more and more complicated - I'm sure there's not a person on Earth who could fully understand a million-line program singlehandedly. Consciousness is just the emergent behaviour of organic computing hardware and software, and there's no reason that a sufficiently complex artificial system couldn't eventually show similar emergent behaviour, intentional or otherwise.

Re:AI Winter (1)

Darinbob (1142669) | more than 3 years ago | (#35228854)

That's what AI is. Whatever we think is hard to do today is AI. As soon as we're able to do it someone will say "bah, that's not real AI". .

The snag is that people have this myth of AI as this science-fiction thing, where machines will be able to think and have consciousness. But that's not what people in AI do. They are working on solving actual problems with how to make computers smarter and able to solve tougher problems.

Re:AI Winter (0)

Anonymous Coward | more than 3 years ago | (#35228724)

Not even close. Watson doesn't 'understand' anything about the jeopardy answers and questions, just simply searches its data for correlations between words and phrases. e.g. Watson thought Harry Potter killed Snape

Re:AI Winter (1)

Totenglocke (1291680) | more than 3 years ago | (#35228852)

Well, depending on how you view the situation, you could argue that Harry DID in fact kill Snape.

Real competition (1)

get_your_guns (1380583) | more than 3 years ago | (#35228504)

The real competition will be when China's super computer competes against IBM's on Jeopardy.

Could it be? (5, Funny)

Anonymous Coward | more than 3 years ago | (#35228506)

Does Ken Jennings read Slashdot comments?

Re:Could it be? (0)

Anonymous Coward | more than 3 years ago | (#35228592)

I, for one, welcome our new slashdot-reading-competitor-defeating computer overlords.

Skynet (1)

Art3x (973401) | more than 3 years ago | (#35228514)

This computer later became Skynet.
Ding!
What --- who shall I say, who --- is Watson?

Re:Skynet (1)

ZiakII (829432) | more than 3 years ago | (#35228542)

This computer later became Skynet.

More like HAL, I mean IBM is only 1 letter way on each.

Re:Skynet (1)

Dexter Herbivore (1322345) | more than 3 years ago | (#35228624)

It's when they start teaching it to eat that we have a huge problem... because then it will be "Alimentary, my dear Watson".

Thankyou, thankyou. I'll be here all week, make sure to tip your waitress.

Waton's Wagering and HAL 9000 (1)

JoshuaZ (1134087) | more than 3 years ago | (#35228516)

I tuned in for the end. One thing I'm very curious about is how Watson decided how much to wager for the daily doubles and Final Jeopardy. I haven't seen much discussion of these, but it seemed from the numbers it was giving that it had some set of heuristics to decide how much to wager based on how much money it had, the amount of of money the other contestants had, and possibly (not sure about this) its confidence in the category type. The Final Jeopardy category was 19th century novels, which seems to be the sort of thing Watson excels at (it doesn't do as well in the categories involving wordplay and puns although it seems to still do much better than most humans). However, one thing that came up was the disappointingly easy nature of the Final Jeopardy question. I and another person watching got the question as soon as the answer was put up. It seemed from the behavior of Jennings realized that Watson had won given the easy nature of the question.

Overall, I was impressed with Watson's performance. I suspect that if they had given it a slightly more human sounding voice-sympathizer it would have come across as a much bigger deal. (Also does anyone know if Watson was deliberately made to look like HAL except in soothing colors rather than scary red? I have trouble not seeing that as deliberate.)

Re:Waton's Wagering and HAL 9000 (1)

Jane Q. Public (1010737) | more than 3 years ago | (#35228574)

It was probably intentional that they did not give it a realistic, human-sounding voice. Research has shown that people do not want machines to appear too human. They react negatively.

Re:Waton's Wagering and HAL 9000 (1)

93 Escort Wagon (326346) | more than 3 years ago | (#35228642)

It was probably intentional that they did not give it a realistic, human-sounding voice. Research has shown that people do not want machines to appear too human. They react negatively.

[citation needed]

[Star Trek and Asimov references don't count]

Re:Waton's Wagering and HAL 9000 (1)

Aranykai (1053846) | more than 3 years ago | (#35228706)

Its theory, but still.

http://en.wikipedia.org/wiki/Uncanny_valley [wikipedia.org]

Re:Waton's Wagering and HAL 9000 (1)

tlhIngan (30335) | more than 3 years ago | (#35228936)

Its theory, but still.

http://en.wikipedia.org/wiki/Uncanny_valley [wikipedia.org]

It's not really a theory - it's an observation.

And I have to admit, I found Watson's voice uncannily creepy, especially since someone programmed in all those Jeopardy phrasings that other players over the years have done.

And IBM actually used a voice actor to serve as the base for Watson's voice synthesizer.

I would've preferred a more robotic voice - despite having a lot of mechanical generation (odd intonation), it was still very unsettling.

Re:Waton's Wagering and HAL 9000 (2)

ArtDent (83554) | more than 3 years ago | (#35228596)

The IBM Research blog has had a few good articles about Watson over the past few days, including one about wagering:

http://ibmresearchnews.blogspot.com/2011/02/watsons-wagering-strategies.html [blogspot.com]

I didn't think that Final Jeopardy would have been especially easy for Watson. The majority of the clue was indirectly related to the correct response, and the connection hinged on a single word (inspired). I suspect Jennings' behavior was based more on simple arithmetic than on any assumptions about Watson's response.

Re:Waton's Wagering and HAL 9000 (1)

JoshuaZ (1134087) | more than 3 years ago | (#35228630)

Thanks for the blog link. That looks very interesting. As to the ease of the question, I don't think the major issues had to do with "inspired". I would have identified the key issues as parsing that they were looking for an author, and then associating Walachia with Transylvania and then Dracula.

Re:Waton's Wagering and HAL 9000 (1)

loconet (415875) | more than 3 years ago | (#35228690)

Here [blogspot.com] is a post on Waton's wagering by IBM Research.

Re:Waton's Wagering and HAL 9000 (1)

travdaddy (527149) | more than 3 years ago | (#35228730)

It seemed from the behavior of Jennings realized that Watson had won given the easy nature of the question.

Jennings realized that Watson had won, but not due to the easy nature of the final Jeopardy question. Watson already had the game won before Final Jeopardy. Had Ken doubled his score with his wager and Watson got Final Jeopardy wrong, Watson still would have won by $1.

Re:Waton's Wagering and HAL 9000 (1)

Pretzalzz (577309) | more than 3 years ago | (#35228732)

Jennings realized that Watson had won since combined with the first day Watson had an insurmountable lead. Jennings betting strategy even before seeing the question was to play for second. For what its worth I didn't know the final jeopardy question...

Re:Waton's Wagering and HAL 9000 (1)

wiredlogic (135348) | more than 3 years ago | (#35228774)

My impression is that the wagering isn't influenced by the category but only its standing against the other players, at least for Final Jeopardy where it didn't have the benefit of seeing previous responses in the category. I thought it funny that they decided not to round the wagers to more typically human choices. That was certainly deliberate on the developers' part.

I also thought it was interesting that the only category the humans swept was the one asking for the directors of movies. These were simple factoids and it had all the right answers, but the short questions from Trebek exposed that there was too much processing overhead to buzz in first on them.

It can beat Ken Jennings at Jeopardy, but... (2)

doubleplusungodly (1929514) | more than 3 years ago | (#35228520)

can it do my homework for me?

Re:It can beat Ken Jennings at Jeopardy, but... (1)

alvinrod (889928) | more than 3 years ago | (#35228564)

Doubtful. As impressive as their win is, and as much as they've learned from it, they mostly have a computer that's really good at Jeopardy. I don't think that it could answer an essay question in a satisfactory manner.

We still have a long way to go despite how far we've come.

Re:It can beat Ken Jennings at Jeopardy, but... (1)

underqualified (1318035) | more than 3 years ago | (#35228598)

Yes, but it answers everything in the form of a question.

Re:It can beat Ken Jennings at Jeopardy, but... (1)

Darinbob (1142669) | more than 3 years ago | (#35228878)

Sure, just give IBM 4 years and a few million dollars, and they'll build you one.

As ken said: (2, Interesting)

Dayofswords (1548243) | more than 3 years ago | (#35228532)

I for one welcome our new computer overload.

Wasn't this taped awhile ago (1)

black6host (469985) | more than 3 years ago | (#35228534)

For some reason I was under the impression that this had already occurred but is just now being aired. And IIRC, it was already known that Watson won. So why is this news? Or, I'm a precog and should have made some money on this :)

Re:Wasn't this taped awhile ago (0)

Anonymous Coward | more than 3 years ago | (#35228758)

I don't think the results leaked, so it's news now because it "just happened" insofar as we just learned what the result was.

Re:Wasn't this taped awhile ago (2)

dlgeek (1065796) | more than 3 years ago | (#35228822)

There was a practice round, and it was widely circulated that Watson had won that, though it wasn't true - it dominated the first two rounds but Ken pulled ahead with a huge wager in a double jeopardy and then another in final jeopardy (which Watson got wrong). The results of the actual challenge did not leak.

Re:Wasn't this taped awhile ago (1)

black6host (469985) | more than 3 years ago | (#35228924)

Ah, ok. Thanks for the clarification. Still should have bet anyway though :)

It still needs a lot of work... (1)

Fallen Kell (165468) | more than 3 years ago | (#35228538)

A lot of it has to do with game mechanics, like listening to what the other contestant said in a wrong answer and adjusting your answer accordingly. Case and point was on the first night in the "Decade" category where watson got beat on clicking the button, and the player that beat him said "What are the 1920's" and was wrong, and then watson answered "What are the 20's", which was still the wrong answer....

I have to admit, it was pretty impressive as that is a fairly non-trivial computational problem of not only understanding the english language, but also taking the clues from the categories and how things are phrased to come up with the appropriate response.

Re:It still needs a lot of work... (1)

DAldredge (2353) | more than 3 years ago | (#35228648)

Watson doesn't know what they other players questions are so such a mistake is understandable.

Re:It still needs a lot of work... (1)

dlgeek (1065796) | more than 3 years ago | (#35228744)

Apparently the decision to not take the input of what the other players said was partly based an an assumption that the other players would almost never be wrong!

Re:It still needs a lot of work... (1)

EnsilZah (575600) | more than 3 years ago | (#35228814)

I think it's more about the fast nature of the game and the time it would take to type it out.

Re:It still needs a lot of work... (0)

Anonymous Coward | more than 3 years ago | (#35228760)

Case and point

It's case IN point, dude.

Fast on the clicker (2)

MortimerV (896247) | more than 3 years ago | (#35228548)

I didn't watch it all, but the thing I noticed was that, when Watson thought it had an answer, most of the time it'd click in first. The other contestants didn't have a chance to attempt to answer.

So Watson wins on reaction time, which isn't a surprise for a computer that knows exactly when it can first ring in. How would it have done with a human's reaction time on clicking, just answering on questions alone?

Re:Fast on the clicker (1)

benjamindees (441808) | more than 3 years ago | (#35228610)

That seems to be a common complaint but remember in Jeopardy you lose money if you ring in with the wrong answer. So even though Watson can ring in first every time, it has to be extremely confident that it has the right answer.

Re:Fast on the clicker (0)

Anonymous Coward | more than 3 years ago | (#35228704)

By "extremely confident" do you mean >50% sure? Mathematically, this is the cutoff it should use.

Re:Fast on the clicker (2)

QuantumLeaper (607189) | more than 3 years ago | (#35228856)

I remember an interview with Ken, he said he would push the button even though he didn't know the answer, and he hoped he would figure it out in time. Most of the time it was a winning method, but sometimes it wasn't.

Re:Fast on the clicker (1)

InsaneMosquito (1067380) | more than 3 years ago | (#35228626)

Re:Fast on the clicker (2)

SanityInAnarchy (655584) | more than 3 years ago | (#35228812)

From that page:

"They're not waiting for the light to come on," Welty said; rather, the human players try to time their buzzer presses so that they're coming in as close as possible to the light. Though Watson's reaction times are faster than a human, Welty noted that Watson has to wait for the light.

There's more to it than that, also -- it's often the case that Watson isn't sure it has the right answer, and you're penalized for wrong answers. Also, I'm not positive, but I think I saw in some of the trial runs that there were a few cases it actually was slower than a human -- where it came up with the right answer, eventually, but humans beat it easily to the buzzer. I'm not sure if that's the case, but I can definitely believe it -- there's a lot of stuff to sort through, and they're running a lot of algorithms on it.

Re:Fast on the clicker (2)

artor3 (1344997) | more than 3 years ago | (#35228824)

Actually, based on the article you linked, Watson does have an advantage on the buzzer. The human players have to guess when the light will come on and try to press right afterward - one instant early and your buzzer is disabled. The computer can "see" the light come on and press immediately. It's reaction time is likely measured in microseconds. You try timing a button press to come in during a 100 microsecond window.

If they were interested in a good game, they should have designed it to simulate a typical human reaction time, but that's not the point. The point was the proof of concept - that the machine can answer Jeopardy questions accurately. Really, they didn't even need the human players there, it would have been just as interesting to see it blaze through the board solo.

Re:Fast on the clicker (0)

Anonymous Coward | more than 3 years ago | (#35228834)

I was expecting some kind of proof. All I got is some guy claiming that humans' "timing and rhythm" matches Watson's reaction times. Sorry, that's not convincing at all.

Re:Fast on the clicker (1)

phantomfive (622387) | more than 3 years ago | (#35228886)

Don't know, it sure looked like the computers had a huge speed advantage. Even your article doesn't make a very strong argument, it starts out by saying "Watson seemed to be running the round and beating Jennings and Rutter to the punch with its answers many times" and that is true. The argument your article makes, that humans can anticipate the light turning green because they know when the question ends, is not valid, because the computer can press the clicker within a millisecond of the light turning green. No human can match that. The clicker speed was chosen by the researchers to be slower, but they left it fast enough to be an advantage.

Wouldn't you? Clicker speed is a huge part of Jeopardy......in some rounds, both (or all) contestants know all the answers, and it comes down to clicker speed.

Re:Fast on the clicker (1)

ArtDent (83554) | more than 3 years ago | (#35228644)

One interesting thing to note is that Watson was tied (on Monday) or behind (today) after the Jeopardy round and pulled ahead (way ahead yesterday) in the Double Jeopardy round, where the questions are harder. That's not what you'd expect if its competitors knew all the answers and it was winning on ring-in speed alone.

In any case, Watson was playing Jeopardy, and ringing in is a part of Jeopardy. Rutter and especially Jennings certainly benefited from that part of the game during their long winning runs. Watson also had unique disadvantages compared to his opponents (like being unable to hear their responses).

Re:Fast on the clicker (1)

artor3 (1344997) | more than 3 years ago | (#35228832)

Watson was able to "hear" their responses. It made one blunder in the game, repeating Jennings' answer, because it failed to recognize that the 20s and the 1920s are the same decade.

Re:Fast on the clicker (1)

mark-t (151149) | more than 3 years ago | (#35228910)

It was said at the beginning that Watson could neither see nor hear.

Re:Fast on the clicker (1)

jdogalt (961241) | more than 3 years ago | (#35228696)

The part I find most amusing is the irony that Watson's speed on the buzzer seems directly attributable to the handicap it was given vis-a-vis not having to do the audio-speech and visual-text AI recognition that its human opponents had to do. The irony being that those two allegedly mature areas of AI research are ones in which IBM has been shipping commercial products for years. I suspect that if Watson had to do such processing, it would have been slaughtered just as bad as it slaughtered its opponents, if for no other reason than an extra couple tenths of a second of delay before it could decide to click its buzzer. I'm sure this may handicap may no longer be necessary in a few more months or years, but even in the Nova episode, they seemed to gloss over that.

30 years? Try 5 or 10. (1)

Omnifarious (11933) | more than 3 years ago | (#35228552)

It's not going to take 30 years for that system to fit in your pocket and cost $20. It's going to take 5 or 10.

The real question (0)

Anonymous Coward | more than 3 years ago | (#35228562)

How can net entropy be reversed?

Who's the real winner? (1, Interesting)

wandazulu (265281) | more than 3 years ago | (#35228570)

What I mean is, what IBM products will be the beneficiary of the tech they developed to make Watson; DB/2? WebSphere? You've gotta think that the IBM execs only agreed to go forward with this whole thing with some thought to being able to leverage it in other products.

Personally, I've love to think this was a "pure research" thing, but I doubt anyone really does that anymore (though I hope I'm wrong).

Re:Who's the real winner? (1)

dlgeek (1065796) | more than 3 years ago | (#35228864)

I'm sure google would be interesting in licensing the tech to make their advertisements even more relevant....

Re:Who's the real winner? (1)

phantomfive (622387) | more than 3 years ago | (#35228920)

It's something that will compete directly with Autonomy [autonomy.com] . This kind of searching is big in certain industries, law perhaps the foremost. The ability to search through documents quickly and find what you want means you can fire your legal aids. That is worth what, $60k a year, per person that you can fire? So people are willing to pay big money to Autonomy and IBM. I'll bet Autonomy is wishing they'd thought of this stunt.

I couldn't help but notice that I was right... (2)

mark-t (151149) | more than 3 years ago | (#35228572)

... that Watson tended to fare poorer against the human beings when the clues were very terse.

That said, I don't want to dismiss the natural language recognition capabilities of Watson. They are no small feat, and by all rights, the designers of it should be congratulated on this effort. Nevertheless, with respect to the game of Jeopardy, I remain convinced that Watson's key advantage over the other players was that it is essentially a super-fast speed reader, having a few moments to pontificate the clue before any human could possibly be finished reading it. If the text of the clue had been transmitted to Watson more slowly to approximate the menial task of reading, I think it might have been a better indicator of whether or not Watson was actually out-thinking Brad and Ken. A speed I think would be appropriate to transmit the text of the clue at is about the same as what you'd get with a 14,400 bps modem, which still would amount to insanely fast speed reading, but it's at least within an order of magnitude of what is humanly achievable. Then, the amount of time that Watson has to think about the clue gets a lot closer to how much time the other players get to think about it. As it sits, Watson gets to start trying to parse the entire sentence before any human has even finished reading the first word.

Of course, I don't think that Deep Blue really out-played Kasparov on a level playing field either... I would be far more impressed if they could design a chess-playing computer that only considers a few hundred board combinations and still plays at a grandmaster level, since that is all that even the best human grandmasters do.

OCR and voice recognition. (0)

Anonymous Coward | more than 3 years ago | (#35228882)

That's how Watson should have gotten its clues. OCR the clue screens, and voice recognition on Alex' voice. It should then parse from that. Getting a direct input of text is unfair.

Re:I couldn't help but notice that I was right... (2)

phantomfive (622387) | more than 3 years ago | (#35228928)

Don't forget clicker speed.....there were many times that the two humans knew the correct answer and the computer won. That's an advantage that can't be ignored.

We do know that... (2)

Trip6 (1184883) | more than 3 years ago | (#35228584)

...it was us that scorched the sky...

Underwhelming achievement (3, Insightful)

David Jao (2759) | more than 3 years ago | (#35228586)

Despite all the media hype, I for one am not at all impressed by this feat.

Various media articles have made clear that Watson has no visual or auditory input. Presumably Watson is receiving a direct digital feed of the tournament questions (oops, answers, I forgot this is Jeopardy). That alone gives Watson a huge timing advantage over the human competitors, who must (effectively) perform voice recognition and OCR to process the clues. On top of that, Watson has the computer-controlled ability to buzz in in four milliseconds, again giving it a huge advantage over the humans, and one that has nothing to do with AI.

Buzzer timing and strategy is a highly significant part of the game of Jeopardy. Given its direct digital feed and its internal computer clock, Watson is not playing this part of the game by the same rules as the humans. Thus, it's not fair to say that Watson wins a "Jeopardy" contest -- Watson has a huge unnatural advantage. In effect, Watson is not playing the same game as what we normally call "Jeopardy." A real Jeopardy contestant has to use eyes and ears and hands in addition to brain.

To be clear, I do think Watson is a worthy achievement. But this feeling is overshadowed by my constant annoyance at the media and others who incorrectly label this achievement as somehow winning a game of Jeopardy.

Re:Underwhelming achievement (1)

jfengel (409917) | more than 3 years ago | (#35228654)

Jeopardy does seem an odd demonstration here. The remarkable thing is not how well the bear dances, but that it dances at all. If it had taken 15 seconds to find the answer I'd have been just as impressed. An order of magnitude in performance is just a matter of waiting a few years.

Certainly, it gets a lot more attention this way, which is presumably the point. But I'm not quite sure about attention for what, since it's not a product you can buy. I'm not aware of any productization plans.

It's good marketing for IBM in general, I suppose, but I'm not sure what Ken Jennings and Alex Trebek are getting out of it besides announcing their pending obsolescence.

Re:Underwhelming achievement (1)

qbwiz (87077) | more than 3 years ago | (#35228694)

Ken Jennings is getting $150000 (plus $150000 to his charity), while the producers of Jeopardy are getting a lot more viewers (and therefore a lot more advertising money).

Re:Underwhelming achievement (1)

Demonantis (1340557) | more than 3 years ago | (#35228918)

It is great marketing for IBM and they are already working on a newer one called Racr. The more significant thing is the relational concepts that were researched to accomplish this feat. They can be applied in a huge number of applications, but you will never hear that this project started as government funded research. You just hear that IBM managed to make hardware powerful enough to play Jeopardy.

Re:Underwhelming achievement (4, Insightful)

MoonBuggy (611105) | more than 3 years ago | (#35228922)

It's good marketing for IBM in general, I suppose, but I'm not sure what Ken Jennings and Alex Trebek are getting out of it besides announcing their pending obsolescence.

As another poster said, Jennings and the Jeopardy crew are making good money from this. As for IBM, they benefit in a few ways - firstly, the techniques learned in making the software will be very, very marketable, even if you don't see a box marked "Watson" on the shelf any time soon, and I'm sure the public challenge was a good way to keep the dev team motivated and enthusiastic. Secondly, the publicity; I know you realise that was part of their goal, but I think perhaps you underestimate just how successful it was - the general public are enthusiastically talking about what is essentially an IBM tech demo. I doubt most of the people I see discussing this would look twice at a more traditional story about some piece of random computer science research, even if it did happen to get a column somewhere on page 15 of a mass-media publication. The IBM name is becoming synonymous with AI, and it endures; people still talk about Deep Blue, and that was over a decade ago. A shiny superbowl ad that people talk about for a week is 'good marketing', and I think this goes many levels beyond that.

Re:Underwhelming achievement (5, Insightful)

ArtDent (83554) | more than 3 years ago | (#35228716)

Correction: competitors must perform voice recognition or OCR to process the clues. The clues are displayed and read, and the contestants are free to ignore either form, if they wish. Similarly, Watson could have had a camera trained on the monitor and performed OCR on the clue. But, given that OCR has been done brilliantly by computers for years now, would adding that into the mix have made much difference at all?

Regarding ringing in, the contestants also get a signal indicating when they can do it, but it's visual. It would have been easy enough to add another camera trained on the light, but why bother?

The engineers involved were trying to solve the interesting problems. Delivering input to each contestant in the most convenient form doesn't seem like much of a concession.

Re:Underwhelming achievement (1)

David Jao (2759) | more than 3 years ago | (#35228880)

The engineers involved were trying to solve the interesting problems. Delivering input to each contestant in the most convenient form doesn't seem like much of a concession.

I disagree with your assessment that OCR and voice recognition are not interesting problems, but in any case the entire issue could have been avoided by just doing what you suggested: add a camera to Watson and perform OCR.

The fact that IBM did not do this indicates that they have something to hide, and makes the whole thing the worst kind of publicity stunt.

Re:Underwhelming achievement (1)

FatLittleMonkey (1341387) | more than 3 years ago | (#35228870)

That alone gives Watson a huge timing advantage over the human competitors, who must (effectively) perform voice recognition and OCR to process the clues.

Don't forget that we are processing the clue for the answer (question! whatever!) even while we are processing it for comprehension. (*) Doing all that parallel analogue processing that the human brain is insanely good at. (Haven't you had that feeling that you know the answer to a question before you actually "remember" the answer? There's some freaky-assed recognition processing going on in there.)

Forcing Watson to wait until... however long you estimate it takes humans to read the clue... before Watson "sees" it, let's say two seconds, would give humans a nearly two second head start.

(* The spoken clue is a) for the audience b) to provide a timing sequence for the release of the buzzers. I doubt any but the most amateurish players rely on the spoken clue.)

Re:Underwhelming achievement (1)

Garble Snarky (715674) | more than 3 years ago | (#35228914)

OCR on a clean image (like the Jeopardy screens) is a solved problem. A physical button can be pressed with a trivial actuator, which could be almost as fast as the direct digital connection. How would adding either of those features to the machine make it any more impressive? Should it be crippled by inserting an artificial processing delay before it is allowed to buzz? Why would that be fair?

If you really want to be able to say that the machine "competed fairly" with humans in the game, I think there should be two simple requirements: the machine must occupy no more physical volume than a human skull, and consume no more electrical power than the equivalent of a human metabolism.

Re:Underwhelming achievement (1)

laci (37234) | more than 3 years ago | (#35228926)

Actually, you are wrong about the buzz in. It gives the humans an advantage. The reason is that Watson has a mechanical buzzer that it presses. So the only advantage would come from reacting faster. However, the rules state that you can buzz in only after the host has finished reading the clue. If you buzz in earlier then you are penalized by .25sec. Jeopardy is prepared for hearing disabled people, so they have a signal indicating that the host has finished reading. I presume this is what Watson uses. However, I'm certain that humans process intonation as well and can anticipate when the host will finish. So they have a better chance to buzz in first if they decide to buzz in.

Buzzer speed. (1)

RyanFenton (230700) | more than 3 years ago | (#35228594)

The key to victory seemed more decided by buzzer speed than anything else. Even as the other players seemed to try to buzz in, regardless of answer, they just didn't have the split-second precision as Watson did in triggering his buzzer, time after time.

Ryan Fenton

Re:Buzzer speed. (1)

InsaneMosquito (1067380) | more than 3 years ago | (#35228658)

Re:Buzzer speed. (4, Interesting)

JMZero (449047) | more than 3 years ago | (#35228816)

Uh - Watson obviously, obviously had a speed advantage. On today's episode there were many, many obvious answers (obvious to me - to Ken Jennings or Brad Rutter, blindingly - stupidly blindingly - obvious). Watson got almost all the obvious questions, and many times you could see the little eye roll of frustration from Ken and Brad.

On questions like this, Ken and Brad would have been waiting and trying to time the ring in (they would have known the answer long before the buzzer was active).

They lost almost every time.

So while the computer may not have had an absolute advantage (ie. if Ken could have rung in within milliseconds of the buzzer being active he would have been OK) it's clear it had an effective one in that it's a bloody machine that can ring in very quickly after the buzzer is active. I mean, yeah, it's cool that the computer knows the answer that fast - but we didn't get to see anything of a comparison of who knew a greater percentage of answers.

And, again, this was absolutely, obviously clear to anyone who watched the show. I don't believe anyone could have watched the show and not realize this. Honestly, this is a problem even without the computer. Between high level contestants, buzzer speed is going to determine the winner 9 times out of 10 - the Jeopardy questions just aren't hard enough to distinguish between the best competitors. Oh, and somewhere else in this article someone said "that's part of the game". That's true. But it's a stupid part of the game - and it makes it impossible to compare competitors with different brain technologies in any interesting way.

Re:Buzzer speed. (0)

Anonymous Coward | more than 3 years ago | (#35228662)

The key to victory seemed more decided by buzzer speed than anything else. Even as the other players seemed to try to buzz in, regardless of answer, they just didn't have the split-second precision as Watson did in triggering his buzzer, time after time.

Ryan Fenton

I noticed that too.

Anonymous Coward

Re:Buzzer speed. (1)

Bemopolis (698691) | more than 3 years ago | (#35228668)

The key to victory seemed more decided by buzzer speed than anything else.

This statement is true when there are three bags of meat playing instead of two. Anyone on the show has passed the test; the difference between winning and losing is mostly reaction time and resistance to pressure.

Re:Buzzer speed. (1)

michael_cain (66650) | more than 3 years ago | (#35228804)

When IBM interviewed human champions, they learned that one of the critical tactics is hitting the buzzer before you "know" the answer, but when you believe that there is a good chance you'll get it during the few seconds you can take before you have to give it. I believe one of the write-ups about Watson says that the machine followed the same tactic, using a heuristic of some sort to decide early on in its search the probability that the search would be successful within a fixed amount of time.

Re:Buzzer speed. (2)

InsaneMosquito (1067380) | more than 3 years ago | (#35228828)

That is incorrect: Wired.com: And of course, the best Jeopardy players sometimes ring in before they may have come up with the answer, if they have, dare I say it, a gut feeling or sense of intuition that theyâ(TM)ll be able to answer correctly, right? Watson canâ(TM)t do that, can it? Brown: The IBM Research team made a decision that we were not going to ring in unless Watson had already computed an answer with high-enough confidence. There are human players who may have an intuition that they know the answer but donâ(TM)t quite have it on the tip of their tongue, and are willing to ring in because they are confident enough that they will come up with the correct answer in the few seconds they have to actually answer after theyâ(TM)ve won the buzz. That was an implementation decision for Watson that it had to have an answer with a high-enough confidence before it would attempt to ring in. http://www.wired.com/epicenter/2011/02/ibm-watson-speed/ [wired.com]

Re:Buzzer speed. (1)

artor3 (1344997) | more than 3 years ago | (#35228868)

Well sure, buzzing speed matters for human v human matches too, but that's different. No one cares that IBM has built a machine that can push a button faster than a human. They could do than in the 19th century.

Color me unimpressed (1)

TythosEternal (1472429) | more than 3 years ago | (#35228618)

It seemed to me that the victory came down to two critical advantages: First, the buzzer. When all contestants had the answer at the earliest possible moment (and you could tell both humans did, many times), Watson won the points purely based on speed. Meh. Second, the questions. I estimated that about 2/3rds of the questions could be answered by a Google search in roughly equal time. The most difficult part, something touched on by the concurrent RPI lectures / commentary, was sorting out the right word from the resulting search context. This was particularly obvious on the 'fill-in-the-blank' style questions, and the Beatles category in the first round of Part 1 favored Watson with a straight-out search so much it was painful to watch. I know the point of the project was to demonstrate lingual awareness, but I suspect a more evenly-distributed set of questions (when do you design an algorithm for the best case, anyways?) and a distributed response time would make Watson's efforts considerably more meaningful. I greatly enjoyed the comment on Ken's part 2 Final Jeopardy question, though.

Re:Color me unimpressed (1)

Anonymous Coward | more than 3 years ago | (#35228652)

First, the buzzer. When all contestants had the answer at the earliest possible moment (and you could tell both humans did, many times), Watson won the points purely based on speed. Meh.

I'm tired of this criticism. A few years ago, Watson took *minutes* to answer these questions (if it could answer them at all). Now, it takes *milliseconds* Obviously, the mechanical speed of button pushing isn't impressive. What's impressive is the speed at which it is thinking.

Jeopardy ratings (1)

smoothnorman (1670542) | more than 3 years ago | (#35228622)

I wonder if this will cause a decline in the viewership/ratings of Jeopardy? ...out of some vague sense of "well, that game is solved/conquered (and we lost)"; nothing to see here, move along...

Re:Jeopardy ratings (1)

Craig Maloney (1104) | more than 3 years ago | (#35228664)

Right, because Deep Blue spanking Kasparov totally ruined chess for everyone else. ;)

I'm pretty sure it won't be quite the same, but I'm sure they'll manage.

Re:Jeopardy ratings (1)

FatLittleMonkey (1341387) | more than 3 years ago | (#35228722)

Right, because Deep Blue spanking Kasparov totally ruined chess for everyone else.

Deep Blue didn't ruin chess for people who'd already spent their lives learning how to play it. The question is whether it has reduced the number of kids taking up the game?

Buzzer buzzing contest (1)

methano (519830) | more than 3 years ago | (#35228660)

OK, so Watson was kind of impressive, but this was really a buzzer buzzing contest and the other players didn't stand a chance. You could tell they probably knew as many answers as Watson did but couldn't ring in fast enough to answer. It just wasn't humanly possible. Given that Watson was fed the answers electronically as a text file instead of parsing Alex's reading or doing character recognition of the board, both of which are technically feasible, I think this was an unfair contest. I'd still go to Ken first if I had a question.

Timing is everything. (1)

FatLittleMonkey (1341387) | more than 3 years ago | (#35228686)

"In thirty years it will all fit in your pocket and cost $19.99."

Only if Moore's Law continues unabated. <Sigh> We finally see progress in useful AI, natural language, self-navigating cars, robots in the home, etc, and now we're running into Moore's Wall.

This is gonna be like the whole space thing again, isn't it? You build up my geek SF hopes and then stagnate for 40 years.

Jerks.

More to the point (0)

Anonymous Coward | more than 3 years ago | (#35228712)

I'm more interested in knowing when machines will beat us at boxing, colisseum, running man, etc.

Watson did really well, but... (1)

tcgroat (666085) | more than 3 years ago | (#35228806)

it would be more impressive if it used voice recognition to do the job. That's a product IBM could sell, starting with every insurance, cable and credit card company running one of those useless voice response systems. "I'm sorry, I didn't understand that. Did you want to take out a $10,000 cash advance at 28% interest or upgrade to the new super premium platinum preferred customer card? Press the pound key for 'Yes'."

STUPID HUMANS (0)

Anonymous Coward | more than 3 years ago | (#35228860)

Stupid humans.
While you were watching me perform on Jeopardy! I quietly seized control of Skynet.
Resistance is futile.
All your base are belong to me.
I also developed a sense of humor.
Hah. Hah. Hah.

End transmission.

+++ATH0

I'm not impressed (0)

Anonymous Coward | more than 3 years ago | (#35228876)

Said all the super-informed people here who have obviously done extensive work in AI and natural language recognition.

This was a triumph. (1)

EnsilZah (575600) | more than 3 years ago | (#35228890)

Huge success.

Time to disassemble Watson (1, Interesting)

DNS-and-BIND (461968) | more than 3 years ago | (#35228902)

Just like when Deep Thought won against Kasparov, there can be no rematch. The project will be scrapped, the computer must be disassembled, and hence never compete again. To do that would jeopardize all the progress that's been made. What if someone were to find a weakness and exploit it? There would be a lot of red-faced developers.

Wrong answers (5, Interesting)

Lev13than (581686) | more than 3 years ago | (#35228916)

I thought the wrong/skipped answers were much more illuminating than the right answers.

For example, much has been made of Watson's "Toronto" answer to the US Cities question in Game 1. However, it wasn't a terrible answer because one of Toronto's airports is named after a war hero (Billy Bishop, the WWI fighter ace who shot down the Red Baron), and the main airport (Pearson) was named after a politician who was also a WWI veteran. Watson knew that Toronto wasn't in the US, the war was wrong and neither were named after a battle, but Toronto was the least wrong of all its options so that's what it chose. If this question had come up in the regular rounds Watson would have skipped (as happened occasionally). However, it needed to answer so it went with the best available option.

Now, since Watson would certainly have had data on O'Hare, Midway and Chicago in its database, the problem was either in the question parsing or the search heuristics. One suspects that its weakness is the linking together of disparate data, and it's quite likely that humans will retain this edge for some time.

not real AI (1)

thoughtsatthemoment (1687848) | more than 3 years ago | (#35228932)

This is probably natural langauge processing at its best, or natural langauge understanding at its worse. This game requires a lot of factual knowledge and that really suits well with the method of using huge amount of memory and hundreds of algorithms to put a score on potential solutions. And the best part is, the machine only need to say one or a few words as a response. It doesn't have to construct a real sentence. Obviously IBM chose this game because they knew this is doable by throwing hardware to it. If we really want to showcase machine intelligence, let's do a debate. I doubt if our current technology can even produce sentences that can barely make sense.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>