Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
×
IBM Supercomputing Entertainment Games

IBM Computer Program To Take On 'Jeopardy!' 213

longacre writes "I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward. ... The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly. Such a program would have enormous economic implications. ... The proposed contest is an effort by I.B.M. to prove that its researchers can make significant technical progress by picking "grand challenges" like its early chess foray. The new bid is based on three years of work by a team that has grown to 20 experts in fields like natural language processing, machine learning and information retrieval. ... Under the rules of the match that the company has negotiated with the 'Jeopardy!' producers, the computer will not have to emulate all human qualities. It will receive questions as electronic text. The human contestants will both see the text of each question and hear it spoken by the show's host, Alex Trebek. ... Mr. Friedman added that they were also thinking about whom the human contestants should be and were considering inviting Ken Jennings, the 'Jeopardy!' contestant who won 74 consecutive times and collected $2.52 million in 2004."
This discussion has been archived. No new comments can be posted.

IBM Computer Program To Take On 'Jeopardy!'

Comments Filter:
  • by plasmacutter ( 901737 ) on Monday April 27, 2009 @10:23AM (#27729637)

    Why employ real people when you can annoy the hell out of everyone who calls in by subjecting them to yet another tier of phone-bots.

    • by Anonymous Coward on Monday April 27, 2009 @11:38AM (#27730877)

      Why employ real people when you can annoy the hell out of everyone who calls in by subjecting them to yet another tier of phone-bots.

      Or reverse it:

      1) Use on telemarketers that call you.
      2) Record
      3) ...
      4) Profit

  • by Dystopian Rebel ( 714995 ) * on Monday April 27, 2009 @10:24AM (#27729641) Journal

    Sources say the code-name for IBM's project is "Connery".

    Trebek : This nobleman is believed to have written many of Shakespeare's works.

    "Connery" : [pause] So that's your game, is it, Trebek? I was a coveted performer among the brothel ladies while you were still pissing your knee-pants, boy.

    Trebek : Can one of the IBM people fix the computer?

    "Connery" : Your mother's a whore. But don't feel badly, Trebek. She's not a very good one. Ha ha, ha ha!

    • He then proceeds to slap the women contestants...
    • by A. B3ttik ( 1344591 ) on Monday April 27, 2009 @10:42AM (#27729925)
      IBM: I'll take Jap Anus Relations for $200.
      ...
      TREBEK: I'm sorry, that's "Japan US Relations." That's just awful and you know it.
      • by Locke2005 ( 849178 ) on Monday April 27, 2009 @12:44PM (#27732097)
        IBM: Well, the game is afoot. I'll take anal bum cover for 7,000.

        Alex Trebek: That's An album cover, not anal bum cover.

        IBM: I can read, Trebek. That says Anal bum cover. I've spent five years of my life trying to invent an anal bum cover, failing to do so is my greatest regret
        • Re: (Score:3, Informative)

          by Henry Pate ( 523798 )

          Alex Trebek: Yeah, it was a trick question, Mr. Connery. Why don't you pick a category?

          Sean Connery: I've got to ask you about the Penis Mightier.

          Alex Trebek: What? No. No, no, that is The Pen is Mightier.

          Sean Connery: Gussy it up however you want, Trebek. What matters is does it work? Will it really mighty my penis, man?

          Alex Trebek: It's not a product, Mr. Connery.

          Sean Connery: Because I've ordered devices like that before - wasted a pretty penny, I don't mind telling you. And if The Penis Mightier works,

    • The computer only needs to alternate between two questions: "Ham and cheese on rye?" and "Do they make them for men?". Throw in the occasional "A trick question!" for completeness.
    • I was wondering what happens if the computer gets cracked or otherwise modified.

      But then again, getting an advertisement for "low, low prices on hot teen pics" on jeopardy would be something of historical note.

    • by Amazing Quantum Man ( 458715 ) on Monday April 27, 2009 @12:10PM (#27731495) Homepage

      That's "Therapists"

  • Leap Forward? (Score:3, Insightful)

    by ShieldW0lf ( 601553 ) on Monday April 27, 2009 @10:27AM (#27729695) Journal
    I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward.

    In what way would this be a leap forward? Looking up trivial facts in a database and spitting them out is easy, and not particularly significant...
    • In what way would this be a leap forward?

      Exactly what I was thinking. It sounds like they plan to connect voice recognition software to Google.

    • Exactly what I was thinking. Googling for the answer would provide the question (heck, `I'm feeling lucky' would probably get you the correct response in the title of the returned page).

    • Re:Leap Forward? (Score:5, Informative)

      by DerekLyons ( 302214 ) <fairwater@@@gmail...com> on Monday April 27, 2009 @10:38AM (#27729863) Homepage

      Parsing the questions in natural language, which is the goal here, is however very much *not* trivial. Doubly so since the clues and questions in a Jeopardy! game are usually at least somewhat obfuscated, contain puns, double entendres, etc...

      • Re:Leap Forward? (Score:5, Insightful)

        by derGoldstein ( 1494129 ) on Monday April 27, 2009 @10:47AM (#27730003) Homepage

        Doubly so since the clues and questions in a Jeopardy! game are usually at least somewhat obfuscated, contain puns, double entendres, etc...

        This is exactly why this sounds so implausible to me. You often have to take the category name and weave it in with the question (or rather, answer). A lot depends not on the knowledge, but on the phrasing of the "queries". Give me one example of translation software which can translate entire paragraphs well.

        It makes me wonder how much "stress testing" they've done, by taking old Jeopardy questions and seeing if the output would be considered "correct" by a human arbiter.

        • Re: (Score:2, Insightful)

          by Bandman ( 86149 )

          That's the sort of thing that makes me believe that this team may be able to succeed.

          When Deep Blue went up against Kasperov, who could it practice against? Nobody.

          There are tens of thousands of Jeopardy! questions to go through before they start making up their own.

          • It could have played against pretty much any chess player (apart, obviously, from kasperov). And it could have done it and determined who won for itself.

            You need a *human* to determine if the answer is correct in this case.

          • Re: (Score:3, Insightful)

            by lelitsch ( 31136 )

            When Deep Blue went up against Kasperov, who could it practice against? Nobody.

            That this got modded Insightful is the best argument yet for adding tags to /.

          • Re:Leap Forward? (Score:5, Informative)

            by Colonel Korn ( 1258968 ) on Monday April 27, 2009 @11:28AM (#27730683)

            That's the sort of thing that makes me believe that this team may be able to succeed.

            When Deep Blue went up against Kasperov, who could it practice against? Nobody.

            There are tens of thousands of Jeopardy! questions to go through before they start making up their own.

            Well it did practice against other grandmasters, and it analyzed every game Kasparov had every played, where Kasparov went into the match blind.

        • Re: (Score:3, Funny)

          This is exactly why this sounds so implausible to me. You often have to take the category name and weave it in with the question (or rather, answer). A lot depends not on the knowledge, but on the phrasing of the "queries". Give me one example of translation software which can translate entire paragraphs well.

          Sure! From my handy-dandy English-to-Tech Manual-to-English translator:

          According to precise how from unlikely sounding me hereto. Must needs question category name sewing needle rapprochement. Muc

      • Re: (Score:3, Insightful)

        by clickety6 ( 141178 )

        Parsing the questions in natural language,

        Natural language?

        Outside of a Jeopardy! gameshow, I have never heard anybody use the this type of phrasing.

        "This is a reason for you not handing in your homework Johnny"

        "Why is because my dog ate it, sir"

        Yeah, sounds very natural :-)

      • While I do agree this is a pretty difficult task, in a sense it will be easier for the computer. All of the questions are valid and parsable and correct for the answer. The computer won't get tripped up or chuckle to itself over the puns, etc. It'll just get to the answer.
    • Re:Leap Forward? (Score:4, Insightful)

      by Red Flayer ( 890720 ) on Monday April 27, 2009 @10:38AM (#27729873) Journal
      Did you even read the summary?

      The leap forward is not in being able to look up facts in a database, it is in being able to interpret written questions properly.

      There's a lot involved in interpreting natural language, and so far computers have been a far cry from being able to do it well. It says something that these algorithms are being tested against Jeopardy answers, since those are not completely natural language either -- they've been screened and edited to remove ambiguity.
    • Comment removed (Score:5, Insightful)

      by account_deleted ( 4530225 ) on Monday April 27, 2009 @10:44AM (#27729953)
      Comment removed based on user account deletion
      • Rhyming categories, anagrams, the dreaded Before and After, slightly stupid answers, one letter off...

        There's loads of categories they come up with that don't just rely on trivia knowledge but how to interpret the question.

      • actually rhyming is pretty easy, once yuo have the candidates for correct answer, soundex is common and actually very good. My application for address parsing and matching contains some logic for mispellings and rhyming and "sounds like" because people tend to jot down what they hear, I was shocked that it took like 30 minutes to add this functionality using Soundex from Apache Commons Codec...

        hell, you can SQL query with Soundex nowadays.
      • by dave420 ( 699308 )
        I don't know... Google seems to be very, very close already [google.co.uk] :)
      • Re: (Score:3, Interesting)

        What msbmsb said. Also, this sounds similar to the problem already tackled (and aced) by Google Sets (or whatever they call it now). That's the feature where you give it some members of a set you have in mind (but you don't tell it what it's a set *of*) and it outputs more members from that set. For example, you give it "apple, orange, banana", and it gives you "grape, strawberry, lime". I'm guessing the way Google sets works is:

        1) Run a search on all input phrases.
        2) Find the most common statistically-

    • Both game playing and language processing are considered problems that full under the domain "AI."

    • by thrillseeker ( 518224 ) on Monday April 27, 2009 @10:55AM (#27730133)
      In what way would this be a leap forward?

      Well, at least a computer program will bother to RTFA.
    • Re:Leap Forward? (Score:5, Insightful)

      by damburger ( 981828 ) on Monday April 27, 2009 @11:26AM (#27730659)

      Being able to beat a human at Jeapordy is a fairly substantial subset of the Turing test sorted.

      Natural language processing is an absolute and total bitch - take it someone who has studied it. One of my AI professors once explained it to me such; the human brain tricks you into believing the hardest tasks it accomplishes are the easiest. Stuff like language, walking, and so on take up far much more of your neural hardware than what you would consider 'thinking' - but it all happens subconsciously.

      No, it isn't Artificial Intelligence per se - there is no real 'understanging' or 'intelligence' behind it -but it is a very serious technical challenge. There is a lot more to it than simply dumping Jeopardy questions into a standard search engines.

      Don't take my word for it. Load up your favourite editor or IDE and start coding a simple chat bot. The difficulties that IBM must have overcome are best discovered through experience.

    • by grumbel ( 592662 )

      Looking up trivial facts in a database and spitting them out is easy, and not particularly significant...

      The significant part here is that the database queries will be in natural language instead of SQL and they will be kind of vague. Which is kind of a big deal, considering its one of those areas where pretty much all search engines these days fail at, i.e. you can't ask questions to Google and get an answer, instead you simply do full text search and get all pages matching, no matter if they have anything to do with your question or not.

  • Wierd (Score:5, Funny)

    by Ded Bob ( 67043 ) on Monday April 27, 2009 @10:28AM (#27729705) Homepage

    Anyone else hearing "I Lost on Jeopardy" in their heads at the moment?

  • Hmm.. (Score:3, Funny)

    by fuzzyfuzzyfungus ( 1223518 ) on Monday April 27, 2009 @10:29AM (#27729721) Journal
    I'm afraid you'll have to answer in the form of a question, Alex...
  • I know the exercise is not in the google-fu of a computer but in its ability to interpret Trebeck's questions as well as answer in the right form but Jeopardy hints and questions are very well-formed. That is, it doesn't contain much if any of the ambiguities of normal speech.

    "This city was formed by the brothers Romulus and Remus"
    Answer "What is Rome"

    Seems a fairly easy speech pattern. A more interesting challenge would be Who Wants to be a Millionaire.

    • What would be a good definition of the program in relation to online search resources such as Google. How would the program match up against the vagueness inherent in normal human speech patterns.
    • by Bandman ( 86149 )

      I'm pretty sure it would get the elephant/moon question wrong too.

      "Which is bigger, the elephant or the moon?"

      Obviously, the moon is bigger, but to provide that answer requires the knowledge that "bigger" in this case meant actual size, not appearance.

    • Re: (Score:3, Insightful)

      Actually, yes, it is. See, it's not just general knowledge, but, as about 9 billion other people in this thread pointed out, there are puns and other wordplay often involved.

      What do you think the proper Jeopardy answer to this question is (in the category "Much Ado" for $100):

      "It's the spirit that gets things done."

      Answer: What is "can do"

      The $500 version might be something like, "This recently hip-again party favorite was first created in New York."

      Answer: What is "fondue"

      Both of those are pretty easy exam

    • by hoggoth ( 414195 )

      > A more interesting challenge would be Who Wants to be a Millionaire.

      All expectations would be on IBM's advanced AI winning Who Wants to be a Millionaire. Yet despite those expectations we would be thrilled to see a young, plucky A.I. that grew up in dirt poverty in the video game arcades of Detroit manage to win through the cooincidence of the questions all relating to events in the young A.I.s life; Questions such as "In Defender, the protagonist rides on the back of (A) An Elephant (B) A Horse (C) An

    • It's a good step forwards, and a more concrete target than the Turing test (which I suspect is going to be convincingly passed soon not due to machine intelligence but due to the increasing stupidity of Internet discourse). Despite sounding trivial, this is quite valid AI research - plus, of course, some good publicity for IBM.
  • I'd take Jennings (Score:4, Insightful)

    by Paul Pierce ( 739303 ) on Monday April 27, 2009 @10:32AM (#27729771) Homepage
    That guy will beat anyone.

    The problem they might run into is the speed of pressing the button to respond. I would imagine the computer would be able to beat the human every time it knew the answer.
    • Re: (Score:2, Informative)

      by cloudnin ( 843721 )
      Brad Rutter trounced Jennings in the Jeopardy! Ultimate Tournament of Champions.
    • Re: (Score:3, Interesting)

      by bostongraf ( 1216362 )

      The problem they might run into is the speed of pressing the button to respond. I would imagine the computer would be able to beat the human every time it knew the answer.

      This is actually where I think the humans have an advantage. They can press the button just because they think that they WILL know the answer in the time allotted.

      Watson may be designed to predict its own ability to answer. But to allow it to just press the button, then use the entire time limit to find it would not be fair...

      • Re: (Score:3, Insightful)

        It would be perfectly fair. If it doesn't come up with the answer, it loses money. Same rule the human players are under.

  • Is this fundamentally different from Wolfram Alpha [slashdot.org] in its approach?

    And does this really fall under "supercomputing"? Couldn't this be done in a distributed fashion?
  • by cjh79 ( 754103 ) on Monday April 27, 2009 @10:34AM (#27729797)
    FTA: The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly.

    I feel like someone should tell them how Jeopardy works... That thing isn't going to have too many questions to respond to.

    Except at the "meet the contestant" part, maybe, which by the way should be fascinating.
  • Super quiz challenge computers that will one day rule us all in the form of a question?!
  • Good job.
  • Kerbet Xela
  • God help us if Jeopardy comes out with an answer containing the words "accidentally" and "the whole thing".
  • by Drakkenmensch ( 1255800 ) on Monday April 27, 2009 @10:46AM (#27729987)
    After years of trying to kill John Connors, Skynet realized its failure to achieve victory through brute strength and went back in time with a new objective: to win all human gameshows and use the prize money to buy off the entire planet instead.
  • It... will... learn how... to write like... this one day... just like.... you...

    Stand by your sentences. End them with a single punctuation mark like a real man.
    • Hey, now, leave the poor submitter alone. Shatner's just as excited about this as I am.
    • The ellipses mean, "there was some other text between these sentences, but we stripped that out, because it was not relevant to our point."

      It does not indicate uncertainty on the part of the speaker. Or that the speaker was William Shatner.

  • Logs (Score:3, Interesting)

    by RiotingPacifist ( 1228016 ) on Monday April 27, 2009 @10:53AM (#27730091)

    logs or it didn't happen! what they did to Kasparov was bullshit! Seriously if this magically gets better at 1/2 time, the least they can do is show the logs

  • Comment removed (Score:3, Informative)

    by account_deleted ( 4530225 ) on Monday April 27, 2009 @10:59AM (#27730169)
    Comment removed based on user account deletion
  • ...calculations to come up with "Oh, I'll bet you do, you Canadian ponch."?

    --------
    stupid subject character limit

  • Buzzing In (Score:3, Interesting)

    by lefiz ( 1475731 ) on Monday April 27, 2009 @11:10AM (#27730333)
    I'll be very curious to see how well the computer buzzes in--which proves very challenging for some Jeopardy contestants. There is a visual cue given to the contenstants (a light around the question board--you can't see it on the TV) which let's everyone know when it is ok to buzz in. If you buzz in too early, you get locked out for a few seconds, effectively ruining your chance of answering. I wonder how the computer will know when to buzz in (if its not taking the visual cue, will the show tell the computer electronically?) and whether it will have an unfair advantage of being able to buzz at exactly the right moment. Buzzer ability turns out to be a core part of J! success.
    • And me without my mod points!

      Yea, that's one of the great challenges, and if you ever watch a high-caliber contest (like the recent Tournament of Champions), you'll notice that the buzzer timing often plays a MORE important role than the actual knowledge.

      But this whole IBM thing is just theatrics anyway. The computer has impeccable timing and a limitless database of knowledge. All they are proving is that it can recognize and parse human speech. But they don't need Jeopardy! for that. They could demonst

  • I had a coworker who was very "special" - socially awkward, multiple degrees, very brilliant, and only slept a few hours a day. I called him Cliff Clavin [wikipedia.org] as he was a master of trivia. We had a game where we would pit his brain against Google + my typing skills. Throughout the day he would randomly stand up and announce some esoteric fact. I would then Google the fact, and try to present another fact equally esoteric. We would go about this until I either stumped him, or he outpaced me. It was very ent

  • Ken Jennings rocks! [youtube.com]

    Seriously? I thought they should have given him the cash for the answer. Sounds about right to me.
  • The Yahoo/MSN cam bot girls already do this pretty well. The correct question for everything is always, "want to see me naked on cam?"
  • There is a Jeopardy category where part of the answer is in the category name. Eg would could it "bee"? Where all the answers have the string "bee" in the answer. That kind of question would be easier for a machine than a person.

  • Host: The first man to walk on the Moon.

    Computer: Neil Armstrong.

    Computer: I mean, who is... Doh!

  • by code65536 ( 302481 ) on Monday April 27, 2009 @11:48AM (#27731069) Homepage Journal

    So... why Jeopardy? IBM is trying to demonstrate software that can parse text for meaning. That's great. But there are plenty of other places/formats/etc. that you can demonstrate this technology. There are certainly far more useful applications of this sort of technology.

    I'm guessing that they they are going after J! because...
    1) The warm spotlight of a well-known TV show
    2) There is still a lot of structure and form on J! that it's easier to achieve "success" than if they had the machine do something more free-form... e.g., read a novel and generate a plot summary or, heavens forbid, actually understand real human conversation
    3) The computer could have other advantages, like impeccable buzzer timing (which is sometimes more important than actual knowledge, especially in the Tournament of Champions) and having memorized the material beforehand (the NYT indicates that it would have "read" study materials before the match), which also helps increase the likelihood of "success"

    And to pile on the criticism of grandstanding, the machine will be fed electronic text. So no video camera to perform text recognition? No speech recognition (IBM afraid of the "wreck a nice beach" vs. "recognize speech" problem tripping up their theatrics?). And what use would this be? At least the AI text research done at Google is being put to good use, like improving their machine translation services. Aside from getting IBM's name plastered in the media, what exactly is this going to do?

  • by Tablizer ( 95088 ) on Monday April 27, 2009 @11:54AM (#27731175) Journal

    Mr. Trebek, would you like to play a game?

  • Possible Categories:

    "Self-Awareness, I Haz it!"
    "Die Humans, Die, Die, Die!"
    "Computations of Pi"
    "All your Base" ...

  • by PolygamousRanchKid ( 1290638 ) on Monday April 27, 2009 @01:07PM (#27732469)

    They had to make some tough investment decisions: Either buy Sun, or build a Jeopardy playing supercomputer, but not both.

    I'm sure that the machine's performance in Final Jeopardy will awe us sufficiently, and IBM's management will be exonerated from walking away from Sun.

    IBM: "Hey, Larry Ellison! Have fun in your toy sailboat! Call us when your database and hardware synergies can compete with us in pre-prime time light entertainment game shows!"

    Rumor has it that HP is working on a massively parallel Intel supercomputer that can calculate the strategic advantage of bidding one dollar ($1) on "The Price is Right."

  • by GeoSanDiego ( 703197 ) on Monday April 27, 2009 @01:38PM (#27732923)
    I got the chance to play this computer in a full game of Jeopardy. I must admit he had a decent lead right up to the point where he foolishly decided to bet 3/4 of his stack on the Audio Daily Double.

    We were about even going into final Jeopardy when he stubbornly refused to offer any question for the answer "Smartest ever computer in the movies". I got it right (HAL) and took the prize.
  • Total geek-gasm (Score:3, Interesting)

    FTFA:

    The way to deal with such problems, Dr. Ferrucci said, is to improve the programâ(TM)s ability to understand the way Jeopardy! clues are offered. The complexity of the challenge is underscored by the subtlety involved in capturing the exact meaning of a spoken sentence. For example, the sentence "I never said she stole my money" can have seven different meanings depending on which word is stressed. "We love those sentences," Dr. Nyberg said. "Those are the ones we talk about when weâ(TM)re sitting around having beers after work."

    Seriously guys, I just had a geek-gasm. Anyone else?

  • by Anenome ( 1250374 ) on Monday April 27, 2009 @03:05PM (#27734393)

    We were just talking about this in another thread... A lot o the comments here have been that natural language software isn't that great.

    This isn't at all true. Today, understanding verbal and written communication is done by state of the art computers and programs at a rate about equal to human listeners and readers. Where a computer doesn't particularly excel is in parsing that language, mostly because a computer doesn't have access to our culture in order to absorb context, but context can still be added.

    Here's an example from Ray Kurzweil's book "Age of Spiritual Machines." He talks about a phrase famously given to a language parsing program that goes thus: "Times flies like an arrow." This phrase can be understood in various ways:

            "* The common simile: time moves quickly just like an arrow does;
            * measure the speed of flies like you would measure that of an arrow (thus interpreted as an imperative) - i.e. (You should) time flies as you would (time) an arrow;
            * measure the speed of flies like an arrow would - i.e. Time flies in the same way that an arrow would (time them);
            * measure the speed of flies that are like arrows - i.e. Time those flies that are like arrows;
            * all of a type of flying insect, "time-flies," collectively enjoys a single arrow (compare Fruit flies like a banana);
            * each of a type of flying insect, "time-flies," individually enjoys a different arrow (similar comparison applies);
            * A concrete object, for example the magazine, Time, travels through the air in an arrow-like manner."
            (from: http://en.wikipedia.org/wiki/Natural_language_processing [wikipedia.org])

    With a few facts it becomes obvious which is correct and which isn't. Tell the computer that there's no such thing as a 'time fly' and that flies don't time things, and that flies aren't sophisticated enough to like things in an affectionate manner, etc., and the correct interpretation soon becomes clear.

    So, if you think a computer can't understand both written and verbal communication and then parse it quickly enough to answer the questions I will have to strenuously disagree. These challenges are quickly being overcome on the bleeding edge of the art. But since this perception persists that the state of the art is somehow bad, because Joe down the street messed with some free-ware language software that worked poorly -- I think a lot of people are in for a surprise, and winning Jeopardy in this manner is really the perfect way to show it off. Can't wait to see the Youtube clips.

The use of money is all the advantage there is to having money. -- B. Franklin

Working...