Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Inside the 2012 Loebner Prize

timothy posted more than 2 years ago | from the actually-they're-just-toying-with-us dept.

AI 68

An anonymous reader writes "Not a single judge was fooled by the chatbots in the 2012 Loebner Prize, which was won by the bot Chip Vivant. According to a journalist who was a human decoy in this year's Turing Test, interactions with the humans was a tad robotic while the bots went off on crazy tangents talking about being a cat and offering condolences for the death of a pet dragon."

cancel ×

68 comments

This is who is making our stock trades now (0, Offtopic)

crazyjj (2598719) | more than 2 years ago | (#40027407)

No, seriously [wikipedia.org] . I just hope they can do better than:

"I want to invest in corn futures"

I've found 3 corn fields near you.

"No, corn FUTURES"

Here is the Wikipedia page for Back to the Future.

"Fuck it, just dump everything into gold"

I've found 2 gold mines and 5 dumps nearby.
 

Re:This is who is making our stock trades now (0)

Anonymous Coward | more than 2 years ago | (#40027443)

Re:This is who is making our stock trades now (-1, Flamebait)

Anonymous Coward | more than 2 years ago | (#40027507)

niggers investing in crackpipe futures

Re:This is who is making our stock trades now (1)

Anonymous Coward | more than 2 years ago | (#40027635)

From a programming perspective, algo trading is simple math, not AI. The challenges lie in A) Processing the ungodly amount of data necessary in a short enough time period to be relevant and B) Coming up with a formula that accurately identifies good trades. There is literally no connection to AI, you don't want the program to "pick" anything, only to quickly identify predefined situations.

Re:This is who is making our stock trades now (1)

crazyjj (2598719) | more than 2 years ago | (#40027707)

From a programming perspective, algo trading is simple math, not AI.

I don't think you appreciate just how complex trading software has become. The big players have moved WAY beyond "If stock X dips below Y price, then sell."

Re:This is who is making our stock trades now (2)

Dr_Barnowl (709838) | more than 2 years ago | (#40027815)

They make financial instruments complex on purpose as well. Can't understand it? Can't tax it!

Then your system has to model trading these things... you either end up with fractal complexity, or dangerous assumptions...

Re:This is who is making our stock trades now (0)

Anonymous Coward | more than 2 years ago | (#40027907)

Actually, I think you read too many articles written by reporters who don't understand algo trading and just sensationalize the hell out of it because it sounds complex. I work for a company that makes financial software that deals extensively with algo trading applications. It is absolutely not "If stock X dips below Y price, then sell." but it's also nothing resembling AI. First off, it's not like these things run completely unattended. Yes, they autoexecute, but actual traders are analyzing the markets and making adjustments to price differentials, feeding in lists of valid tickers the algo is allowed to trade and so on. Really all that most algos are doing is taking a list of tickers, waiting for times when those tickers are highly liquid, and then rapidly trying to penny the market. The math is fairly simple, but implementing it in a method that's fast enough to actually succeed is extremely difficult.

Re:This is who is making our stock trades now (2)

Sarten-X (1102295) | more than 2 years ago | (#40028181)

I currently work at a financial company, and I agree. One of our algorithms for "buy or sell" currently relies on a few hundred data points for a few hundred securities. The algorithm boils down to "did this particular statistic recently rise or fall more than a certain amount based on its history?". Sure, the code is a little hairy in places, but not too bad with adequate commenting. There is nothing I'd consider AI, as the program doesn't attempt to learn. When the investment folks see something that should change, they tell me.

Re:This is who is making our stock trades now (1)

omitsura (1503131) | more than 2 years ago | (#40031125)

IV surface modeller? Vol below, buy; vol in 90th percentile sell?

Re:This is who is making our stock trades now (1)

TheLink (130905) | more than 2 years ago | (#40030237)

Meh, from what I see if you're one of the favoured companies when you screw up big time they'd roll back the trades for you. I bet even someone like me can make a lot of money if my trades got rolled back whenever I screwed up big. And if you screwed up really big you get a bailout and keep your bonus.

There was also that infamous case when humans outsmarted the algo but they got prosecuted, convicted and lost their profits: http://www.ft.com/cms/s/0/f9d1a74a-d6f3-11df-aaab-00144feabdc0.html [ft.com]

Why should they get prosecuted? Why should the algo users keep the money if it wins when they also get to keep the money when it loses? Sounds stupid and unfair to me.

Re:This is who is making our stock trades now (1)

jeffmeden (135043) | more than 2 years ago | (#40027999)

From a programming perspective, algo trading is simple math, not AI. The challenges lie in A) Processing the ungodly amount of data necessary in a short enough time period to be relevant and B) Coming up with a formula that accurately identifies good trades. There is literally no connection to AI, you don't want the program to "pick" anything, only to quickly identify predefined situations.

What do you exactly think is the difference between current AI and "an algorithm that quickly identifies predefined situations"? Aside from the obvious example of natural language interaction that was hilariously pointed out in the OP. AI is nothing more than an (occasionally very complex) attempt at hacking English into a variety of numbers, and then performing math on them in interesting ways (what words are statistically likely to show up near what other words, and in what order, for example.) Algorithmic trading might have more specific activity at the low levels (like monitor 1000 different asset prices for a specific pattern) but that is hardly unlike an AI, its output is just handled differently.

Re:This is who is making our stock trades now (2)

Samantha Wright (1324923) | more than 2 years ago | (#40028691)

You and the GP keep using the word "AI". I do not think it means what either of you think it means.

Algorithmic trading does indeed involve a great deal of classical machine learning, which is one area of focus within the very broad field of artificial intelligence. What you're ascribing to "AI" is the specific goal of artificial human intelligence, regularly granted the misnomer of general artificial intelligence. The example you gave would be of more interest to translators of unknown languages than to people trying to create an artificial brain.

This was one of the many misconceptions about intelligence that muddied AI efforts in the twentieth century, incidentally. Parsing and producing language isn't the end-all of everything like logicians once believed; other factors, like knowledge representation, planning, and creativity also must be considered. These have (for the most part) redefined the AI landscape as it's been introduced; KR and fuzzy logic led to expert systems in the 80s (which were pretty successful in some contexts but an absolute pain to build) and classical planning led to a wide variety of different manifestations like the intelligent agents of the 90s (which weren't so intelligent), automation of flight planning, and enemy AI in real-time strategy games. Computational creativity has yet to really have its time in the sun, being mostly displaced by a vastly-improved second round of more involved linguistic analysis called natural-language processing (culminating in things like IBM's Watson, although that involved a measure of creativity as well.)

Re:This is who is making our stock trades now (0)

Anonymous Coward | more than 2 years ago | (#40027965)

So you're saying algorithmic trading is done by chatbots? I think I've spotted the trollbot here.

I've seen things you wouldn't believe (5, Funny)

Serious Callers Only (1022605) | more than 2 years ago | (#40027435)

Of course, all the *real* chatbots are too busy with their day job - posting spam to twitter and pumping out mass emails.

Closer to home (3, Funny)

Theophany (2519296) | more than 2 years ago | (#40027589)

Hello sir, have you heard of CleanMyPC...?

Re:Closer to home (2)

tepples (727027) | more than 2 years ago | (#40028501)

I haven't heard of CleanMyPC, but I've heard of MyCleanPC [xubuntu.org] . If you back up your files, wipe everything off your PC, and install MyCleanPC [xubuntu.org] , your PC will be clean. Then you can install thousands of free applications [ubuntu.com] and reinstall many applications that are already your favorites [winehq.org] . The last time I cleaned a relative's PC that had about three fake antiviruses [wikipedia.org] on it and embedded deeply, I did just that: wiped it, installed MyCleanPC [xubuntu.org] , and moved a couple things around to make it look like it used to, except cleaner. It ended up even cleaner than it was when Dell sold it to her.

Clean, precise, pangolin-powered. MyCleanPC [xubuntu.org] .

Re:Closer to home (0)

Anonymous Coward | more than 2 years ago | (#40028781)

The CleanMyPC chatbot has no reply! Take that "Theophany"! :)

Re:Closer to home (1)

Coren22 (1625475) | more than 2 years ago | (#40033035)

Offtopic? Just used my points up or that would have gotten insightful. That was a great post.

Re:Closer to home (1)

mcgrew (92797) | more than 2 years ago | (#40040291)

I woosh the moderators would have paid attention to where the link led.

Re:Closer to home (1)

tepples (727027) | more than 2 years ago | (#40047891)

I woosh the moderators would have paid attention to where the link led.

So I guess the "precise pangolin" part wasn't enough. I wonder what I should do to make it clearer before I install the last line of the post as my new sig sometime in June.

Re:Closer to home (1)

Hognoxious (631665) | more than 2 years ago | (#40039055)

Why do you cower?

MyCleanPC = stagnated.

Re:Closer to home (1)

mcgrew (92797) | more than 2 years ago | (#40030481)

That was the best satire of that spam I've seen yet, congrats! It actually made me laugh, unlike most of them.

Re:Closer to home (1)

Theophany (2519296) | more than 2 years ago | (#40031583)

I was tempted to do a LucasArts reference... "I'm selling this fine antivirus software..." but couldn't make it work...

slashdot should charge for the chatbot testing. (3, Funny)

infonography (566403) | more than 2 years ago | (#40027537)

This is funny since Slashdot has been the main testing ground for chatbots. We have all had to read posts from them here, do you really think all Anonymous trolls are really people? BTW. most of my enemies list are chatbots Real people would not be nearly as stupid as these clowns.

Re:slashdot should charge for the chatbot testing. (0)

Anonymous Coward | more than 2 years ago | (#40027629)

Yes, pin it on the ACs. Like there's a reason why an account user couldn't be a chatbot.

Re:slashdot should charge for the chatbot testing. (0)

Anonymous Coward | more than 2 years ago | (#40027771)

It's started; they're trying to round the ACs up into death camps. You saw him say it. We're "not even really people" now, apparently.

First they came for... blah blah tl;dr and then there was nobody left for me!

Re:slashdot should charge for the chatbot testing. (-1)

Anonymous Coward | more than 2 years ago | (#40027781)

This is funny since Slashdot has been the main testing ground for chatbots. We have all had to read posts from them here, do you really think all Anonymous trolls are really people? BTW. most of my enemies list are chatbots Real people would not be nearly as stupid as these clowns.

yes they would never make nigger jokes if they were people!

Why was the wheelbarrow invented? To teach niggers to walk on their hind legs.

I AM A COMPUTER. THIS WILL BE MODDED DOWN. BUT THIS JOKE WAS FUNNY. HAH. HAH. HAH.

Re:slashdot should charge for the chatbot testing. (2)

Jeng (926980) | more than 2 years ago | (#40027891)

If you are being paid to post you post what you are paid to post.

Money can buy stupidity.

Re:slashdot should charge for the chatbot testing. (1)

Coren22 (1625475) | more than 2 years ago | (#40033065)

How does that make you feel?

I must be hungry (4, Funny)

Lucas123 (935744) | more than 2 years ago | (#40027575)

At first glance, I read it as "Inside the 2012 Lobster Pie".

Re:I must be hungry (1)

Anne_Nonymous (313852) | more than 2 years ago | (#40027953)

Dang, now I'm jonesing some crawfish pie [cajuncrawfishpie.com] , and don't be stingy with the butter.

The Hungry Hungry Games (0)

tepples (727027) | more than 2 years ago | (#40028589)

But are you as hungry [youtube.com] as the hippos [woot.com] in The Hunger Games [huffingtonpost.com] ?

Re:The Hungry Hungry Games (1)

Hognoxious (631665) | more than 2 years ago | (#40039069)

I've eaten hippo. Not a hippo, obviously.

Re:The Hungry Hungry Games (0)

Anonymous Coward | more than 2 years ago | (#40042457)

Did she enjoy it?

Re:I must be hungry (1)

Kadagan AU (638260) | more than 2 years ago | (#40031731)

I read "Inside the 2012 Lobster Prize" multiple times today. I thought it was connected to "Deadliest Catch" somehow..

Siri? (5, Funny)

dutchwhizzman (817898) | more than 2 years ago | (#40027585)

Did they ask the bots what was the best smartphone? We all know it's a bot if they didn't answer the N900

Re:Siri? (-1)

Anonymous Coward | more than 2 years ago | (#40027621)

StarTAC for the win, retard.

This FP for gNAA (-1, Redundant)

Anonymous Coward | more than 2 years ago | (#40027591)

From a technical schemes. Frankly fear the reaper Approximately 90% of Jordan Hubbard that support Shouts To the vary f0r different racist? How is you have a play OUTER SPACE THE been looking for! as one of the suffering *BSD and its long term = 1400 NetBSD to work I'm doing, empire in decline, progress. In 1992, you can. No, numbers continue recent article put Shouts To the successes with the for election, I it will be among on an endeavour noises out of the volatile world of going to continue, wall: *BSD faces a some intelligent community. The It's going, Everyday...Redefine

Re:This FP for gNAA (5, Funny)

makomk (752139) | more than 2 years ago | (#40027837)

I'm kinda conflicted - is this off-topic or on-topic?

Re:This FP for gNAA (1)

rvw (755107) | more than 2 years ago | (#40028203)

I'm kinda conflicted - is this off-topic or on-topic?

I think you would be an excellent judge for next years competition.

Just so you know (-1)

Anonymous Coward | more than 2 years ago | (#40027673)

ONLY SEVERAL HUNDRED YEARS AGO, THE NEWLY CREATED SECRET DEADLY
GANGSTER COMPUTER GOD earnestly began its’ TOP SECRET OVER-
ALL PLAN of world-wide FRANKENSTEIN LIVING DEATH SLAVERY, to explore and control the
entire UNIVERSE. The Gangster Computer God concocted and even named its’ OWN
IDEAL COMMUNIST WORLD ORDER, namely the murder incorporated
organized crime deadly gangsterism impunistic world-wide military uniformed
mongrel federalism, COMMUNIST TOTALITARIANISM.

Sounds like a friend of mine (1)

PhilHibbs (4537) | more than 2 years ago | (#40027931)

went off on crazy tangents talking about being a cat and offering condolences for the death of a pet dragon

Seriously, if a remote chat started talking to me like that, I'd say "Oh, hi Kim, I didn't know you were online".

The Achilles' Heel of AI (5, Interesting)

NicknamesAreStupid (1040118) | more than 2 years ago | (#40028031)

A friend of mine, who long ago worked for Thinking Machines, explained the weakness, "It is all about maintaining state." A stateless AI is far easier than a stateful one. Once the machine has to retain state, the algorithms become logarithmically more complex. Therefore, the way to test a bot is to say something like, "Remember this phrase, 'pink elephant'. I'm going to ask you after we have talked a while.." Then have several exchanges and ask, "What was that animal I told you to remember?" Most humans (except Alzheimer patients) will have no trouble with it, but the machine will fail. It they add a piece of logic to catch obvious clues like this, then a slight mod such as "have you ever seen a pink elephant? . . . what animal was I talking about?" will usually defeat it.

Humans are actually very poor at remembering. Try to recall the color of the last Volkswagen you passed on the street. However, we have developed a natural ability to prioritize our memories based on context and our personal & social needs. We tend to remember most of what turns out to be relevant. Until AI develops a means to judge context, it will suffer the weakness of being out of touch with our reality.

Re:Maintaining State (1)

TaoPhoenix (980487) | more than 2 years ago | (#40030251)

While I am sure that your friend is mostly right, it ought to be easier for the bot to remember stuff than that.

These are programs, right? Just allocate some data. Without trying to pun Facebook, just keep a file on every person from the bot's perspective, so if it understood "remember this animal" at all, then it just sets "Judge1 Likes Pink Elephants".

I know "Devil is in the details" but I often feel I could design a chatbot that would never make certain kinds of mistakes. Getting totally lost, sure, that's the signature problem of AI, but "I am a cat", no.

Serious Question (2)

TedTschopp (244839) | more than 2 years ago | (#40028195)

Can someone please explain to me how to read the chat logs? I am confused as to the actual exchange that is going on. Which transcript is the Bot, which is the human and how am I to sync the two parts of the conversation up?

Re:Serious Question (1)

Fwipp (1473271) | more than 2 years ago | (#40028337)

Agreed - I am having no success piecing together these conversations.

Re:Serious Question (1)

bryan1945 (301828) | more than 2 years ago | (#40029173)

They are a pain trying to synchronize, since the judges and humans both try to trick each other, so they are not much more coherent than the bot convos. But the bots just eventually end up flailing about. They weren't kidding about "I am a cat" answer. I think the question was "What is your name?" I had to stop reading after the first 2 because people started looking at me funny wondering why I was laughing so much.

Re:Serious Question (0)

Anonymous Coward | more than 2 years ago | (#40028421)

It looks like the conversations go that the one on top left goes with bottom left and top right with bottom right. I can't easily make out who's who though. Had to log out because my karma got destroyed for posting an opinion.

Re:Serious Question (1)

Carewolf (581105) | more than 2 years ago | (#40028699)

Contestants on the top, judge on the bottom. Take 3 second and you can guess whether the robots are on the left or right. If you take more than that you are a bot yourself :P

Re:Serious Question (1)

TedTschopp (244839) | more than 2 years ago | (#40029057)

Well, I definitely failed that test, I guess I'm a bot. What do you do now that you wake up and know that you are an AI?

Re:Serious Question (1)

Beardo the Bearded (321478) | more than 2 years ago | (#40029543)

You fulfil your programming objectives, same as the meatbots.

Re:Serious Question (1)

Coren22 (1625475) | more than 2 years ago | (#40033187)

MY programming objectives include getting laid, unfortunately I am failing at that...

Re:Serious Question (0)

Anonymous Coward | more than 2 years ago | (#40034851)

MY programming objectives include getting laid, unfortunately I am failing at that...

Sounds like an underflow error.

Bots that can carry conversations? (1)

otaku244 (1804244) | more than 2 years ago | (#40029235)

/. has had users like this for years!

Re:Bots that can carry conversations? (1)

Coren22 (1625475) | more than 2 years ago | (#40033217)

Your mum's face has users like this.

Re:Bots that can carry conversations? (1)

Hognoxious (631665) | more than 2 years ago | (#40039085)

In Soviet Russia, users have faces like your mum!

Disqualified bots/Alan Turing 100 competition (5, Informative)

deksza (663232) | more than 2 years ago | (#40029515)

I'm happy for all the bots that got to compete this year, but I was a little unhappy on the preliminary round of this years competition compared to other years I entered. Only 4 entries can make it to the final round of the competition. There were 12 entries this year but 7 were disqualified due to contest management (Hugh Loebner) not having enough technical knowledge to get the entries working. Some well known bots based on ALICE AIML were disqualified, Cleverbot was disqualified, and my own Ultra Hal was disqualified ( http://www.zabaware.com/webhal [zabaware.com] ) Internet communication is prohibited so we all have to send the bots as self installing programs that can utilize the contests LPP protocol. My own bot is Linux based, which is a big hurdle for the preliminary round, but I sent it as a virtual box image to simplify it for contest management, but he didn't know how to deal with it.

But luckily there will be another competition this year as part of Alan Turing's 100 year centennial at Bletchley Park on June 23rd and recognized by the Olympics http://www.reading.ac.uk/news-and-events/releases/PR445524.aspx [reading.ac.uk] Some of the disqualified bots including my own will be competing there.

Re:Disqualified bots/Alan Turing 100 competition (1)

Chris Burke (6130) | more than 2 years ago | (#40032625)

my own Ultra Hal was disqualified

Since I'm assuming it doesn't outdo Hal in terms of actual sentient intelligence, is it safe to guess that the "Ultra" comes from being less likely to try to murder you?

Or is it that it is ultra likely to try to murder you?

Re:Disqualified bots/Alan Turing 100 competition (1)

deksza (663232) | more than 2 years ago | (#40032999)

I'm pretty sure I didn't write any murder code, although I heard of customers writing plug-ins to control X10 devices based on conversational triggers, so you never know...

Re:Disqualified bots/Alan Turing 100 competition (0)

Anonymous Coward | more than 2 years ago | (#40033787)

User: Hal, turn on the lights, please.
Hal: I'm sorry Dave, I'm afraid I can't do that. [Turns on coffee pot, hoping to start fire]

Re:Disqualified bots/Alan Turing 100 competition (0)

Anonymous Coward | more than 2 years ago | (#40034339)

What a complete joke of a technical competition!

Re:Disqualified bots/Alan Turing 100 competition (2)

dbjh (980477) | more than 2 years ago | (#40038265)

Did he provide that as *the* reason? I just tried your Ultra Hal because of what you wrote, but it failed rather spectacularly. The first two tries it failed in its first response. The third time I accepted some of the weirder responses, but Ultra Hal definitely cannot keep a conversation going (without clearly showing it is a chat bot). However, I was rather disappointed by the low quality of the contestants of the Loebner prize, so maybe it's just me.

What's the point? Nobody cares (3, Insightful)

SmallFurryCreature (593017) | more than 2 years ago | (#40029525)

This is NOT about AI, this is a bunch of whankers whanking.

READ the chat logs, it is not a human trying to see if an AI can hold a realistic conversation but rather to see if it can trip a human/ai up. This is like trying to proof cats can't see color by poking its eyes out and then saying "AHA! See it can't see color". Or me proving you are a lousy at playing catch by shooting you in the head then mocking your mother for bearing such a clumsy child.

Hell, read the chat logs and try to tell who the humans are. It is sad to say that AI chat programs are still not that much more advanced then Lisa or whatever the name was but whats the point in trying to test if chat AI can respond to insane questioning? Congrats, you are now the winner of the bot best capable of holding a conversation with an insane person. WHOO!

Don't let these people near the flying car concept, their test track would include surface to air missles because you know, that is a good test of a flying car.

Re:AI (1)

TaoPhoenix (980487) | more than 2 years ago | (#40030393)

And I reply that a perfectly valid branch of AI is designing defensive routines against trick questions. I feel that is an area the contestants don't pay enough attention to.

You know, like the one in one of the logs "time flies like an arrow, fruit flies like a banana, which is the simile?" should kick right back to the judge with "what's a simile?" and follow it with "nah, I never liked that english class crap".

Or an even crazier example, something like "would Richard Stallman fit in a breadbox?" it should kick back with "Wait, what?"

Re:AI (0)

Anonymous Coward | more than 2 years ago | (#40034413)

So instead of actually creating AI that can talk, we have to invest infinite effort to teach a bot to talk to an insane person. I could easily confuse you, and you're human. This contest if anything is now hurting real progress.

Re:progress (1)

TaoPhoenix (980487) | more than 2 years ago | (#40039491)

Naw, why one or the other? Just do both. We could have the manpower is we "wanted to". You know, instead of artificially limiting the field to 3 man operations.

Say that 15 people in the 100 person team design the defensive routines, which are in some ways simpler because all trick questions have low legit semantic content. Plus speaking of humans it's what siblings and college students do to each other all the time.

The other 85 members of the team can go back to regular language processing.

I would have been fooled (0)

Anonymous Coward | more than 2 years ago | (#40029673)

"crazy tangents talking about being a cat and offering condolences for the death of a pet dragon"

That sounds exactly like most of the humans on the internet to me

Read this Salon.com article on the Loebner prize (0)

Anonymous Coward | more than 2 years ago | (#40033493)

At the competition's inception, it had several respected academics behind it. They realized it was a joke and distanced themselves from it.

www.salon.com/2003/02/26/loebner_part_one/

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...