Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Psychologist Beating Math Nerds in Race to Netflix Prize

Zonk posted more than 6 years ago | from the must-calculate-harder dept.

Programming 205

s1d writes "An almost-anonymous British psychologist named Gavin Potter has suddenly risen to the top of the Netflix prize charts. With his very first attempt, he got a score which took the BellKor team seven months to reach. Currently at a score of 8.07, he has only five teams ahead of him now in the race for the ultimate Netflix algorithm. 'Potter says his anonymity is mostly accidental. He started that way and didn't come out into the open until after Wired found him. "I guess I didn't think it was worth putting up a link until I had got somewhere," he says, adding that he'd been seriously posting under the name of his venture capital and consulting firm, Mathematical Capital, for two months before launching "Just a guy." When he started competing, he posted to his blog: "Decided to take the Netflix Prize seriously. Looks kind of fun. Not sure where I will get to as I am not an academic or a mathematician. However, being an unemployed psychologist I do have a bit of time."'"

Sorry! There are no comments related to the filter you selected.

Beating everyone? (5, Funny)

Anonymous Coward | more than 6 years ago | (#22645744)

5th place is not winning. Are you guys Ron Paul supporters too?

Re:Beating everyone? (1)

FF0000 Phoenix (516214) | more than 6 years ago | (#22646270)

Well, before everyone gets all in a tizzy, let me just say that I'm already on the case to return the favor. As a mathematician and computer scientist, I'm working on a computer program using some pretty heavy-lifting math to allow people to psychoanalyze themselves. I call it the ELIte analyZing mAchine, or ELIZA for short. How does that make you feel, Mr. Psychologist Person?

Re:Beating everyone? (4, Funny)

rcw-home (122017) | more than 6 years ago | (#22646406)

Welcome to your therapy session.

Well, before everyone gets all in a tizzy,

Surely not everyone?

let me just say that I'm already on the case to return the favor.

Can you elaborate on that?

As a mathematician and computer scientist, I'm working on a computer program using some pretty heavy-lifting math to allow people to psychoanalyze themselves.

Why do you mention computers?

I call it the ELIte analyZing mAchine, or ELIZA for short.

Why do you mention computers?

How does that make you feel, Mr. Psychologist Person?

Does that question interest you?

Re:Beating everyone? (2, Funny)

Anonymous Coward | more than 6 years ago | (#22646760)

My copy of emacs doesn't give any of those responses.

Re:Beating everyone? (0)

Anonymous Coward | more than 6 years ago | (#22646782)

Doctor Sbaitso!?

Re:Beating everyone? (1)

krazytekn0 (1069802) | more than 6 years ago | (#22646472)

Hmm, and I always thought it was called Elite MAChine for pSychology.

Re:Beating everyone? (0)

Anonymous Coward | more than 6 years ago | (#22646592)

No, more like Bush supporters. Mission Accompished!

Sorry couldn't resist.

Re:Beating everyone? (0)

gr8scot (1172435) | more than 6 years ago | (#22646600)

rate = distance / time

If the amount of progress he has made is more than the teams of mathematicians did in the same amount of time or less than he has spent, with only his daughter's assistance on "bits of calculus," it's quite reasonable to describe him as "beating" them at the primary task of the competition. Whether the margin by which he is beating them is wide enough to overcome the head start he gave them by entering late will determine, at the end of the contest, whether he has won, but for now the present progressive tense and the verb "beat" are both correct. I would guess at least some of those mathematicians feel pretty beaten right now, as do some other Republican candidates who have gotten fewer votes with more GOP support than Ron Paul, since you just had to mention that, Troll.

Domain Knowledge (5, Insightful)

Anonymous Coward | more than 6 years ago | (#22645772)

It's called domain knowledge people. It helps being a psychologist when you're write a program reacting to people's behavior. If programmers knew how to do that, they would get laid more.

Re:Domain Knowledge (-1, Offtopic)

Anonymous Coward | more than 6 years ago | (#22646018)

There's a strong social obligation to love sex, just like there's a strong social obligation to hate work. A lot of people love their work and couldn't be bothered wasting their time on mating rituals.. then all the brainless breeders call them "losers" because they've made two choices that fly in the face of social pressure.

Re:Domain Knowledge (5, Funny)

wasted (94866) | more than 6 years ago | (#22646116)

There's a strong social obligation to love sex, just like there's a strong social obligation to hate work. A lot of people love their work and couldn't be bothered wasting their time on mating rituals.. then all the brainless breeders call them "losers" because they've made two choices that fly in the face of social pressure.

Like they really had a choice...

Re:Domain Knowledge (1, Interesting)

Anonymous Coward | more than 6 years ago | (#22646206)

Yes, they did. Every man has a choice between his work and his social life. Many men, great men, choose to dedicate themselves to their work. For the vast majority of these men we call them "geeks". For a smaller number, who end up making the mistake of getting married, we call them bastards. Einstein was one such bastard. He treated his wife and children terribly - and not just because he was basically a German man - but because his work came first.

Besides which, sex as described by the kind of people who think geeks really should be trying harder to get it is basically sport. Geeks have enough common sense to recognize that sport is no fun.

Re:Domain Knowledge (1, Insightful)

Anonymous Coward | more than 6 years ago | (#22646490)

we call them bastards. Einstein was one such bastard. He treated his wife and children terribly - and not just because he was basically a German man - but because his work came first.

I love it when women aren't interesting enough, men are blamed. Seriously, if she's too shallow to take an interest in the things he likes, then she should have not married him. Some women are so busy trying to find a sugar daddy they forget they're going to have to live with the "bastard."

Re:Domain Knowledge (1, Interesting)

Anonymous Coward | more than 6 years ago | (#22646520)

Aaah, but many psychologists would argue that dedication to work is simply a means to an end. Basically you become important, rise to the top if you will. And then pull a Bill Clinton. The question is not "why would a great man be adulterous and risk losing his position." in the eyes of many behavioral researchers but "Why not? That was the point of gaining the position in the first place." Note: I don't personally ascribe to this viewpoint but it's hard to argue with if you believe that there is no absolute moral code which then implies a creator/supreme engineer of some kind.

Re:Domain Knowledge (0)

Anonymous Coward | more than 6 years ago | (#22646596)

Maybe to some dedication to work is an end in itself.

Re:Domain Knowledge (2, Interesting)

g00nsquad (971393) | more than 6 years ago | (#22646658)

Anything is arguable by psychologists. However, if you don't need to believe in an absolute moral code, you also don't need to believe in an archetypical behaviour pattern to which we all adhere (read: behaviour pattern that dictates everything we do is part of some elaborate mating ritual). As such, it may be the case that there are many different reasons why people dedicate themselves to work.

In the case of great scientists, artists, politicians or inventors, it may simply be about pure fascination with their particular interest (eg. Darwin studying and classifying barnacles for 8 years - 1856 to 1864), an altruistic (read instinctive if you will) desire to make living conditions better for their counterparts (eg. Norman Borlaug, 1970 Nobel Peace Prize Laureate, for his research in wheat agriculture leading to the Green Revolution), or some other reason.

In short, perhaps not everything is about f-cking, however f-cking certainly is everything to some.

Re:Domain Knowledge (0)

Anonymous Coward | more than 6 years ago | (#22646978)

Hate to be the one to say it, but if its no fun...You're not doing it right.

I understand the amount of single minded dedication that is required to rise to the top of one's field - But that's what your youth is for. If you're in your 30s and you've not yet reached such levels of achievement, it's time to pack it in and find someone, or you're going to be one lonely SOB who offs himself in his apartment if you're not careful.

Re:Domain Knowledge (0)

Anonymous Coward | more than 6 years ago | (#22646200)

Ok, I was just trying to make a joke, but...

It doesn't matter what the "brainless breeders" call us. We don't spend the time working on our social skills, because we don't care. Yes, I'm saying the same thing as your saying. But in this case, with this programming problem, the social skills are applicable. Actually its more complicated than that. You might know how to pick up a girl, but you might not know why it worked. This is where the psychologist has the advantage and can turn why it worked into code.

Re:Domain Knowledge (2, Interesting)

Yold (473518) | more than 6 years ago | (#22646470)

Exactly what I thought, even if he is beating math-nerds, you cannot create a machine-learning algorithm without using fairly sophisticated mathematics. Since psychology is largerly based on statistics, I am sure the guy has a firm grounding in the subject. I am sure he isn't basing his algorithm on Aedipus Complexes (sic) and ink blots.

What's the point? (5, Interesting)

causality (777677) | more than 6 years ago | (#22645782)

Is there some merit to this story other than "your sterotypes can be wrong", which is itself cliche enough to be considered a stereotype in its own right? I like Henry David Thoreau's explanation of such trivia:

And I am sure that I never read any memorable news in a newspaper. If we read of one man robbed, or murdered, or killed by accident, or one house burned, or one vessel wrecked, or one steamboat blown up, or one cow run over on the Western Railroad, or one mad dog killed, or one lot of grasshoppers in the winter -- we never need read of another. One is enough. If you are acquainted with the principle, what do you care for a myriad instances and applications? To a philosopher all news, as it is called, is gossip, and they who edit and read it are old women over their tea. Yet not a few are greedy after this gossip. There was such a rush, as I hear, the other day at one of the offices to learn the foreign news by the last arrival, that several large squares of plate glass belonging to the establishment were broken by the pressure -- news which I seriously think a ready wit might write a twelve-month, or twelve years, beforehand with sufficient accuracy.


And yes, I have karma to burn. Yes I do.

Knee-jerk reactions (-1, Redundant)

causality (777677) | more than 6 years ago | (#22646016)

Ah yes, typical Slashdot. Apparently you must only respond to the text within the story and within the summary, because we have this nice neat little box for you to think in. You may NOT question the merit of the story, why, that is ...... offtopic to the story, somehow. No, it doesn't make sense to me either.

Let's recap. I saw the story, and I honestly thought it was rather pointless. I replied to the story, not just to say that I believed it was pointless but to explain why and included a quote that I believe illustrated the point more eloquently than I was likely to do. You see, all of this is about the story. Someone give me a calm, rational explanation for how this is offtopic, please (that you didn't like what I said does not automatically make it offtopic). Because what this really looks like is "you like and agree with the story or you shut the fuck up, otherwise we mod you into oblivion" which is an insult to every decent non-trigger-happy mod out there. Apparently some of you forget that questioning decisions and questioning "the way things are" from time to time is generally a healthy thing, even if the answers are not readily available or are not what you wish them to be.

I'm fully expecting this to be modded to -1 as well. Now, occasionally people do surprise me, but really most of you are reactive (and not terribly proactive) and therefore rather predictable. To mods who mod this down, you aren't proving anything (except maybe my description of you), you aren't surprising me in the slightest, and I have plenty of karma to burn so you aren't damaging me in the slightest. What you are doing is losing by forfeit my tiny little challenge that you explain how a comment about a story is not on-topic to that story.

Re:Knee-jerk reactions (-1, Flamebait)

Anonymous Coward | more than 6 years ago | (#22646288)

You're a cunt. C-U-N-T, cunt. Go crawl back under your CUNT ROCK.

Re:Knee-jerk reactions (-1, Flamebait)

Anonymous Coward | more than 6 years ago | (#22646396)

I love cunt rock! Eagles, Lynyrd Skynyrd, Georgia Satellites, the whole freakin' genre.

Re:What's the point? (0)

Anonymous Coward | more than 6 years ago | (#22646216)

"And I am sure that I never read any memorable news in a newspaper"

  Yes, yes, who needs news? It is all just scurrilous gossip anyway. I don't need to know what is happening in the various wars and intrigues that besiege our planet, nor is my curiosity piqued by the latest developments in science and technology.

In short, that quote is terribly irrelevant here. Try posting it in the comments of a blog devoted to Britney Spears and the monitoring of other celebritrash. No-one is stopping you from posting articles that you feel are relevant - however, believe it or not, there is diveregence of opinion as to what constitutes relevance.

Re:What's the point? (1)

ScentCone (795499) | more than 6 years ago | (#22646368)

If we read of one man robbed, or murdered, or killed by accident, or one house burned, or one vessel wrecked, or one steamboat blown up, or one cow run over on the Western Railroad, or one mad dog killed, or one lot of grasshoppers in the winter -- we never need read of another.

Right. Unless it's your brother that was robbed, your mother's house that burned, your life savings tied up in the boat that sank, your cow that was run down, or your new neighbor's rabid dog that needs killing.

Re:What's the point? (2, Funny)

0123456789 (467085) | more than 6 years ago | (#22646416)

Right. Unless it's your brother that was robbed, your mother's house that burned.


If you don't find out about those until you read about them in the newspaper, your family qualifies as dysfunctional. Really dysfunctional.

Re:What's the point? (5, Funny)

besalope (1186101) | more than 6 years ago | (#22646518)

Right. Unless it's your brother that was robbed, your mother's house that burned, your life savings tied up in the boat that sank, your cow that was run down, or your new neighbor's rabid dog that needs killing.
Meh, that just sounds like the making of a country music hit.

Average psychologist (5, Informative)

Anonymous Coward | more than 6 years ago | (#22645808)


that had an undergraduate degree in psychology, a masters degree in operations research [wikipedia.org] that after being well employed for a number of years -- "In 2006, he left his job at IBM to explore the idea of starting a PhD in machine learning, a field in which he has no formal training. When he read about the Netflix Prize, he decided to give it a shot -- what better way to find out just how serious about the topic he really was?"

He just so happens to be... (1)

Aegis Runestone (1248876) | more than 6 years ago | (#22645848)

Just a guy who happens to own everyone else. ^_^

Re:He just so happens to be... (1)

SEWilco (27983) | more than 6 years ago | (#22645940)

Aaaaaaah! He's in my head! Get him out, gethimout!

psych majors (1)

rastoboy29 (807168) | more than 6 years ago | (#22645850)

Very few psych majors do psych professional.

I have a BA from U. of Houston, which I have used to do tech support and sales for a software company, and developed my own FPS as a solo project.

That's true (4, Interesting)

Crazy Taco (1083423) | more than 6 years ago | (#22645950)

Very few psych majors do psych professional. I have a BA from U. of Houston, which I have used to do tech support and sales for a software company, and developed my own FPS as a solo project.

I've also found this to be true. Lol, I actually knew people in college that did nothing but program computers in their spare time, and took psych because it was easy, wouldn't distract them and gave them more time to do the programming they wanted. They didn't ever expect to practice psych.

Re:That's true (1)

RockModeNick (617483) | more than 6 years ago | (#22646564)

I'm a computer nerd, but switched to psych after the dot crash, and now have a useless psych BA.

Re:That's true (1)

BlueBoxSW.com (745855) | more than 6 years ago | (#22646640)

Psych was HARD to major in. The easy classes were the business classes I took to fill out the minor.

I was the only student who asked questions.

I have studied AI, and think we're a long way off.

I got the MASTERS in technology.

Misleading summary (5, Informative)

Zen Programmer (518532) | more than 6 years ago | (#22645856)

The summary makes 2 references to Gavin Potter being a psychologist, but it ignores the part of the article that notes he has a master's degree in operations research. This is very much an OR problem. Still, it is impressive that he has been able to do as well as he has considering his competition. Good luck to him!

Re:Misleading summary (1)

mmortal03 (607958) | more than 6 years ago | (#22646730)

No kidding. Operations Research is basically an area of applied mathematics, but unfortunately, not many people even know what it is, hence the confusion. It is no wonder that an individual with Psychology and Operations Research degrees is doing so well.

Interdiscplinary approach (5, Interesting)

oceaniv (1243854) | more than 6 years ago | (#22645866)

The topic is incredibly fascinating. And just a thought, up to 5 centuries ago "scientists" were incredibly versatile people, with mastery over a few fields at a time... A lot of people argue that this was out of necessity, but could the versatility have been important development of multiple renaissances (In Greece, East/West Asia, and Europe)? And could the bottleneck specialization of fields that has occurred in the past three centuries simply be period of transition/stifling new ways of thinking? Could the emergence interdisciplinary experts lead to another 'renaissance' of sorts?

Re:Interdiscplinary approach (3, Interesting)

Metasquares (555685) | more than 6 years ago | (#22645988)

I think so [projectpolymath.org] .

Re:Interdiscplinary approach (5, Interesting)

Kenshin (43036) | more than 6 years ago | (#22646034)

See, I hate this current mode of profession.

I work for a small company. My current job isn't stable, and doesn't pay well, so I'm taking an IT course so I can land a (hopefully) stable job/career.

However... in my current job I wear all kinds of hats. Server's down? I'll fix it. Marketing materials need to be designed? I'll do it. Proposal needs to be edited? I'm there. Computer needs more RAM? I'll install it. Photo of product needs to be masked-out? Done. Need to do some research? I'll get on it.

The kind of job I'm being trained for... I'll be stuck on the straight and narrow, handling one sort of task. When companies want an IT guy, they want an IT guy. I don't know how I'll be able to handle that. I LIKE having different responsibilities. I don't want to be one guy on an org chart with a specified duty.

Blah. I really went on a tangent there...

Re:Interdiscplinary approach (5, Insightful)

mattOzan (165392) | more than 6 years ago | (#22646176)

I like being jack-of-all-trades too. But I'm finding that one must tread very carefully on this path.

If you aren't careful, you end up being the fall guy for a widening array of mishaps.

For instance, you help set up the video projector a couple of times for presentations. Then during the next presentation the projector fails. In some eyes it will be your fault, because you're now the "PowerPoint Guy." Nevermind that the bulb was past its recommended use hours, or that the presenter forgot his VGA dongle, or whatever.

It seems like if I want to come out as the go-to guy for some area of tech, I'd damn well better get up to pro-level speed really quick. Because soon I'm going to have to be mitigating crises and solving complex problems that before were just chalked up to "well, that thing's always been a problem." Yeah, now it is *your* problem!

Re:Interdiscplinary approach (1)

ivan256 (17499) | more than 6 years ago | (#22646630)

The natural place for a jack-of-all-trades to land is in his/her own business.

Re:Interdiscplinary approach (2, Insightful)

Cylix (55374) | more than 6 years ago | (#22646310)

Surprisingly enough, there are fields that pay much better which require a broad range of expertise.

However, no one will dare mention all of the real requirements. You see, the valid candidates will run screaming away because it looks to be too much, but what you end up with is a person who scoffs at all the extra that wasn't mentioned when he was hired.

Thus, eventually the cycle continues until you no longer need the jack of all trades and have many very specialized people who cannot get anything completed.

Welcome to Corporate America.

Re:Interdiscplinary approach (1)

kaiser423 (828989) | more than 6 years ago | (#22646312)

I'm with you man. I would prefer to get 10 years of experience rather than 2 years of experience 5 times (aka working in a job with one hat where you do the same thing over and over again).

Of course you're always going to specialize...and hoping learning new things in that field everyday. The second that I have been practicing at a high level for a couple of months in a field is the second that I start training my replacement while I look for something new to sink my teeth into. That way I mostly master a trade and move on. Now, a lot of my knowledge is rusty, but since I'm used to picking up new things, re-learning is not a huge deal whenever it comes back up.

Yea, losing that is something that I would mourn also.

Uh, no... (0)

raehl (609729) | more than 6 years ago | (#22646038)

In the old days, scientists were more versatile because each area of science was only at the elementary level. If you were wealthy enough to not need to spend all day making sure you had enough food to eat, knew how to read, had access to books, and had the means to travel, congratulations, you had all the necessary qualifications for 'Renaissance Scientist'.

I suppose you also had to be willing to focus on scientific pursuits instead of eating, drinking, music, building obscenely large residences, invasion of thine neighbor, and/or peasant subjugation.

Hell, those scientists didn't even start figuring out basic laws of motion until the 1500's. Electricity? Not until Ben Franklin! And now any 3rd grader learns about electric circuits.

I think every Slashdot reader understands pretty much all science through the 1700's at least. Anybody here understand string theory? I'm sure a few people do, but I ain't one of them. Old school scientists we more versatile because in the 1600's, science was easy!

Re:Uh, no... (0)

Anonymous Coward | more than 6 years ago | (#22646214)

I think every Slashdot reader understands pretty much all science through the 1300's at least.
Fixed it for ya.

Re:Uh, no... (0)

Anonymous Coward | more than 6 years ago | (#22646256)

Ben Franklin? Thales of Miletos worked with static electricity about 2000 years before Franklin.

Re:Uh, no... (0)

Anonymous Coward | more than 6 years ago | (#22646276)

Us scientist types, we call that "charge".

Re:Uh, no... (5, Insightful)

Anonymous Coward | more than 6 years ago | (#22646268)

What a bunch of drivel. Just because their level of knowledge isn't what we have today, doesn't make it any "easier." Do you have any idea at all, or can you even comprehend, the kind of mathematics that were employed back in the day to solve anything? Take a look at the Principia for example. The geometry is insane. I'm a graduate student in Physics and I can't really follow his proofs.

Furthermore, because early scientists did not have as much to build on, that makes it all the more difficult. Where was Faraday to get his inspiration on lines of force? What lead Maxwell in the right direction to unifying light with electromagnetism?

It's great that 3rd graders know about electric circuits. That's the point of scientific progress. That doesn't make the original task trivial in any sense.

In other words, I hate you.

Re:Uh, no... (0, Offtopic)

mha (1305) | more than 6 years ago | (#22646820)

Are the moderators insane, stupid or both? WHy did the parent of the parent get "5-insightful" when his post is so disrespectful of enormous achievements that it can rightly be labeled "stupid", while the parent gets "-1"? Stupidity (of moderators) knows no limits these days.

Related and strange and funny (or not!), why, when I have to meta-moderate, don't ever get any of these idiots? So much trivia gets a "+5 insightful", so much "modding by opinion", statistically it is impossible I didn't get a chance to meta-punch one of those guys...

All of 18th century science? Really? (4, Insightful)

BorgCopyeditor (590345) | more than 6 years ago | (#22646418)

OK, how do you solve a cubic? Or grind the lenses for a telescope? Or build a water pump? How do you dress a head wound? Or isolate pure gases? Or calibrate a thermometer?

If you know how to do all those things, good for you. My point is that a secondary school education in "science" does not by itself provide one with the same understanding as the great scientists who first among all others figured out how to do each of these things.

Re:Uh, no... (1)

Stalyn (662) | more than 6 years ago | (#22646510)

Perhaps these things seem simple now because they were solved so long ago. Genius is genius. I have no doubt if Newton was alive today he would have no trouble understanding and solving the major outstanding problems faced by today's scientists. Also that's not to say there haven't been modern era "Renaissance" scientists, John von Neumann immediately comes to mind. I think however the GP's point was that sometimes real progress is made through the application of one field on another seemingly unrelated field. That in today's community of scientists there might be a degree of overspecialization which is acting as a detriment.

Re:Uh, no... (1)

Rakishi (759894) | more than 6 years ago | (#22646560)

Genius is worthless alone if the field requires 100 textbooks worth of knowledge to even known the basics of. You have to understand those 100 textbooks or you can't make something that requires what those textbooks say as a basis. Otherwise you'll just spend all your time recreating what those textbooks say and never create anything new.

On the back of giants (2, Interesting)

bussdriver (620565) | more than 6 years ago | (#22646892)

Subjects get so complex and so specialized there HAS to be limits to how many giants a human can climb and still have time enough to become a giant themselves. Abstraction helps to greatly extend this range; the people behind abstractions/simplifications may not be considered giants because they do not produce progress themselves but just facilitate others so they can extend their reach into the unknown.

There does not appear to be that many 'giant' scientific figures anymore despite the exponential scientific growth. Maybe it is just an appearance and there are more; but are there more proportionally to the number scientists?

How many big leaps in knowledge have been made in the old fields like physics for example? If the decline does not exist now, won't it exist at some point??

WWLD? (What would leonardo do?) (4, Funny)

TiggertheMad (556308) | more than 6 years ago | (#22646420)

Could the emergence interdisciplinary experts lead to another 'renaissance' of sorts?

Oh, I don't know. If only we had a art history & statistics dual major to figure it out...

Re:Interdiscplinary approach (1)

elloGov (1217998) | more than 6 years ago | (#22646426)

I too am a versatilist. Scientist up to 5 centuries ago didn't have little expenses that can lead one to tight corners. Things like phone bills, heating, this tax that fee, socializing money, fancy clothing, hell water even costs money these days. You get my point. Could your bottleneck hypothesis be true? Absolutely! However until versatility is one day the king, specialization will rule. We are paid to be good, very good, "expert" at one thing, kinda like machines :). For versatilists to be rewarded, we need tangible evidence. Unfortunately, in a world where capitalism (money) rules, profit is best measured by specialized tasks.

Re:Interdiscplinary approach (1)

Rakishi (759894) | more than 6 years ago | (#22646852)

It's quite possible to be paid for being versatile but that requires you to actually have some concrete advantage of being that way. Likewise you actually need to be good (not just decent or passable) in at least some areas and have those areas be related (ie: see first sentence). Even in that case you should go into management in most likelihood because then you can do more good by "simply" telling other specialists what to do than by actually half-assing things yourself.

That said there are whole fields that are in essence the combination of existing ones such as molecular biology (biology, chemistry, physics, math), bio-mechanics (biology, medicine, mechanical engineering ,electrical engineering), computational biology (biology, cs, math), bio-statistics (biology, statistics, math) and so on (I can go on for pages and pages just from the degrees and specializations offered at my school).

Re:Interdiscplinary approach (1)

Rakishi (759894) | more than 6 years ago | (#22646816)

There are a fair number of scientists who know multiple subjects, most of them simply don't become well known. They may make some great new application of physics to biology but they probably won't make breathtaking discoveries in either. Likewise since it's easier to just study one subject and people generally seem to only be interested in one area you also get fewer people who can and do work across multiple fields.

Actually any good scientists nowadays probably knows more from across different fields than those "generalists" from centuries back. A molecular biologist may have to, for example, know advanced chemistry and advanced physics. So really it's hard to say someone is a specialist if we've simply redefine his "field" to cover what used to be 5 separate ones.

Re:Interdiscplinary approach (2, Interesting)

Antique Geekmeister (740220) | more than 6 years ago | (#22646976)

Or could the existence of things called "printed books" lead to the spread of knowledge among others who could use it, rather than having to build up their theoretical knowledge from personal experience and a few precious manuscripts? And make it more broadly available so that people with less than mutant brilliance can contribute andn publish their contributions for others?

I don't think you need to read in any great cultural change to a cross-disciplinary approach here: the problem is one well-suited to this man's exact skills, an algorithmic computation of likely human behavior. Encouraging cross-disciplinary work because people focused too tightly on one field will miss available tools from other fields is helpful, and certainly helps keep me paid for work that ranges throughout computing and engineering fields.

Umm.... (5, Informative)

palegray.net (1195047) | more than 6 years ago | (#22645886)

He might be a psychologist, but his venture firm is named Mathematical Captital [mathematicalcapital.com] , after all. His partners appear to have advanced degrees involving mathematics.

Psychologist? (5, Insightful)

Otter (3800) | more than 6 years ago | (#22645890)

I don't understand why Wired insists on playing along with Potter's pretense of being an "unemployed psychologist". He's a PhD candidate in machine learning, has a masters in operations research, is ex-IBM and Pricewaterhouse, runs a VC firm -- he has plenty of quantitative and computational training and experience, probably more than most of the contestants.

Re:Psychologist? (5, Funny)

palegray.net (1195047) | more than 6 years ago | (#22645900)

You can't just come barging in here with your fancy facts and logical arguments. You'll ruin the whole conversation that way! And we don't take kindly to strangers in these parts...

Re:Psychologist? (5, Funny)

QuantumG (50515) | more than 6 years ago | (#22645922)

Alternatively: It's a bit hard to argue with you when you keep making sense.

Re:Psychologist? (2, Insightful)

Ambiguous Coward (205751) | more than 6 years ago | (#22646140)

Stranger to these parts? Dude, did you notice his user id? 3800. Yes, three-thousand-eight-hundred. He's been around these parts since before you were born. Which, I suppose, may actually be "strange"...as in, he's stranger than everyone who's ID is > 3800, but less strange than 3799 other people. Still!

-G

Re:Psychologist? (4, Funny)

poena.dare (306891) | more than 6 years ago | (#22646304)

I knew user 3799, and let me tell you, user 3800 is no user 3799.

Re:Psychologist? (3, Funny)

Chris Johnson (580) | more than 6 years ago | (#22646828)

Ah, those were the days.

You kids get offa my lawn!

Re:Psychologist? (1)

Igmuth (146229) | more than 6 years ago | (#22646894)

Go back in the house, Gramps!

Re:Psychologist? (0)

Anonymous Coward | more than 6 years ago | (#22646404)

protip: That was the last part of his joke

Eh... (1)

modecx (130548) | more than 6 years ago | (#22646550)

You must be new here?

Re:Psychologist? (1)

sjelkjd (541324) | more than 6 years ago | (#22646664)

What is the sound of a joke being missed?

Re:Psychologist? (1)

jspraul (146079) | more than 6 years ago | (#22646674)

whoosh

Re:Psychologist? (5, Insightful)

Vexorian (959249) | more than 6 years ago | (#22645932)

Well, it is just that dog bytes man"Psychologist beats math nerds" is a more interesting headline than actual facts.

Re:Psychologist? (1)

Vexorian (959249) | more than 6 years ago | (#22645954)

Man, I need to use the preview button more often, I tried to say this:

Well, it is just that "Psychologist beats math nerds" is a more interesting headline than actual facts.
I didn't know we weren't allowed to use the <s> tag.

Re:Psychologist? (1)

Ostsol (960323) | more than 6 years ago | (#22645998)

> I don't understand why Wired insists on playing along with Potter's pretense of being an "unemployed psychologist". Because people like 'underdog' stories.

Re:Psychologist? (1)

at_slashdot (674436) | more than 6 years ago | (#22646050)

With all these qualifications it's interesting that his daughter helps him with the math for the project.

The important question (-1, Troll)

Anonymous Coward | more than 6 years ago | (#22646136)

Is she hot?

Thinking about how people shop (4, Interesting)

GregPK (991973) | more than 6 years ago | (#22645912)

My extensive retail experience says people like to shop by the following methods. Ratings, Genre, Alphabet.

So, if I was to setup a movie viewing for them. I'd setup something along the lines of a Genre, rating(R,PG-13,G), Alphabet.

It's kind of a takeoff on my video game organization method that increases sales of video games by 30 percent. I called it ABSRG short for Alphabetize By Section(4 foot section), Rating(M on top T in the middle and E towards the bottom.), Genre(Sports, driving, shoot em up). Please note, this cannot be patented, I already let it go out for more than a year(Started in 1998)and I have the pictures and time notes to prove it.

Re:Thinking about how people shop (1)

pwsegal (781691) | more than 6 years ago | (#22646024)

It would be an interesting exercise for somebody to try and patent your idea as a test to how (in)effective the patent office really is.

Re:Thinking about how people shop (1)

GregPK (991973) | more than 6 years ago | (#22646052)

They might be able to patent it, but they wouldn't be able to enforce it. I've got this on file with Walmart, Circuit City, Fry's, Target, Gamestop. Walmart actively uses it for their POG.

Re:Thinking about how people shop (2, Interesting)

WK2 (1072560) | more than 6 years ago | (#22646180)

Maybe they can't win a court case, but they could probably get at least 1 mega$ settlement by suing Walmart, Circuit City, Fry's, Target, and Gamestop.

Free Idea (5, Interesting)

Speare (84249) | more than 6 years ago | (#22646120)

Okay, I'm not trying for this prize, but there's one thing about Netflix "recommendations" that bugs me so I'm throwing out this complete freebie of an idea. If it helps someone get a 0.001% improvement to add this ONE little additional check, great.

I am learning Japanese. I have been watching several hundred Japanese-language movies for the past couple years. I don't watch movies in Greek, Spanish, Turkish, Farsi, Italian, Russian, German, or Hebrew. I did watch Amelie four years ago but that doesn't mean I love French movies. Most of my recommendations are for foreign films, but only a small fraction of those recommendations are for Japanese movies.

Apparently, Netflix doesn't have a column in their database saying WHAT language a movie uses principally, it just has a flag saying it is not English. It's the only explanation I can see for not checking for such a strong correlation. I admit, I might not be sharing the experience of the most common movie-renting drone in the bunch, but I doubt I'm the only person who has such a lopsided taste in movies. If the language (or alternate soundtrack languages) ARE known in the database, please see if the renter has a bias for movies in a particular language.

Re:Free Idea (1)

dedazo (737510) | more than 6 years ago | (#22646228)

Yes, Blockbuster is the same. It's annoying.

Re:Free Idea (0)

Anonymous Coward | more than 6 years ago | (#22646616)

I'm not trying either. I find it difficult to get excited about something that will be copyrighted for twice my lifetime, closed source, proprietary, and probably patented. It would be like giving up a child for adoption... even if code went to a good home with advantages and utility I could never give the algorithm, I'd always hate myself for giving up all my rights to MY baby...

As for the language recommendation... I've noticed this same issue myself. This million dollar prize really is peanuts to these guys. They spent $71 million on R&D [nasdaq.com] last year, yet they have the crappy recommendation system they have now. As far as they're concerned, a million bucks is dirt cheap. Personally, if I were this guy and I somehow "won" I would try to refuse the prize in order to negotiate a better deal.... meh, It'd never work... they probably have a clause that voids your rights just for entering.

Re:Free Idea (2, Informative)

justinlee37 (993373) | more than 6 years ago | (#22646716)

Contest participants retain ownership of the code they write, but the winning team must license it (non-exclusively) to Netflix.

Not English! (1)

Typoboy (61087) | more than 6 years ago | (#22646648)

Mr Podsnap's world was not a very large world, morally; no, nor even geographically: seeing that although his business was sustained upon commerce with other countries, he considered other countries, with that important reservation, a mistake, and of their manners and customs would conclusively observe, 'Not English!' when, PRESTO! with a flourish of the arm, and a flush of the face, they were swept away. (Dickens. Our Mutual Friend. Chapter 11. )

Re:Free Idea (0)

Anonymous Coward | more than 6 years ago | (#22646692)

Perhaps Netflix's algorithm has recognized your penchant for languages, and is suggesting some other languages you might care to learn.

Re:Free Idea (0)

Anonymous Coward | more than 6 years ago | (#22646778)

I am learning Japanese. I have been watching several hundred Japanese-language movies for the past couple years.

If you're motha-fuckin trying to motha-fuckin learn a motha-fuckin language by motha-fuckin watching motha-fuckin movies, you be one motha-fuckin crazy son-of-a-bitch.

how many GB is the dataset? (1)

martin-boundary (547041) | more than 6 years ago | (#22646170)

Does anyone know how big the dataset download is and what the format is? The website wants a registration before it lets you download the dataset.

Re:how many GB is the dataset? (4, Informative)

Paeva (1176857) | more than 6 years ago | (#22646230)

The dataset is about 660MB to download. It unpacks to 2GB of about 18,000 text files.

Re:how many GB is the dataset? (1)

martin-boundary (547041) | more than 6 years ago | (#22646300)

Thanks, I just want to have a rough idea of the mathematical complexity of the problem :)

As I understand from the rules, each customer rates a bunch of movies. How many customers are there? How many movies? How many ratings does each customer do (on average, perhaps)?

Re:how many GB is the dataset? (4, Informative)

Paeva (1176857) | more than 6 years ago | (#22646354)

There are 480,189 customers that rated 17,770 movies. The total number of ratings that you're given is 100,480,507. Each user/movie/rating is accompanied by the date of the rating, as well. You then have to submit predictions for the ratings of 2,817,131 additional user/movie combos (they tell you the user, the movie, and the date, and you need to predict the rating). You submit these predictions to Netflix, and they tell you the root-mean-square error between your predictions and the actual ratings that those users gave those movies.

Re:how many GB is the dataset? (1)

martin-boundary (547041) | more than 6 years ago | (#22646414)

Many thanks. Just one more question: for the test set, are the users and/or movies all existing users and/or movies (from the training set), or can they be essentially unrelated (ie a movie which was not rated by anybody before, or a completely new user)?

Re:how many GB is the dataset? (2, Informative)

Paeva (1176857) | more than 6 years ago | (#22646486)

Each of the movies and each of the users represented in the test set have some corresponding known ratings. I think the minimum is around 10 known ratings for users and maybe 3 or 4 for movies. And there are examples on the opposite end of the scale... one user has rated nearly every movie in the training set, and most of those ratings were 1 star. If you have any more questions, feel free to check out my blog - my Slashdot profile links to it, and you can find my email there. I'm Dan Tillberg, currently in seventh place in the contest ;). I'd love to help answer any questions you or anybody else might have (or at least help point you to good resources), and I certainly encourage anybody that's interested to try their hand in the contest :).

Poor math nerds... (3, Funny)

Gorimek (61128) | more than 6 years ago | (#22646188)

Being beaten up is normal for any nerd, but by a Psychologist - that's gotta hurt...

Re:Poor math nerds... (2, Funny)

Kingrames (858416) | more than 6 years ago | (#22646398)

How does that make you feel?

Re:Poor math nerds... (1)

trytoguess (875793) | more than 6 years ago | (#22646834)

He beats you, makes you think it's your fault, and that you secretly want to sleep with your parents?

Slightly different take... (2, Insightful)

zappepcs (820751) | more than 6 years ago | (#22646302)

Yes, the story is old now (by Internet standards) and no, he's not actually winning. What he did do was make a great leap in the success of his process by using a better group of knowledge for a base to work with. This is neither new or amazing in anyway. The only reason it makes news is that it's so much common sense that the story is told as if he has had some huge breakthrough.

Even if he only gets to 9.25% I will bet he gets offers to work with AI researchers around the globe. That is, after all, what their stated goal is - more or less. Every programmer knows about the GUI wars, and has read stories about how programmers have trouble writing code or designing web sites that are intuitive for users. If you want to see you code break, put a user on the keyboard and wait a few minutes.

Just about everything that I do with computers shows me something that could be more impressive or intuitive. Can you say 'click START to shutdown' ? Applying psychology and math to a computer problem is a problem that programmers are faced with all the time, and the industry as a whole fails on this repeatedly. A matter of personal interest, hobby robotics, holds a particular problem that seems simple but is not and demonstrates the scope of the problem with this story. Try to build a small robot that can wander around your house and never get stuck behind the couch, or anywhere else. Even cockroaches can accomplish this, but sophisticated robotics cannot.

We've all seen people come from nowhere, solve a problem because they looked at it a different way than everyone else based on their experiences. I think that it is about time that we started doing more of this. The biggest problem that I can see thus far is that people don't act like computers, they seldom repeat anything with precision. Can you say manufacturing robot? Everyone of us has personal tastes, and it's usually only when Hollywood tells us what movies are good that we all fall in line. Sure, some 'blockbusters' fail, but they make money because of the hype. When you remove the hype, it falls apart. Picking out what other people like or might like based on a very small data set is a difficult task. Not everyone likes kids or movies for kids. Not everyone likes hollywood-ized cookie cutter movies. The task is daunting at best.

Apply that thinking to other things, and you can see why some websites work and others do not. Why some software works and why others fail. Should F1 be the help key or F3? Why not CTRL-H? Maybe your preferences for such things differ from mine. What I'm getting at is that predicting what a human will do is not simple. Categorizing movies by story, style, genre etc. is like applying a tag cloud to it and matching the tag hits of one group to your personal tag choices. It kind of works, kind of does not. Either way, it needs to be applied more often. Just today I received a thank you note from the local Honda dealer where I got my seat belt replaced under warranty. I bought the car 15 years ago from the dealer my mother likes, and is two states away from me now. The dealer that send the card is local to me (2 states from my mom) but they sent the card to her, at MY address. Tell me how a human would have done that?

The basic problem is that we humans accrue various bits of information and make decisions based on that. Our thinking process halts when something 'just doesn't make sense' to what we are doing. Computers don't do that... yet. Perhaps this guy is on to something, but then maybe not. A human would not only ask what other people liked this movie, but also "you really liked that piece of crap?"

To put the I in AI is going to take a lot of rethinking. Simply acting like a perfect human won't do it. Oh, you liked that movie? yeah, me too, I love the city where it was filmed. -- get a program to do that? That oddball out-of-left-field thinking is what will make the software very good at predicting what you will or will not like, maybe.

Have you ever tried to figure out what kind of music someone would like from your collection? Yes, it's not easy unless their tastes are so much like you that you can't miss. That is the scope of the netflix problem.

Now, to add the punchline. Get your trigger finger on the off-topic button, but the MPAA has been telling us what we want to watch for so long that it has skewed the data beyond repair. Well, perhaps. The Independent Film Channel has helped to re-introduce non-hollywood films. Movie rental stores have helped. As long as film distribution remains largely in the hands of a few people, the problem will remain. The data, the tag clouds remain unintelligent when most of the populace is hand fed a shortlist of films from a precious few people in Hollywood.

I'm willing to bet that this guy's progress is stilted because of the skewing of the data because of the MPAA effect. If it has not been remade by Hollywood, North Americans hardly ever see the film, and this slants the data. Sure, knowing the slant might help, but I think not. The same skewing has taught North Americans that bottled water is better than tap water... WTF is that? Yes, humans make decisions based on information that is often not good information. Programs rarely ever do. They normally fail when not enough information is available. This is an attempt to answer as best as they can without enough information.

If you've read this far, you probably get it now. This algorithm is the reverse of the Hollywood advertising machine. One tries to get everyone to like the same thing, the other tries to figure out what everybody likes. When both are operating in the same space, you are sure to get a divide by zero error every now and then.

I think he has done well, and hope that more people build on his success.

And? (3, Interesting)

Secret Rabbit (914973) | more than 6 years ago | (#22646382)

There a joke that I heard (that's actually pretty much true):

A Physicist goes to a Mathematician for advise on solving a Differential Equation. The Physicist explains this and writes the equation on a Blackboard. The Mathematician stares at the equation for more than half an hour. Finally, he says, "Yes, it has a solution."

Basically, the Maths (even applied) are about details and considering them *very* carefully. With this in mind, is it any surprise that they are somewhat "slow"? Especially when they are starting from scratch within the problem domain?

Guess he wont be.... (1)

moezaly (1197755) | more than 6 years ago | (#22646488)

free after all since he is certain to find a job and thus wont have the time to do this anymore

Not all of the Top 5 are just Math Nerds (3, Insightful)

Shazow (263582) | more than 6 years ago | (#22646570)

The team at the University of Toronto, who are using a neural networks approach, are led by Geoffrey Hinton [wikipedia.org] who has a bachelors degree in experimental psychology.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?