Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Netflix Prize May Have Been Achieved

Soulskill posted more than 5 years ago | from the universal-exclusion-of-pauly-shore-did-the-trick dept.

Announcements 83

MadAnalyst writes "The long-running $1,000,000 competition to improve on the Netflix Cinematch recommendation system by 10% (in terms of the RMSE) may have finally been won. Recent results show a 10.05% improvement from the team called BellKor's Pragmatic Chaos, a merger between some of the teams who were getting close to the contest's goal. We've discussed this competition in the past."

cancel ×

83 comments

Sorry! There are no comments related to the filter you selected.

Jews did it. (1)

Luke727 (547923) | more than 5 years ago | (#28490025)

Also 9/11.

No info about the Netflix prize (2, Insightful)

Daimanta (1140543) | more than 5 years ago | (#28490045)

C'mon, the Netflix prize isn't THAT well known. At least you could have given some basic info about it.

Re:No info about the Netflix prize (5, Informative)

Reikk (534266) | more than 5 years ago | (#28490095)

Background: The Netflix Prize is an ongoing open competition for the best collaborative filtering algorithm that predicts user ratings for films, based on previous ratings. The competition is held by Netflix, an online DVD-rental service, and is opened for anyone (with some exceptions). The grand prize of $1,000,000 is reserved for the entry which bests Netflix's own algorithm for predicting ratings by 10%.

Re:No info about the Netflix prize (4, Informative)

MrMista_B (891430) | more than 5 years ago | (#28490173)

What, you didn't even read the /summary/?

I know, this is Slashdot, but 'some basic info about it' is /right there/.

Re:No info about the Netflix prize (1)

Abstrackt (609015) | more than 5 years ago | (#28490787)

So RTFS means read the first sentence eh? ;)

Re:No info about the Netflix prize (2, Informative)

Korin43 (881732) | more than 5 years ago | (#28491059)

Except it doesn't mention what an improvement of 10% means (unless you know what RMSE means, which I don't).

Re:No info about the Netflix prize (4, Informative)

bogjobber (880402) | more than 5 years ago | (#28490239)

If the first sentence didn't explain it enough, perhaps you could RTFA.

Re:No info about the Netflix prize (0)

Anonymous Coward | more than 5 years ago | (#28491233)

"The long-running $1,000,000 competition to improve on the Netflix Cinematch recommendation system by 10%..."

It's right there in the opening sentence. Easy to miss.

1 Million split 7 ways (4, Funny)

basementman (1475159) | more than 5 years ago | (#28490049)

Let's see, $1,000,000 split 7 ways gives us $142,857.14 each. Let's say taxes take half, now you are down to only $71,428.57 each. Unless one of them kills all of their partners like in The Dark Knight that ain't much of a prize.

most of them did it (4, Insightful)

circletimessquare (444983) | more than 5 years ago | (#28490105)

for simple intellectual satisfaction, like a giant puzzle or a game of chess

money is not the motivation for everything in this world

Re:most of them did it (1)

basementman (1475159) | more than 5 years ago | (#28490211)

Jesus guys I thought the Dark Knight comment would tip you off, I was trying to be funny.

alas, basementman (1)

circletimessquare (444983) | more than 5 years ago | (#28490245)

batman even has a better sense of humor than you ;-)

Re:alas, basementman (1, Funny)

Anonymous Coward | more than 5 years ago | (#28490443)

I'm BatCoward.

Re:most of them did it (2, Insightful)

morgan_greywolf (835522) | more than 5 years ago | (#28490291)

Well, it was for AT&T. No, they don't want the prize money; they're donating it charity. But what they do have now is an algorithm that can be turned into a commercial product or service. The individual researchers may not have had money as their primary motivator, but their employer sure has hell did.

Re:most of them did it (1)

Saba (308071) | more than 5 years ago | (#28490815)

money is not the motivation for everything in this world

But it's waaaay ahead of what's in second place.

Re:most of them did it (1)

ouimetch (1433125) | more than 5 years ago | (#28491063)

That may be true, but $1,000,000 is still some pretty freaking sweet motivation...

Re:most of them did it (1)

hydromike2 (1457879) | more than 5 years ago | (#28492003)

right, money is just what pays for the giant puzzle and the food consumed solving it

Re:most of them did it (0, Flamebait)

OeLeWaPpErKe (412765) | more than 5 years ago | (#28493855)

Heh, it's more than I make in a year. Much more I might add.

And they could always pay it to a charity. You know "Society for the improvement of John Q. Myself", headquartered in Bermuda, to "cheat" taxes. The whole Obama administration are tax dodgers, or at least, one could be forgiven for thinking so, meaning it can't be that wrong. If "tax-cheaters anonymous" is good enough a government for the american people, surely they can forgive an enterprising tax dodging geek, right ?

Re:most of them did it (0)

Anonymous Coward | more than 5 years ago | (#28510059)

money is not the motivation for everything in this world

True... Some men just want to watch the world burn...

Re:1 Million split 7 ways (5, Insightful)

quanticle (843097) | more than 5 years ago | (#28490151)

Well, just like the Ansari X Prize didn't cover the costs of developing and launching a suborbital rocket, the Netflix Prize isn't really meant to be a large enough prize to fully fund the development of a new recommendation algorithm. The purpose of the prize is to stimulate interest and get people started. The real reward will come when they turn their algorithm into commercialized software - the rewards from making such a thing applicable outside of Netflix could be large indeed.

Re:1 Million split 7 ways (0)

Anonymous Coward | more than 5 years ago | (#28490271)

That's one perspective on it, and a bit of a leap that this is will see much use outside of Netflix. On the other side, Netflix got tons of people to do something for much less than it would have cost them in-house. So: lots of losers, a few people who did okay and have a small chance of getting good money from it, and Netflix getting labor on the cheap. If the people who entered the competition did it out of a genuine labor of love then bully for them, but if someone entered that competition for the money then it was a bad decision.

Re:1 Million split 7 ways (1)

QuantumG (50515) | more than 5 years ago | (#28490589)

The X-Prize was designed to encourage the creation of a vehicle that would demonstrate the feasibility of a new market. It was backed up with a whole lot of market research which showed that people would happily sign up for a flight on such a vehicle. The anticipated business plan that it was trying to encourage was:

1. Build a vehicle that is very reusable and can put passengers into space.
2. Win the prize and get the PR.
3. Take bookings and reservation fees to fund the next flight.
4. Fly the first passengers in your winning vehicle.
5. Repeat 3, 4 until Profit!!

Unfortunately they didn't anticipate that some big personality like Burt Rutan would come along and do:

1. Convince an investor to invest *twice* the price to build a vehicle.
2. Win the prize and get the PR.
3. Use the PR to get more venture capital.
4. Took bookings and reservation fees for a new vehicle that hasn't even been built yet.
5. Never fly the winning vehicle again.. hang it in the Smithsonian like a freakin' monument or something.
6. Watch your schedule slip, try to catch up by cutting corners, have a fatal accident.
7. Announce that your vehicle will be delayed still more as you work on making it a "platform" for doing shit that wasn't in the original spec (can you say space shuttle?)
8. Demonstrate a fly over of your carrier aircraft, after having to delay the fly over due to malfunctions.
9. ?????? We're still waiting.
10. Profit?

Re:1 Million split 7 ways (0)

Anonymous Coward | more than 5 years ago | (#28493745)

what the hell are you rambling about? you just did that to invoke an overused meme, right? commercialization of rutan's winning craft is well underway.

Re:1 Million split 7 ways (1)

QuantumG (50515) | more than 5 years ago | (#28493837)

And will continue to be "well underway" for the next 5 years with no-one having flown in it is my bet..

Re:1 Million split 7 ways (5, Insightful)

neokushan (932374) | more than 5 years ago | (#28490177)

Pretty sure having it on their CV means they can effectively write their own pay cheque in terms of job opportunities.

Re:1 Million split 7 ways (1)

Pollardito (781263) | more than 5 years ago | (#28520401)

When they write their own paycheck, they should put Blockbuster in the account name

Re:1 Million split 7 ways (0)

Anonymous Coward | more than 5 years ago | (#28490671)

Man, I don't know about you (obviously), but $71k would go a pretty long way towards making my life a lot better.

And yes, I know money doesn't buy happiness, but I would be content with rent being paid and food on the table without worry.

Re:1 Million split 7 ways (1)

SunTzuWarmaster (930093) | more than 5 years ago | (#28493445)

If this was your only job, you would have to pay rent/utilities on the building where you work in addition to rent/utilities on the building where you live. You also have to pay your own health insurance, dental/vision, and retirement/401K.

The above should cut it down to around a bit above (if not below when you have an office) the average starting salary of an Engineer (55K at last I checked).

$1M ain't what it used to be.

Re:1 Million split 7 ways (1)

denzacar (181829) | more than 5 years ago | (#28491165)

Unless one of them kills all of their partners like in The Dark Knight that ain't much of a prize.

Yeah, but if they deliver him to the competition in a body bag they get another 500 hundred grand.
A million if alive, so they can teach him some manners first.

Re:1 Million split 7 ways (0)

Anonymous Coward | more than 5 years ago | (#28491547)

Well, I have already killed two of the other guys. Hope fully this doesn't tip off the others.

Re:1 Million split 7 ways (0)

Anonymous Coward | more than 5 years ago | (#28492427)

"Let's say" that taxes don't take half. There's pretty much nowhere in the country where someone earning 143k would be in the 50% tax bracket.

Re:1 Million split 7 ways (1)

GuruBuckaroo (833982) | more than 5 years ago | (#28492529)

only $71,428.57 each ... that ain't much of a prize.

That'd pay off my house, with a little to spare for new windows. I call that much of a prize.

Re:1 Million split 7 ways (1)

WinstonWolfIT (1550079) | more than 5 years ago | (#28493085)

First burn 70 large for laffs. Then say the prize isn't impressive. In that order.

Re:1 Million split 7 ways (0)

Anonymous Coward | more than 5 years ago | (#28493731)

Resume item worth much more.

Do they keep the prize money? (1)

whoever57 (658626) | more than 5 years ago | (#28490053)

Assuming no-one turns up a better score in the next 30 days, do the team members who work for Yahoo and AT&T get to keep their share of the prize money?

Re:Do they keep the prize money? (3, Informative)

lee1026 (876806) | more than 5 years ago | (#28490067)

AT&T have committed to giving all money to charity. The person at yahoo developed his entry while working at AT&T, so I will be surprised if yahoo gets any of it.

Bah, too complex; here's my analysis (1)

Smidge207 (1278042) | more than 5 years ago | (#28490055)

I just think there cannot be such 5 grades for movies. Since a 3-star or 4-star rating is differently given by people on a more or less random basis depending on the understanding that the user has about the difference that there should be between 3 and 4.

Are you sure that the contest rules don't allow to change stuff like that?

Are there some studies pointing to having 5 grades for evaluating movies being better than just having 2 as in Good and Bad?

I just think it can be split up like this:

Bad - Good
Waste of time - Interesting
Hate - Love
Forget - Remember
Ban - Bookmark
Discard - Add to Favorites
Ignore - Enjoy
Avoid - Recommend
Advise against - Support

Etc.. I think that maybe the algorithm can be more effective once ratings are just given in this way. Having only Love and Hate ratings might be a bit extreme, but all the ratings in between are for the bulletin boards, and not for a perfect recommendations system in my opinion. Maybe there should be a popup that lets the user choose which kind of definition suits best for their positive or negative rating. For example some movie might better fit "Interesting" than it would fit "Love", and some other movie might fit "Ignore" better than it would fit the description of "Hate"..

Well done! (4, Informative)

Slurpee (4012) | more than 5 years ago | (#28490059)

Well done Bellkor.

But now the real race begins.

Now that the 10% barrier has been reached, people have 30 days to submit their final results. At the end of the 30 days, whoever has the best result wins.

This is going to be a great month!

Re:Well done! (2, Informative)

nmb3000 (741169) | more than 5 years ago | (#28490953)

Now that the 10% barrier has been reached, people have 30 days to submit their final results. At the end of the 30 days, whoever has the best result wins.

That's true, but like the story title indicates, the prize may have been achieved. From the contest rules:

The RMSE for the first "quiz" subset will be reported publicly on the Site; the RMSE for the second "test" subset will not be reported publicly but will be employed to qualify a submission as described below. The reported RMSE scores on the quiz subset provide a public announcement that a potential qualifying score has been reached and provide feedback to Participants on both their absolute and relative performance.

So the publicly available submission beat the 10% mark, but only by a narrow margin of 0.05%. The private submission must also have surpassed 10% for them to be considered a preliminary winner, otherwise the contest goes on (at least, this is my understanding). In any case, I agree that the outcome should be interesting.

Re:Well done! (2, Interesting)

Wildclaw (15718) | more than 5 years ago | (#28493545)

Actually, this email has been sent out

"As of the submission by team "BellKor's Pragmatic Chaos" on June 26, 2009 18:42:37 UTC, the Netflix Prize competition entered the "last call" period for the Grand Prize. In accord with the Rules, teams have thirty (30) days, until July 26, 2009 18:42:37 UTC, to make submissions that will be considered for this Prize. Good luck and thank you for participating!"

they were able to get the extra 0.5% over the top (4, Funny)

circletimessquare (444983) | more than 5 years ago | (#28490071)

by simply ignoring data from anyone who ever rented SuperBabies: Baby Geniuses 2, Gigli, From Justin to Kelly, Disaster Movie, any movie by Uwe Boll and any movie starring Paris Hilton

suddenly, everything made sense

i assume you're joking, but.. (1)

spiffmastercow (1001386) | more than 5 years ago | (#28490283)

That actually makes a lot of sense... Remove the wildcards and you'll be able to get a much more accurate result for everyone else. You might suffer a bit when you're reccomending to people who like movies that are absolutely terrible, but you'd make up for it my not factoring that into the equation at all. This is, of course, assuming that only a few people actually watched those movies.

i was joking, however (5, Interesting)

circletimessquare (444983) | more than 5 years ago | (#28490339)

from the excellent nyt article about the competition in november:

http://science.slashdot.org/article.pl?sid=08/11/22/0526216 [slashdot.org]

it isn't bad movies that are the problem, taste in bad movies can still be uniform

the real problem is extremely controversial movies, most notably Napoleon Dynamite

http://www.imdb.com/title/tt0374900/ [imdb.com]

not controversial in terms of dealing with abortion or gun control, but controversial in terms of some people really found the movie totally stupid, while some people really found the movie to be really funny

movies like napolean dynamite are genre edge conditions, and people who apparently agree on everything else about movies in general encounter movies like this one and suddenly dramatically differ on their opinion of it, in completely unpredictable ways

Re:i was joking, however (3, Interesting)

bill_mcgonigle (4333) | more than 5 years ago | (#28491363)

Yeah, all the recommendation systems where I've bought or rented movies, and most of my friends all said I needed to see 'Fight Club', so I did, and ... meh.

Consider this list of movies I've bought/rated highly:

  12 Monkeys
  V for Vendetta
  Lost in Translation
  Donnie Darko
  A Beautiful Mind
  Dogma

I might be grouped with folks who enjoy flicks about identity, man vs. man, those who aren't easily offended, etc. But there doesn't seem to be as clear a way to find a group of people who find aggression offensive, which is basically the driving theme of Fight Club. Perhaps given enough negative ratings it could be possible, but even though I've clicked 'Not Interested' on all the Disney movies, they keep suggesting I want their latest direct-to-DVD crapfest, so I'm left to assume they're rating mostly based on positive ratings.

Re:i was joking, however (3, Interesting)

Pollardito (781263) | more than 5 years ago | (#28520509)

Perhaps given enough negative ratings it could be possible, but even though I've clicked 'Not Interested' on all the Disney movies, they keep suggesting I want their latest direct-to-DVD crapfest, so I'm left to assume they're rating mostly based on positive ratings.

A co-worker gets almost no recommendations at all from Netflix, and customer service told him that they generate recommendations based on ratings of 4 or 5 (though you'd think that the recommendations that they do generate would have to filter through similar movies that you've rated at 0). He was told to rate the movies that he likes higher in order to fix it, but that's never really accomplished anything as he has several hundred movies in the 4-to-5 range and maybe a dozen recommendations total.

I'm pretty sure that the Disney/children's movie recommendation flood that most everyone seems to be getting is driven by parents who don't actually love those movies, but are rating those movies on behalf of their children. That causes a weird connection to movies that they themselves enjoy, and it makes it seem like the same audience is enjoying both types of movie. They need to have an "I'm a parent" flag somewhere to help them sort that out

Re:i was joking, however (1)

bill_mcgonigle (4333) | more than 5 years ago | (#28520687)

Useful, thanks.

> They need to have an "I'm a parent" flag somewhere to help them sort that out

I'd love to have family member tags in the queue. Associating an age with them would be fine. They have a thing where you can have separate queues for different family members, but it's a separate login IIRC, and a real pain to manage cohesively. I'd much rather just have one queue, and have it send discs in a round-robin for thing1, thing2, and the folks.

Re:i was joking, however (0)

Anonymous Coward | more than 5 years ago | (#28492265)

at first i found the movie totally stupid but then i smoked pot and watched it again and found it totally funny and if i had to rate the movie now i would pick funny. the real problem is in determining who these "fringe reefer addicted" movie-people are, and through logic and deduction of a person's movie review library (for: harold & kumar 1 & 2, how high, don't be a menace in south central while drinking your gin and juice, cheech & chong, etc. et. all) to calculate a "stoner" ratio system which corrects for the controversial quotient. ya dig?

Re:i was joking, however (1)

moosesocks (264553) | more than 5 years ago | (#28501265)

Examining the standard deviation of the ratings received by each film should provide a reasonable (albeit imperfect) indicator of the "controversialness" of any given film.

Of course, there are a few ways to improve upon this, and I'm sure that the winning teams have taken this into account.

Re:i was joking, however (1)

vuffi_raa (1089583) | more than 5 years ago | (#28504247)

it isn't bad movies that are the problem, taste in bad movies can still be uniform

also there are people like my wife and I who share the netflix since we watch most of the movies together and have different tastes in movies: for one I HATE '30s and '40s dramas and she loves them so she will get them sometimes and I love B grade sci fi and horror movies and she isn't the biggest fan of them

Re:they were able to get the extra 0.5% over the t (2, Funny)

interkin3tic (1469267) | more than 5 years ago | (#28490341)

by simply ignoring data from anyone who ever rented SuperBabies: Baby Geniuses 2, Gigli, From Justin to Kelly, Disaster Movie, any movie by Uwe Boll and any movie starring Paris Hilton

Hey, I (along with the rest of my frat, our school hockey team, and most of the town) was in a movie starring Paris Hilton, you insensitive clod!

Re:they were able to get the extra 0.5% over the t (1)

Shikaku (1129753) | more than 5 years ago | (#28490649)

Hey, I (along with the rest of my frat, our school hockey team, and most of the town) was in Paris Hilton, you insensitive clod!

Fixed

Re:they were able to get the extra 0.5% over the t (1)

interkin3tic (1469267) | more than 5 years ago | (#28491003)

Kind of what I was implying...

Re:they were able to get the extra 0.5% over the t (1)

Hurricane78 (562437) | more than 5 years ago | (#28490373)

You can generalize that to any movie with an action hero, and a baby. Any movie with a pseudo-star (Hilton, Spears, Madonna, etc.). And any movie with Uwe Boll or similar people.

On a more serious note: I think the best way to improve recommendations, is to first relate the IMDB rating to the IQ of the rater. I found that more intelligent people do not like movies with a simple plot, because it bores them, and less intelligent people do not like movies with a complex, subtle plot, because they don't get it. You can further separate this into the EQ and the IQ, which will considerably improve the experience for emotional people (like most women).
I found that the best way to quickly and realistically do that, is to check the length of the sentences and the percentage of the questions in the comments of that user.

This will give you a 3D space with a rating, a EQ and a IQ axis.
Now apply the values of the user that wants to get recommendations as coordinates, and order by the distance from that point.
Then apply the traditional recommendation system with a blending factor.

I bet I could code that in less than an hour in Haskell.

P.S.: If you want to patent this, mind you that I will seriously kick your ass for doing so. I do not know any laws when it comes to giving credit. You can probably sue me afterwards, but it won't fix what I did to you. ^^ (Be fair to me, and I will be the nicest man you ever met.)

Re:they were able to get the extra 0.5% over the t (1, Insightful)

Anonymous Coward | more than 5 years ago | (#28491371)

You're being elitist and silly. Perhaps you could do some research, but I have seen no evidence that interest in simple vs. complex plots has anything to do with intelligence. Certainly the type of plot one likes in a movie is something reasonable to consider. But assuming a relationship to IQ or EQ from that is silly.

Re:they were able to get the extra 0.5% over the t (1)

thunderclap (972782) | more than 5 years ago | (#28493147)

by simply ignoring data from anyone who ever rented SuperBabies: Baby Geniuses 2, Gigli, From Justin to Kelly, Disaster Movie, any movie by Uwe Boll and any movie starring Paris Hilton

suddenly, everything made sense

Ok, From Justin to Kelly wasn't really that bad. Now, Ishtar...and Battlefield Earth; those were baaaad.

Wonderful. (0)

Anonymous Coward | more than 5 years ago | (#28490101)

Now will somebody please fix that goddamned Silverlight player?

Interesting (5, Informative)

coaxial (28297) | more than 5 years ago | (#28490207)

I published a paper using Netflix data. (Yeah, that group [slashdot.org] .)

It's certainly cool that they beat the 10% improvement, and it's a hell of a deal for Netflix, since it would have cost them more than a prize money paid out to hire the researchers, the interesting thing is whether or not this really advances the the field of recommendation systems.

The initial work definitely did, but I wonder how much of the quest for the 10% threshold moved the science, as opposed to just tweaking an application. Recommender systems still don't bring up rare items, and they still have problems with diversity. None of the Netflix Prize work address any of these problems.

Still, I look forward to their paper.

Re:Interesting (1)

Anonymous Coward | more than 5 years ago | (#28490439)

Here's the problem with the terms "rare" and "diversity":

1.) Rare could also be defined as unpopular. Trying to recommend unpopular movies is problematic. Is the computer program going to be able to discern under-rated (Glengarry Glen Ross) or just crap (Ishtar)

2.) Suggesting "My Little Pony" when I usually rent black and white Samurai movies could be called diversity. Do you want a program that recommends things that are different or things that are similar?

Re:Interesting (3, Insightful)

Trepidity (597) | more than 5 years ago | (#28490683)

Trying to recommend unpopular movies is problematic. Is the computer program going to be able to discern under-rated (Glengarry Glen Ross) or just crap (Ishtar)

That is indeed an interesting question, and I think it's what the grandparent meant when he pointed out Netflix's contest didn't really address it. The performance measure Netflix used was root-mean squared error, so every prediction counts equally in determining your error. Since the vast majority of predictions in the data set are for frequently-watched films, effectively the prize was focused primarily on optimizing the common case: correctly predict whether someone will like or not like one of the very popular films. Of course, getting the unpopular films right too helps, but all else being equal, it's better to make even tiny improvements to your predictions of films that appear tons of times in the data set, than to make considerable improvements to less popular films' predictions, because the importance of getting a prediction right is in effect weighted by the film's popularity.

You could look at error from a movie-centric perspective, though, asking something like, "how good are your recommender algorithm's predictions for the average film?" That causes you to focus on different things, if an error of 1 star on Obscure Film predictions and an error of 1 star on Titanic predictions count the same.

Re:Interesting (1)

Wildclaw (15718) | more than 5 years ago | (#28493529)

so every prediction counts equally in determining your error

True, but not true. If we had talked about plain mean error I would have agreed. But as it stands, not every error counts the same. Making a serious error (guessing 1 instead of 5) costs 16 times what making a small error (guessing 4 instead of 5) costs.

With popular movies you usually have enough data to come decently close with guesses. Sure, you can optimize them more to get even closer with the guesses. But it is a minor profit. On the other hand, you don't have much data on the less popular movies, so the guesses will be far less accurate, and it will pay off far more to make those guesses more accurate.

If anything, you could say that the competition focused on minimizing the amount of serious errors.

Re:Interesting (2, Insightful)

Trepidity (597) | more than 5 years ago | (#28496237)

That's true, but since there's not a huge range in ratings, that root-squaring doesn't have nearly as big an effect as the many orders of magnitude difference in popularity. I don't recall the exact numbers offhand, but I think the top-10 movies, out of 17,500, account for fully half the weight.

Re:Interesting (1)

failedlogic (627314) | more than 5 years ago | (#28491189)

Awesome combo. I'd pay $100 for a Zatoichi vs My Little Pony movie.

Re:Interesting (2, Insightful)

coaxial (28297) | more than 5 years ago | (#28514361)

1.) Rare could also be defined as unpopular. Trying to recommend unpopular movies is problematic. Is the computer program going to be able to discern under-rated (Glengarry Glen Ross) or just crap (Ishtar)

You know what. I actually like Ishtar. I really do. The blind camel, and the line "We're not singers! We're songwriters!" gets me every time.

So really, the even harder problem is to know when to buck your your friends and go with the the outlier. It's hard, because kNN methods work pretty well, and they're all about going with the consensus of whatever cluster you're in.

Re:Interesting (0)

Anonymous Coward | more than 5 years ago | (#28490479)

Cost? Try a genetic algorithm, a decent machine, and some free time. Hell, I wouldn't be surprised if it was a genetic algorithm that combined the other teams' findings into a fitter solution.
The 'cost' of that would be literally time. Can't get much better than that.

Re:Interesting (1, Interesting)

Anonymous Coward | more than 5 years ago | (#28490569)

You think using a Generic Algorithm could help?! are you kidding??!! :-)

The search space is far too great and what would you actually be searching for with this technique? (just curious)

I would hope to see techniques evolved from an energy variant of Markov Decision Processes! now that 'would' be a nice direction.

Re:Interesting (0)

Anonymous Coward | more than 5 years ago | (#28491603)

The initial work definitely did, but I wonder how much of the quest for the 10% threshold moved the science, as opposed to just tweaking an application.

Exactly. The first big "ah-ha" moment was that matrix factorization is the way to go for datasets like this. The second big "ah-ha" moment was that, hey, we need to be using temporal data, too! It's been my opinion that pretty much everything else has been the operators manually training their programs and blends against the quiz set ("tweaking," to put it gently). Put another way, I seriously doubt Netflix could take all the same code, run it over their up-to-date database, and still see a 10% improvement; I don't know if they'd even see a 9% improvement. Personally, I don't think there's even "10%" more signal there.

"recommendations" (2)

Gothmolly (148874) | more than 5 years ago | (#28490321)

Who listens to these sort of things anyway?

Re:"recommendations" (0)

Anonymous Coward | more than 5 years ago | (#28490379)

Who listens to these sort of things anyway?

Anyone who listens to pandora.

Re:"recommendations" (0)

Anonymous Coward | more than 5 years ago | (#28491989)

Who listens to these sort of things anyway?

Anyone who listens to pandora.

And exactly who would listen to someone who released all evils upon mankind?

I've been doing behavioral data analysis... (0)

Anonymous Coward | more than 5 years ago | (#28490473)

this is my career, doing behavioral data analysis.

I was really excited when they announced the project, and began tinkering with it. Unfortunately, they stripped so much data out of the exercise that it became an academic statistical exercise, rather than an insightful behavioral modeling exercise. It rewarded an approach where the training population was continually wildly segmented, with different model parameters on each segment.

I wish they had opened up more data, there could have been lots of cool stuff in there. I'm not saying it was bad or against it, just not *my* cup of tea. I rather prefer statistically simple models driven off unique customer segments.

real world (1)

TheSHAD0W (258774) | more than 5 years ago | (#28490541)

So... What does this mean in real-world analysis? What does the score represent? Since the score shown seems to be smaller-is-better, does this mean that 85+% of the movies recommended won't be attractive to the target, and less than 15% would be found interesting?

That doesn't seem very accurate...

Re:real world (0)

Anonymous Coward | more than 5 years ago | (#28491965)

Roughly speaking, it means that, on average, the recommender will guess your actual rating for a movie to within ~0.85 points (on a five-point scale). Granted, that doesn't sound much better than being within ~0.95 points (which is what they had). Supposedly, though, it makes a big difference in the way the entire movie list would be sorted for you in order of "most likely to like".

Re:real world (2, Interesting)

Wildclaw (15718) | more than 5 years ago | (#28493591)

~0.85 points (on a five-point scale)

Actually the scale is not 0-1-2-3-4 but 0-1-4-9-16 as they use Root-Mean-Square. Just thought it was worth pointing out.

FaiTlz0rs (-1, Flamebait)

Anonymous Coward | more than 5 years ago | (#28492261)

tto, can 3e a of a solid dose Sanctions, and

I'm confused... (0)

Anonymous Coward | more than 5 years ago | (#28492321)

Recent results show a 10.05% improvement

How many library of congresses is that?

Film recommendations (3, Interesting)

michuk (996480) | more than 5 years ago | (#28493637)

Does anyone find Netflix recommendations any good anyway? I used http://criticker.com/ [criticker.com] for quite a while and was very happy about the recommended stuff. Recently switched to http://filmaster.com/ [filmaster.com] (which is a free service) and it's equally good, even though both probably use a pretty simple algorithm compared to Nextflix.

Re:Film recommendations (2, Interesting)

jfengel (409917) | more than 5 years ago | (#28494989)

Reasonably good, actually. I often add 4 star movies to my queue, and rarely regret it.

The problem is the bell curve. There aren't a lot of 5 star movies out there, and I've seen them. There are a lot of 3 star films, but my life is short and I don't want to spend a lot of time on movies I merely "like".

In fact, it's not really a bell curve. I rarely provide 1-star or 2-star ratings simply because it's not at all difficult for me to identify a film I'm going to truly hate. I don't have to waste two hours of my life to find out whether I'd merely dislike the new Transformers movie or whether it will fill my soul with disgust.

The left side of the curve is actually quite fat with movies that simply won't interest me at all. The existing algorithm is actually fairly good at telling me I won't like them. The hard part is picking out the very few movies that ARE worth my time.

They do show both the average and expected rating for each film. What I'd really like to see is a list sorted by the difference: where do I stand out from the crowd? Such movies are likely to have extra appeal.

So the 10% difference isn't completely worthless, but the real problem is that they're pursuing the wrong goal. There's a lot of information they're dropping on the floor.

Re:Film recommendations (1)

michuk (996480) | more than 5 years ago | (#28495105)

To me it's essential to read a few reviews (even very short ones like "too sentimental" or "powerful and original") by people I know to have a similar movie taste to me. That's why I like the two websites I mentioned above -- they exactly tell you whose opinions you should care for and whose you can safely ignore. On the other hand, http://jinni.com/ [jinni.com] and http://clerkdogs.com/ [clerkdogs.com] have some pretty cool ideas as well - they suggest you movies based on your mood and your taste. Still in beta though, so it does not always work as expected, but worth trying anyways.

Re:Film recommendations (1)

jfengel (409917) | more than 5 years ago | (#28495459)

The key advantage Netflix has over other services is that it's right there. They know what you watch and you don't have to go searching.

Of course you still have to go though the back catalog and talk about all the things you've already seen, which runs in the hundreds for somebody who likes movies. That pain is essentially the same with any service.

But going forward, Netflix can present you the opportunity to rate a movie more easily. It's a small user-interface thing, but significant.

Re:Film recommendations (3, Informative)

coaxial (28297) | more than 5 years ago | (#28514459)

I believe that Netflix is still using Cinematch. You could look into movielens [movielens.org] . It's from the GroupLens group at U Minn.

[E]ven though both probably use a pretty simple algorithm compared to Nextflix.

You do know that Netflix said on the outset "You're competing with 15 years of really smart people banging away at the problem." and it was beat in less than week [slashdot.org] .

That's not to meant as a knock against Netflix's engineers, but more about that they didn't really build a state of the art recommender system. Simple SVD [sifter.org] (aka latent semantic indexing [wikipedia.org] outperfomed them as well.) They did something a bit more than straight up kNN clustering, but that was pretty much it.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>