Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Anonymity of Netflix Prize Dataset Broken

Zonk posted more than 6 years ago | from the there-are-degrees-of-anonymity dept.

Security 164

KentuckyFC writes "The anonymity of the Netflix Prize dataset has been broken by a pair of computer scientists from the University of Texas, according to a report from the physics arXivblog. It turns out that an individual's set of ratings and the dates on which they were made are pretty unique, particularly if the ratings involve films outside the most popular 100 movies. So it's straightforward to find a match by comparing the anonymized data against publicly available ratings on the Internet Movie Database (IMDb) (abstract on the physics arxiv). The researchers used this method to find how individuals on the IMDb privately rated films on Netflix, in the process possibly working out their political affiliation, sexual preferences and a number of other personal details"

cancel ×

164 comments

Sorry! There are no comments related to the filter you selected.

Sexual preferences? (4, Funny)

tygerstripes (832644) | more than 6 years ago | (#21491743)

Who goes out of their way to rate "Anal Whores 3" online?

Re:Sexual preferences? (2, Funny)

morgan_greywolf (835522) | more than 6 years ago | (#21491923)

Bill Clinton?

Re:Sexual preferences? (5, Funny)

mh1997 (1065630) | more than 6 years ago | (#21491931)

Who goes out of their way to rate "Anal Whores 3" online?
The good thing about porn flicks, as a general rule, is that they're too bland to have really bad plots. The search for good dialogue strays too far off the beaten path established by the social mores of the target market, be that old men, college students, or perverts out on dates. There are pornos with solid plots, just rarely pornos with complicated plots.

What they generally aren't is full of capers designed by crackheads in search of sexual relief, or a dominatrix dying to destroy the gold market with a Da Vinci alchemy machine only a cat burglar from Hoboken could steal.

Yes, the plot of Anal Whores 3 is as convoluted as it is kitschy. Mercedes and Veronica Diamond forcibly enlist the help of happy-go-lucky and half-a-second-out-of-prison pizza delivery man Hawk (Peter North) to steal the pieces to a machine that turns lead vibrators into gold. Hawk isn't halfway to a cup of coffee with his wise cracking cohort, Tommy (Johnny Cockring) when he finds himself back in the burglary game. Casing out a heist he meets nun/professional patron of the arts/double agent/love interest Jessie Jane (vows of bestiality can put the kibosh on even the best of cinematic love interests). When you throw in a CIA agent (Dick Coburn) and a couple of double dildos, you've managed to make the world's most convoluted porno....

Re:Sexual preferences? (-1, Offtopic)

Anonymous Coward | more than 6 years ago | (#21491973)

At least the plot isn't as bad as slashdot written fiction.

Re:Sexual preferences? (4, Interesting)

styryx (952942) | more than 6 years ago | (#21492083)

That's the plot of Hudson Hawk. Good flick.

Re:Sexual preferences? (3, Informative)

mh1997 (1065630) | more than 6 years ago | (#21492849)

If I had mod-points, I'd mod you up insightful. I didn't think someone would spot where I copied the review from so fast.

Re:Sexual preferences? (5, Funny)

Minwee (522556) | more than 6 years ago | (#21493109)

Yes, they would have to have watched Hudson Hawk to do that. That narrows the field considerably.

Re:Sexual preferences? (0)

Anonymous Coward | more than 6 years ago | (#21494063)

Is it me, or is this the fist time the verb "to narrow" has been used in a porno context?

Re:Sexual preferences? (1)

ioshhdflwuegfh (1067182) | more than 6 years ago | (#21492251)

Speaking about sexual preferences, "anal whores" and "a pair of computer scientists from the University of Texas", one of this pair, the first one from the signatures, has written a on his personal web site about the second:

Advisor: Vitaly Shmatikov <-- seriously awesome

Re:Sexual preferences? (2, Funny)

Jtheletter (686279) | more than 6 years ago | (#21493829)

The search for good dialogue strays too far off the beaten path established by the social mores of the target market

I see what you've done there..... ;)

Re:Sexual preferences? (1)

ammoQ (454616) | more than 6 years ago | (#21493307)

Most flix at youporn.com have been rated by several users. At least I've heard so.

Probabilities (4, Insightful)

dj245 (732906) | more than 6 years ago | (#21491765)

The researchers used this method to find how individuals on the IMDb privately rated films on Netflix, in the process possibly working out their political affiliation, sexual preferences and a number of other personal details"

This is a loaded statement. The most you can determine is that if a person likes movie A, B, C and D but hated E and F, there is a higher probability they are a guy. If they liked Z but didn't like X, there is a higher probability they might be a republican than not. You're still anonymous.

Unless, of course, you're one of the three people that liked "Glitter". Then I think they might have something on you.

Re:Probabilities (0)

Anonymous Coward | more than 6 years ago | (#21491837)

How does IMDb have your political affiliation or sexual preference?

Re:Probabilities (3, Insightful)

Se7enLC (714730) | more than 6 years ago | (#21491861)

I think they're on to something here. They cracked the anonymity by using the public movie ratings (and the dates those ratings were made) as a key. If the user has rated enough movies (especially some of the less-often-rated movies) you can uniquely identify which user they are. Once you know which user they are, you have now connected a username to the list of private ratings.

Now, they go one step too far to say that you can determine anything but movie preferences out of a movie rating list. Just because somebody liked or disliked brokeback mountain doesn't mean they are gay or straight, just like their opinion of michael moore movies doesn't give political affiliation.

It will tell you what movies they rented, though, and some people might not be happy having their movie-renting history publicly available.

Re:Probabilities (0)

Anonymous Coward | more than 6 years ago | (#21491927)

Yes but if you rate Brokeback Mountain, and many other films featuring gay sex (IMDb is full of adult movies FYI), one can claim to high certainty that you are gay. If you are rating a film which is not mainstream, and which is not being rated by many people, you are chosing the steeper side of Bell curve, and it is easy to guess you.

Re:Probabilities (2, Insightful)

Dare nMc (468959) | more than 6 years ago | (#21491993)

one step too far to say that you can determine anything but movie preferences out of a movie rating list.

also your taking a aggregate of the household. So a household (will call them Chen'ys) had a gay kid, and the devil living in the same house with a Saint... good luck figuring out when the gay kid updates the queue, and when the Wife, or the Devil is at the keyboard.

Re:Probabilities (2, Interesting)

coolGuyZak (844482) | more than 6 years ago | (#21493049)

Some tech-savvy households may enable profiles on Netflix, enabling each person to track their likes & dislikes independently. (I did this for my GF, who has wildly disparate tastes from me). I'm not sure what effect that would have on the data. It'd certainly be neat if the scientists could differentiate between individual and multiple users using a particular profile.

Re:Probabilities (1)

morgan_greywolf (835522) | more than 6 years ago | (#21492031)

Now, they go one step too far to say that you can determine anything but movie preferences out of a movie rating list. Just because somebody liked or disliked brokeback mountain doesn't mean they are gay or straight, just like their opinion of michael moore movies doesn't give political affiliation.
No, but if they liked Brokeback Mountain, every Michael Moore movie ever produced, An Inconvenient Truth, and Fritz the Cat, you can probably bet that they're a card-carrying liberal.

Re:Probabilities (0, Troll)

BorgCopyeditor (590345) | more than 6 years ago | (#21492117)

No shit, Sherlock.

Re:Probabilities (1)

IBBoard (1128019) | more than 6 years ago | (#21492201)

Or maybe they like cowboy films and are open minded, as well as liking expose material and documentaries (not sure about the others).

Maybe they're not, but there's always the possibility.

Re:Probabilities (1)

ThreeGigs (239452) | more than 6 years ago | (#21492035)

some people might not be happy having their movie-renting history publicly available

Being able to "see other ratings by this user" yields their movie rental history, algorithms or no. Is this what the big fuss is about?

Re:Probabilities (1)

ShiningSomething (1097589) | more than 6 years ago | (#21492337)

That's not entirely true. You can rate movies you have not rented (maybe a friend has, or you saw them on TV or at the theater).

Re:Probabilities (1)

ioshhdflwuegfh (1067182) | more than 6 years ago | (#21492401)

That's not entirely true. You can rate movies you have not rented (maybe a friend has, or you saw them on TV or at the theater).
That's not entirely true. You can rate movies you have not seen at all (maybe a friend has, or you saw their trailer, read about them, etc...)

Re:Probabilities (0)

Anonymous Coward | more than 6 years ago | (#21493003)

Just because somebody liked or disliked brokeback mountain doesn't mean they are gay or straight...
Well, for the most part yes. Pretty much those who saw Brokeback Mountain are into the butt sex.

From the paper (1)

JPMH (100614) | more than 6 years ago | (#21492109)

From the paper:

First, we can immediately find his political orientation based on his strong opinions about "Power and Terror: Noam Chomsky in Our Times" and "Fahrenheit 9/11." Strong guesses about his religious views can be made based on his ratings on "Jesus of Nazareth" and "The Gospel of John". He did not like "Super Size Me" at all; perhaps this implies something about his physical size? Both items that we found with predominantly gay themes, "Bent" and "Queer as folk" were rated one star out of five. He is a cultish follower of "Mystery Science Theater 3000". This is far from all we found about this one person, but having made our point, we will spare the reader further lurid details.

Re:Probabilities (5, Insightful)

Chapter80 (926879) | more than 6 years ago | (#21492277)

I think you're missing the point.

If you rate a handful of movies on ImDB, under the persona "MyNickname12345" and that can be traced to your personal MySpace page, you have made that choice. No problem.

If you then submit 100 movie ratings to Netflix, assuming that it is PRIVATE information that will not be linked back to you, and then Netflix releases the data to the public, now the 100 movies can be correlated to you, and your name can be revealed. Researchers have shown how PRIVATE DATA released to the public can be linked to already public information. PROBLEM!

Re:Probabilities (1)

ioshhdflwuegfh (1067182) | more than 6 years ago | (#21493561)

If you then submit 100 movie ratings to Netflix, assuming that it is PRIVATE information that will not be linked back to you, and then Netflix releases the data to the public, now the 100 movies can be correlated to you, and your name can be revealed. Researchers have shown how PRIVATE DATA released to the public can be linked to already public information. PROBLEM!
What is the capital problem you are talking about? What does "PRIVATE DATA released to the public" mean? Are they private? PRIVATE? Public?

Re:Probabilities (0)

Anonymous Coward | more than 6 years ago | (#21494209)

This is, of course, why one keeps multiple screen names or places passwords on things that he doesn't want the world to see.

Liked Brokeback Mountain == gay liberal cowboy (1)

Rogerborg (306625) | more than 6 years ago | (#21492533)

Now do you see the POWAH inherent in that knowledge?

The German Police (1)

thegermanpolice (1194811) | more than 6 years ago | (#21491775)

The German Police will be pleased.

only a matter of time (1)

downix (84795) | more than 6 years ago | (#21491777)

Privacy is becoming a fleeting thing in this interconnected world. Perhaps we should reanalyze our perspective on it all?

Re:only a matter of time (2, Informative)

SatanicPuppy (611928) | more than 6 years ago | (#21491873)

Perhaps if we're obscure and pretentious enough, no one will want to spy on us! Brillant!

The world changes. Learn to live with it.

Re:only a matter of time (4, Interesting)

phobos13013 (813040) | more than 6 years ago | (#21493125)

Actually TFA seems to suggest that the more obscure and pretentious we are, the easier it is the track us. If we become homogeneous drones voting on the top 100 films, we are safe! Even so, I don't plan to become a homogeneous drone...

Re:only a matter of time (1)

pegdhcp (1158827) | more than 6 years ago | (#21494177)

As somebody also mentioned above, it becomes easier to spot you when your parameters put you closed to the edge of bell shaped curve. Also it is important to remember movies being one of the most common forms of art (how many different sculptures you see in a year, and how many movies...) they have a greater granularity to match individual tastes and preferences....

However I guess that, IMDB being public, and Netflix private, there should be a shift in expressed opinions. Assuming that, it is accepted "BAD TASTE" to be interested in a private investigator taking cases on animals, if I liked Ace Ventura I will be more likely to admit it in an anonymous environment...

Re:only a matter of time (1)

blahplusplus (757119) | more than 6 years ago | (#21491969)

But then we'd have to re-analyze capitalism itself, an I don't think society is ready for *that* rich people would simply pay for organizations to falsify their data, it would be one sided.

Do what now? (4, Insightful)

faloi (738831) | more than 6 years ago | (#21491785)

It doesn't sound like the anonymity of the prize set was broken through any fault of NetFlix. It sounds like some sampling of users made the mistake of rating movies on a site where the info is publicly available, and a site where it's not. All they did was correlate the two.

So the lesson is, basically, don't post stuff that you don't want to be public to a website that makes it public, right? This is sounds roughly like blaming the DMV for figuring out a car owners likely political leanings by the bumper stickers on their car.

Re:Do what now? (1)

Dare nMc (468959) | more than 6 years ago | (#21492099)

was broken through any fault of NetFlix.

just because someone choose to go public with liking "The Rise of Theodore Roosevelt" doesn't mean they should know that the company will take some seemingly private data linking you to really likeing "brokeback mmoutain", and the series "The L word" and publicize it later. and that the combination of your post, and the combination now violates netflix's privacy policy (in spirit)
IE they say they will only disclose "on an anonymous basis" anything but your reviews. We know from the AOL disclosure, linking data to a number that represents you. Doesn't qualify as anonymous anymore.

Re:Do what now? (4, Insightful)

IBBoard (1128019) | more than 6 years ago | (#21492169)

Exactly - all they did was found that there was a correlation that might mean that the people are the same on IMDB and NetFlix. There's also the possibility that they're different people and that they just voted similar on different places.

Besides, this all relies on people voting for a) really obscure films so they can be easily identified and b) voting similarly or identically on lots of films so that they can get a better idea as to whether it is the same person based on them liking the same films the same amounts.

Just because two people from two different data sets both like (and are the only people in the data sets to like) lemon and custard jam as well as peanut butter with chips doesn't mean they're the same person, it just means they could be the same person and have similar tastes in obscure foods.

Re:Do what now? (3, Insightful)

Peter Mork (951443) | more than 6 years ago | (#21492253)

Exactly - all they did was found that there was a correlation that might mean that the people are the same on IMDB and NetFlix.

Caveat: I haven't had a chance to pore over the statistical calculations. However, the paper notes that their similarity measure was 38 standard deviations from the norm. Assuming the math is valid, this seems on par with a DNA test, which also provides a correlation. I wouldn't be so quick to dismiss the results until you can find a serious methodological problem.

Re:Do what now? (2, Insightful)

IBBoard (1128019) | more than 6 years ago | (#21493487)

While yes, they did get a very perfect match on that record, the line about it is:

...our algorithm identified the records of two users the Netflix Prize dataset with eccentricities of around 28 and 15, respectively.


Granted they went for a small number of IMDB users due to their TOS, but that's still a tiny fraction. They mention finding a perfect match in IMDB and 1/8th of the NetFlix database towards the start of the report (although the sentence is a bit clunky and unclear). If that's their general accuracy then even if they can perfectly match some people (a statistical possibility) then they can't match enough to leave most people needing to worry.

Re:Do what now? (3, Informative)

arvindn (542080) | more than 6 years ago | (#21493827)

"Besides, this all relies on people voting for a) really obscure films so they can be easily identified "

not true -- obscure films help a little bit but not too much. we put up a recent draft of our paper in which the dependence on obscure movies is much reduced.

"b) voting similarly or identically on lots of films so that they can get a better idea as to whether it is the same person based on them liking the same films the same amounts."

again not true at all. one of the main claims of our paper is that our method is tolerant to an INCREDIBLE amount of noise. we have the math to back this up.

--Arvind Narayanan

Re:Do what now? (2, Insightful)

JPMH (100614) | more than 6 years ago | (#21492275)

Their lesson is that it can take surprising little public information to identify you.

For example, ratings on a scale of 1-5 for 2 movies, and a knowledge of when they were seen to within 14 days, was suffiecient to identify the complete data histories of 40% of the Netflix clients. As the authors say, that's the kind of information cooleagues give out every day around the water cooler.

Repeating the experiment with a knowledge of 8 movies, 6 hits in the database would be sufficient to identify the personal histories of 99% of clients included in the Netflix data.

Re:Do what now? (0)

Anonymous Coward | more than 6 years ago | (#21492355)

Any time you can connect a substantial amount of activity to a specific person, you have the potential of getting more information about that person by cross-correlating the activity with information in other databases. This is one of the most basic (and most disturbing) techniques of data mining, and it's what makes data mining so powerful.

By connecting each transaction with an individual, NetFlix makes it much easier to get through the layer of anonymity. They didn't make it easy to get through, but they didn't make it very difficult either.

Re:Do what now? (2, Informative)

roadkill_cr (1155149) | more than 6 years ago | (#21493191)

True, but in the real world, it's not as simple as that. There are cases of publicly available databases that you gave no permission to grant access to (for example, AOL's release of their search queries). There are other cases when a database has restricted access, but a person with access to it takes it and uses it in comparison with other databases available. Hackers are always a trouble; since some have gotten into such "secure" areas as the CIA and IRS, what's to keep them from potentially getting into any database?

The problem is one of privacy - in the worst case (or, for those who are cynical, common case) we have none. There's been some answers proposed to solve this. If you're interested, I'd start by reading the original paper on k-anonymity, which attempts to create privacy in a world where one can possibly have access to any database, ever. It can be found here: http://privacy.cs.cmu.edu/people/sweeney/kanonymity.html [cmu.edu] . (There are, of course, a multitude of other methods; k-anonymity is just a good starting point.)

In Soviet Russia... (-1, Redundant)

Celarnor (835542) | more than 6 years ago | (#21491795)

In Soviet Russia, movies watch you!

FUCKING RETARD (-1, Offtopic)

Anonymous Coward | more than 6 years ago | (#21491921)

DIE ALREADY

Anonymity broken by stupidity (2, Interesting)

CastrTroy (595695) | more than 6 years ago | (#21491815)

Seems like it was only broken because the identity of the people was posted somewhere else, along with the ratings. My only question is how they connected the rankings on Netflix, to the rankings on IMDB. Does Netflix take the liberty of submitting all the users rankings to IMDB for them, and also include their name with this data? If you just have anonymous dataset A, with anonymous dataset B, you could match up users from both and figure out which person in A is the same person in B, but you still wouldn't know who the person is. However, if you now have dataset B be not anonymous, then it's not too difficult to compare movie ratings and find out who the people are.

Re:Anonymity broken by stupidity (1)

Neon Spiral Injector (21234) | more than 6 years ago | (#21491991)

They are just saying it is likely a person rated a movie on Netflix and IMDb at roughly the same time. That is the correlation which is need to connect the anonymous with the publicly posted information.

While I do rate a few films on IMDb I usually do them in batches, where on Netflix I rate the movie as soon as I'm finished viewing it. So the time link wouldn't be there between my two accounts.

Re:Anonymity broken by stupidity (1)

stranger_to_himself (1132241) | more than 6 years ago | (#21492317)

What NetFlix did that was stupid was include the names of the movies in their dataset. There was no need for this for the prize (unless anybody was using the names for prediction I suppose), anonynous identifiers would have been okay.

Re:Anonymity broken by stupidity (1)

Neon Spiral Injector (21234) | more than 6 years ago | (#21492437)

If a person liked season 1 of Stargate: SG1 it would be a good idea to recommend season 2 to them. Goes for sequels too. So yeah, titles are needed a little bit.

Re:Anonymity broken by stupidity (1)

Constantine XVI (880691) | more than 6 years ago | (#21493035)

And methinks you need more than just titles as well. Ex: Joe rents "Shaun of the Dead", and hates it. It would be a good idea to not suggest "Hot Fuzz", as they share many of the actors, directors, etc.

Re:Anonymity broken by stupidity (1)

Random_Goblin (781985) | more than 6 years ago | (#21494005)

yeah but what would you think if i said i hadn't seen evil dead yet?

Re:Anonymity broken by stupidity (1)

Dare nMc (468959) | more than 6 years ago | (#21494191)

methinks you need more than just titles as well.

exactly why titles aren't needed.
You let the AI algorithm make the basket. When you have processed a big enough data set a good AI algorithm will have matched the pattern already. IE if 99% of the people who hate tt0365748 also hated tt0425112 then the algorithm will have already picked that out of the sample data. It would even more so avoid suggestions like recommending identical title movies, that have nothing in common.

You would only need the aggregate data, if you were picking which movies to create, or how to sell the movie to the person. IE a genetic algorithm may pinpoint the reason, for the correlation between the two movies. But only after the pattern match already picked out the correlation. Now it might make allow you tell the person person who liked "Bridges of Madison county", and "Field of Dreams" that "Twister was also filmed in Iowa" when you recommend it, but that wasn't allowed in this competition.

Re:Anonymity broken by stupidity (1)

zippthorne (748122) | more than 6 years ago | (#21493937)

Not really. They just needed to provide sequel and tie-in information in their dataset. The exact relationship doesn't matter. And you can't get that info from the titles necessarily, anyway. I mean, how is a text-based algorithm going to know that "Serenity" is the movie tie-in to the series, "Firefly?"

Re:Anonymity broken by stupidity (1)

jfengel (409917) | more than 6 years ago | (#21492711)

They were hoping that by giving the name of the movie they could pull in other sources of data (RottenTomatoes, IMDB) to make better guesses. They were seeking more than just better curve-fitting of the data points. If nothing else, the IMDb's huge pile of data points of ratings gives you a lot more fodder for your collaborative filter, but only if you can tie in the names of the movies.

But yeah, that introduces a data leak.

Re:Anonymity broken by stupidity (1)

stranger_to_himself (1132241) | more than 6 years ago | (#21492871)

They were hoping that by giving the name of the movie they could pull in other sources of data (RottenTomatoes, IMDB) to make better guesses.

Are you sure? Are people using data from outside the training set? Because if what you say is true then they're essentially asking people to use some kind of probabilistic record linkage to include external databases, which would automatically include the personal identifiers. This would be highly dubious behaviour.

did it work? (2, Interesting)

Speare (84249) | more than 6 years ago | (#21491859)

The researchers used this method to find how individuals on the IMDb privately rated films on Netflix, in the process possibly working out their political affiliation, sexual preferences and a number of other personal details

{tongueincheek}Yeah, but the question is, will knowing those personal facts generate better movie recommendations?{/tongueincheek}

When there's a significant prize at stake, researchers can try all sorts of slimy tricks to win. (I'm not saying that's the motive behind this report, but there are many "researchers" going for the prize.) And when there's significant profits at stake, a corporation will damn-fire-certainly use whatever means they can use to maximize those profits, regardless of whether it might be "ethical."

Re:did it work? (0)

Anonymous Coward | more than 6 years ago | (#21491937)

When there's a significant prize at stake, researchers can try all sorts of slimy tricks to win.

THERE CAN BE ONLY ONE!

Re:did it work? (1)

johnbr (559529) | more than 6 years ago | (#21492481)

And when there's significant profits at stake, a corporation will damn-fire-certainly use whatever means they can use to maximize those profits, regardless of whether it might be "ethical."
Here, let me fix that for you:
When there's significant profits at stake, individual humans will damn-fire-certainly use whatever means they can use to maximize those profits, regardless of whether it might be "ethical".

How does this break anonymity? (2, Insightful)

Anonymous Coward | more than 6 years ago | (#21491879)

For those who haven't rated movies on IMDB, such as myself - and I imagine a large proportion of subscribers.

Data-mining and the actual problem (4, Interesting)

Anonymous Coward | more than 6 years ago | (#21492005)

There are two things going on here. One, many people are asking how you could identify any personal information about people based on their movie preferences. The answer is data-mining. Very sophisticated techniques exist to do things exactly like this, i.e. take a data set and find out about the people.

The second problem is that by deanonymizing the NetFlix data, you can start to cheat on the NetFlix prize. The requirement to win $1 million is that your recommendation engine is 10% better than the one they are currently using. However, if you can learn the exact preferences of some users in the dataset (i.e. by finding the rest of their ratings on IMDB) then you can hardcode that into your recommendation engine and get the recommendations for these users exactly right. This can boost your score even though your actual system is no better than the existing one. This is known as over-fitting to the data.

Finally, this paper is over a year old. Can we please have some new news?

Re:Data-mining and the actual problem (0)

Anonymous Coward | more than 6 years ago | (#21493017)

This won't work because submitted algorithms are run against data that wasn't made public.

Easy solution (4, Funny)

Thanshin (1188877) | more than 6 years ago | (#21492011)

Every time you feel the need to vote 10 in Glitter, also vote 10 to The Godfather.
Every time you cheer for Brokeback Mountain, also put a 10 in Huge Knockers MXII.
Every time you want to express your love for Dersu Uzala, vote a 10 in Spice World, with added commentaries.

That way, everybody will know you're a security conscious computer scientist. Or a squizophrenic moron.

Re:Easy solution (1)

ioshhdflwuegfh (1067182) | more than 6 years ago | (#21492795)

Exactly. Or, as one character in a comedy I've seen sometimes ago (I can't remember the title) is tripping all horrified in Amsterdam after eating some brownies, saying to his friend something like "I once watched gay porn,... I didn't know it... girls just never showed up,... they never showed up!!". He then jumps around the bar all in horror, a terrible trip he has, then the bar owner tells him something like: "come down white boy, there is no hash in those brownies".

Re:Easy solution (0)

Anonymous Coward | more than 6 years ago | (#21492985)

Eurotrip [imdb.com] . I saw it the other day and it's surprisingly entertaining for a teen comedy.

Re:Easy solution (1)

Sancho (17056) | more than 6 years ago | (#21493407)

That was Eurotrip. You just admitted to watching Eurotrip.

Wait. So did I.

Re:Easy solution (1)

atraintocry (1183485) | more than 6 years ago | (#21493977)

I think that was EuroTrip. "We do not sell Hash brownies here, we are simple Dutch bakery. Now put your clothes back on, white boy!"

Re:Easy solution (0)

Anonymous Coward | more than 6 years ago | (#21493895)

Derzu Uzala! You must be one of the 23 people ever to have seen that film! I liked it the first time, and 20 years later, watched it again and fell asleep while doing so.

requires another (partial)public revealing to work (3, Informative)

call -151 (230520) | more than 6 years ago | (#21492013)

The summary is somewhat misleading- the only accounts that can be identified are those that belong to people who also rate on IMBD and who have thus chosen to make at least some of their ratings public. If person X rates 1000 movies on Netflix and has made 20 or so ratings on IMDB publically available, then it is possible to infer with some small uncertainty which of the anonymized individuals in the NetFlix database they are. Thus you have possibly figured out their ratings of the other 980 movies they rated for Netflix but did not post on IMBD. Interesting, but not earth-shattering or a serious breach of privacy, I would say.

Re:requires another (partial)public revealing to w (2, Insightful)

TubeSteak (669689) | more than 6 years ago | (#21492623)

Interesting, but not earth-shattering or a serious breach of privacy, I would say.
And who exactly are you to say so?
Because it isn't a Credit Card # or SSN it isn't serious?

A) Some people would rather go to jail or commit suicide than admit to something embarrassing they'd rather keep private. Privacy isn't (just) about hiding (illegal) things from the Government.

B) Demographic information is something you can never take back and can never change.
At least I can get a new credit card & SSN.

Re:requires another (partial)public revealing to w (1)

Constantine XVI (880691) | more than 6 years ago | (#21493123)

Re: B, you can usually change any sort of non-biological (and, using extreme measures, some biological ones too) demographic information about yourself. There's nothing that says you can't suddenly turn from liberal to conservative or vice versa, or get married (or turn gay/lesbian), etc.

OT: is there a way to escape greaterthan/lessthan signs?

Re:requires another (partial)public revealing to w (1)

Lijemo (740145) | more than 6 years ago | (#21493747)

OT: is there a way to escape greaterthan/lessthan signs?

apersand-lt-semicolon results in <

apersand-gt-semicolon results in >

(no spaces or dashes.)

The world is not on fire (3, Insightful)

puppetluva (46903) | more than 6 years ago | (#21492089)

This is total hyperbole.

All they researchers are saying is that they can deduce some of your preferences based on your other preferences. Of COURSE you can do that, that was the whole point of the contest Netflix put up.

What they are _not_ saying is that they now know who you are, where you live, or anything uniquely identifying about you. So basically, you are still anonymous.

I'm starting to tire of news headlines that claim the world is on fire when someone actually just does something slightly derivative from the norm and thinks they are brilliant. The noise from these non-events mask actual brilliant achievements and make it seem that everyone is doing banal work.

Re:The world is not on fire (2, Insightful)

Peter Mork (951443) | more than 6 years ago | (#21492365)

All they researchers are saying is that they can deduce some of your preferences based on your other preferences.

The researchers are making a stronger claim. They are stating that based on actual public ratings (available from IMDB) they can generate actual private ratings published by Netflix under the guise of anonymity. As the paper notes, someone competing for the Netflix prize could use this data to improve the accuracy of their prediction algorithm. However, the point of this paper is to reveal that public ratings can be used to identify purportedly anonymous private ratings.

As a comparison, imagine if the public information consisted of the dates that various people went to the doctor for a yearly physical. This is hardly sensitive information. Now imagine that your insurance company provided a list of (id, date, diagnosis) records. Ostensibly, the id field is an arbitrary (anonymous) identifier. The paper shows that based on limited background knowledge (a handful of (date, 'physical exam') records), an attacker could reverse engineer your diagnosis history.

Re:The world is not on fire (1)

ioshhdflwuegfh (1067182) | more than 6 years ago | (#21493261)

The researchers are making a stronger claim. They are stating that based on actual public ratings (available from IMDB) they can generate actual private ratings published by Netflix under the guise of anonymity. As the paper notes, someone competing for the Netflix prize could use this data to improve the accuracy of their prediction algorithm. However, the point of this paper is to reveal that public ratings can be used to identify purportedly anonymous private ratings.
The researchers are making a stronger claim. They state:

As shown by our experiments with cross-correlating non-anonymous records from the Internet Movie Database with anonymized Netflix records (see below), it is possible to learn sensitive non-public information about a person's political or even sexual preferences.

Re:The world is not on fire (2, Informative)

JPMH (100614) | more than 6 years ago | (#21492393)

Othe the other hand, if somebody *already* knows who you are, the lesson is that it can take surprising little public information to identify your entire history of ratings at Netflix.

For example, the authors found for 40% of individuals, accurate ratings on a scale of 1-5 for only *two* random movies,together with a knowledge to within 14 days of when they were seen, would be sufficient to identify an individual in the dataset. As they comment, that's the kind of information cooleagues give out every day around the water cooler.

Repeating the experiment with a knowledge of 8 movies, 6 hits in the database would be sufficient to identify the personal histories of 99% of the people in that data.

Re:The world is not on fire (1)

TubeSteak (669689) | more than 6 years ago | (#21492745)

All they researchers are saying is that they can deduce some of your preferences based on your other preferences. Of COURSE you can do that, that was the whole point of the contest Netflix put up.

What they are _not_ saying is that they now know who you are, where you live, or anything uniquely identifying about you. So basically, you are still anonymous.
Did you even read the summary?

They took anonymous ratings & discovered they can link some of them to IMDB usernames. We can argue over whether or not those IMDB usernames are "uniquely identifying" or "anonymous" but they definitely say something about who you are.

I'm sure a percentage of those IMDB usernames are easily linked to real people through a trivial google search. Does that break this alleged veil of anonymity? Datamining isn't that hard these days.

Re:The world is not on fire (1)

puppetluva (46903) | more than 6 years ago | (#21492901)

I did read the summary and felt that the IMDB linkage was a real stretch.

Linkage of that kind is only useful if the user-populations for IMDB commenters and Netflix commenters are the same (at least 50%) and that most people make the same comments and ratings on both systems in the same way _most_ of the time. Chances are that if the populations are _not_ the same and that the commenters don't mostly duplicate their ratings for every movie in each place. . . In that case, you then you probably get more false positives than positive correlations.

If the ratings on imdb and netflix are nearly exactly the same for certain users, all that you have determined is a potential netflix user-id (which is probably a unique number in the dataset, not necessarily their username) to IMDB username linkages - but not a certain one.

Re:The world is not on fire (1)

arvindn (542080) | more than 6 years ago | (#21493901)

Netflix claimed that the data is anonymous.

They said you couldn't identify a person's record in the dataset even if you know some (or all!) of their ratings.

We showed that that's not true. Even if there's a LOT of noise. That's all there is to it.

--Arvind Narayanan

What are you rating in IMDB vs Netflix (4, Insightful)

SmallFurryCreature (593017) | more than 6 years ago | (#21492221)

As far as I know in IMDB you are rating the overall quality of the movie, not I agree with it OR I want to see more like this.

One example, Shindlers list, great movie, do NOT want to see it again. Same with Grave of the fireflies. Some movies just ain't for multiple viewings. They are my "favorite movies I never want to see again".

On the other hand I got movies I can watch any day of the week, but that I would NEVER rate as highly. Cannonbal run is one such movie. It watch it far too often, but I wouldn't call it a good movie. You can always fine me ready for a Jacky Chan movie or a spagethi western.

Is the netflix rating system a "I liked this movie and want to see more like it" system or a "This movie was brilliant and I would highly recommend it too everyone else" type of rating system?

Granted some people get it confused, probably the same people that use the slashdot moderation system to silence views they don't like, but that only makes basing conclusions on user ratings even more problematic.

I can rate a movie highly even if I do not agree with it, simply because it is good. And I can rate a movie I really like to watch as crap simply because I know I like watching crap.

I don't like the godfather movies, I can see they are high quality, I just don't like them. So my rating them would be fairly high as for quality, but low for 'I want to see more like this'.

I thought that the netflix system was "I want to see more like this" based. Surely nobody is so stupid as to think a quality rating and a "i like this" rating system are the same? Or am I completly in the wrong in seeing a difference between the two? Am I insane in thinking that you can see a movie as being a great artwork and still not liking it or viceversa?

Re:What are you rating in IMDB vs Netflix (1)

apt142 (574425) | more than 6 years ago | (#21492851)

I would think that "I like this" and "This is a good movie" are two different measurements on a film.

I must be a pessimist, but I don't believe the average Joe would agree with that statement. I think most people would see the two statements as synonymous. That is, if they even think about the distinction. Mostly I think they'd just grab their "gut" feeling and go with it.

I suppose we could test the argument by comparing movies that are ranked high on quality with total movie rentals or some other more precise measurement of watching frequency.

Re:What are you rating in IMDB vs Netflix (2, Interesting)

xtracto (837672) | more than 6 years ago | (#21492875)

One example, Shindlers list, great movie, do NOT want to see it again. Same with Grave of the fireflies. Some movies just ain't for multiple viewings. They are my "favorite movies I never want to see again".

Just out of curiosity, why don't you want to see those films again? both of them are really good films and although I would not see them every weekend (as for example Sin City), I enjoy watching them from time to time. The plot is interesting, the photography/drawing is nice and the screen writing is well done.

I find it difficult to understand your statement, "favorite movies I never want to see again", if you do not want to see them again, then you do not enjoy watching them... unless you dislike enjoyment and only watch films that make you cry or have a bad time (I would suggest you United 93... worst film I have seen in a looong long time... or Broeback Mountain, a 1 hour marlboro country ad).

I not not know about the netflix scoring algorithm but I have found criticker.com quite reliable for my tastes.

Am I insane in thinking that you can see a movie as being a great artwork and still not liking it or viceversa?
It might be akin to the "La Gioconda" painting. Everybody says it is the best piece of art of all the time, yet, after having watched it *twice* live in the Louvre I have yet to find something special about it (I prefer for example, paintings from Giovanni Paninni, which is relatively unknown)

Simple as you said, I do NOT enjoy watching them (2, Interesting)

SmallFurryCreature (593017) | more than 6 years ago | (#21493877)

The comment "favotire movie I never want to see again" is one I got from a review of Grave of the Fireflies that I just happened to totally agree with. Don't read the reviews, just watch it yourselve and if you are not into Anime just set that aside for the duration of the movie, then ask yourselve again, if you can understand that comment.

It is powerfull movie, like Shindlers List, but not a happy tale. I am not talking a tear jerker movie here, I am talking a "we will all burn in hell for this" movie. Tear jerkers I can take, Christmas in August is one. Sad tale, nicely told but ultimately human. It makes you sad, not sick of humanity.

Perhaps I am just too emotional about this kinda stuff, one reason might be that I grew up with halfunderstood tales of "that was were your great-uncle was picked up". When you realize just why your grandmother had 9 brothers and sisters yet you never met any. I got one aunt, my grand-parents had 3 kids, a starvation story like GotF hits a lot closer with a history like that. (The dutch hunger winter)

I enjoy all kinds of movies and would NOT have NOT watched these two, but that doesn't mean I want to see them again. There are some people who list Shindlers List as a feel good movie because it 'ends well'. I suppose you might see it that way, I don't.

I can regonize your statements that the photography is nice and the screen writing is well done, but the plot is intresting? To you it is a plot, to me it is a sickening part of history that I am far too close to.

Perhaps it is a bit like how Richard Pryor's monologue about the 200th celebration of the US was not exactly all that cheerfull.

Terry Pratchets Nanny Ogg describers at one point the difference between merry and mirth (or something like that) she describes how she was joyfull when her child was being born but she wasn't exactly chuckling at the time. Enjoying a movie and enjoying it are two different things, at least for me. I can't describe it any clearer.

Re:What are you rating in IMDB vs Netflix (2, Insightful)

Danny Rathjens (8471) | more than 6 years ago | (#21493019)

As far as I know in IMDB you are rating the overall quality of the movie, not I agree with it OR I want to see more like this.

No. You give people way too much credit if you think their ratings on public sites are that nuanced or objective. I think most people just rate things on how well they like it themselves. A significant portion seem to even just give 10s to anything they like, too.

I also find it amusing how the votes tend to congregate somewhere in the 3rd quartile a bit above average(e.g. 7 on a 1-10 scale) rather than 5.5 where it would be if people ranked things more fairly. (I wonder if this is associated with that effect where people always rank themselves above average despite evidence to the contrary, as well.)

Re:What are you rating in IMDB vs Netflix (2, Insightful)

ps236 (965675) | more than 6 years ago | (#21493201)

> I also find it amusing how the votes tend to congregate somewhere in the 3rd quartile a bit above average(e.g. 7 on a 1-10 scale) rather than 5.5 where it would be if people ranked things more fairly

I'm not sure about that. People will tend to watch films they think/hope they will like. So, the ones where they think 'that'll be absolute poop' they won't bother watching, so, hopefully, won't bother rating.

So, people should rate fewer films as 'poop' than as 'great', because they select only the 'hopefully good' films to review.

If you forced people to go to see and review all films, even the ones where you have to drag them screaming through the door, then the average rating would almost certainly decrease considerably.

Re:What are you rating in IMDB vs Netflix (1)

Danny Rathjens (8471) | more than 6 years ago | (#21493363)

Yeah. Not just that, but also people are probably more motivated to actually vote for a movie they liked. Why bother to go look at the imdb entry at all for a movie you didn't like? (Unless it's something that just *needs* to be voted low to warn others. e.g. Highlander II) :)

Re:What are you rating in IMDB vs Netflix (1)

ZorbaTHut (126196) | more than 6 years ago | (#21493325)

In addition to the previous comment, when I rate movies I'm usually rating movies that I remember. If a movie is entirely unmemorable, I'm not gonna remember that I watched it and thus I'm not going to rate it.

That means the above-average movies and the total flops get rated, but not the below-average movies.

Re:What are you rating in IMDB vs Netflix (0)

Anonymous Coward | more than 6 years ago | (#21493661)

To a certain extent you are wrong in that. People don't get netflix subscriptions to rent the same movie(s) over and over again, that's what the movie bin at S-Mart is for, people get netflix subscriptions so that they can see lots of different movies once or maybe twice each. To use your example, having seen one movie that qualified as a single viewing favorite did that stop you from appreciating the next one?

Taste in art is like taste in anything else, it's 100% subjective. That which you describe as great artwork might not be what I describe as great artwork. That which you like I might not. The real difference isn't between liking it and appreciating it, those things work hand in hand to come up with an overall rating, the real difference is between "How do I think I would rate this movie all things considered?" and "How do I think everyone else would rate this movie all things considered?".

The real beauty of the law of large numbers is that it doesn't matter which view you take of how to rate something, or how to mod a post, get enough people doing it and the numbers will converge onto a single value which we can call the rating, because it's all subjective to begin with.

The real problem of the law of large numbers is that it's often used on very noisy datasets, a single value called rating might not be enough information to predict whether or not someone will like a movie, but that's what the prize is all about.

Re:What are you rating in IMDB vs Netflix (1)

arvindn (542080) | more than 6 years ago | (#21494037)

Yes, such differences in meaning exist.

However, when you're talking about dozens of movies, all you need is a correlation. Our algorithm is powerful enough to tolerate a large amount of noise. If you read the paper, we were able to match up users between imdb and netflix with a very high level of confidence, in the sense that the best match was 15-30 standard deviations away from the second best match. In statistics terms, that's a insanely close match.

--Arvind Narayanan

I think a lot of you naysayers... (1)

mdm-adph (1030332) | more than 6 years ago | (#21492247)

...would be a lot more appreciative of this proof of concept if someone trawled Slashdot threads to see how often you feed trolls by responding to comments with a "-1" rating... :P

This is a 'research' paper? (1, Insightful)

RocketJeff (46275) | more than 6 years ago | (#21492311)

First, we can immediately find his political orientation based on his strong opinions about "Power and
Terror: Noam Chomsky in Our Times" and "Fahrenheit 9/11." Strong guesses about his religious views can
be made based on his ratings on "Jesus of Nazareth" and "The Gospel of John". He did not like "Super
Size Me" at all; perhaps this implies something about his physical size? Both items that we found with
predominantly gay themes, "Bent" and "Queer as folk" were rated one star out of five. He is a cultish
follower of "Mystery Science Theater 3000". This is far from all we found about this one person, but having
made our point, we will spare the reader further lurid details.


Finding a paragraph like this in a research paper makes me call into question the motives and intentions of the 'researchers.' They seems sort of like the Jerry Springer of research (since he's just trying to help the families he has on his show...).

They imply that the person didn't like "Super Size Me" because he's probably fat (or are they trying to imply that he has a problem with gaining weight and is jealous?).

Also, they imply that because he rated two "predominantly gay theme" items as poor he must not be homosexual. Or are they implying that because he rented/rated these that he must be gay (because who would ever rent them otherwise).

The fact that they use the "there's more juicy stuff about this guy, but we can't tell because we're serious researchers" line at the end is the pièce de résistance that really shows what motivates these researchers.

Re:This is a 'research' paper? (3, Insightful)

nagora (177841) | more than 6 years ago | (#21492659)

You're missing the point completely. Other people will be using "data mining" of this sort, and making serious decisions about whether you support terrorism, or are just generally not a "good citizen", and they won't be revealing their judgments to the public to let them know what might be going on.

TWW

Re:This is a 'research' paper? (2, Insightful)

amccaf1 (813772) | more than 6 years ago | (#21493345)

From TFA:

He did not like "Super Size Me" at all; perhaps this implies something about his physical size?
Or maybe he's a manager of a McDonalds. Or a part-time Ronald McDonald. Or...

Re:This is a 'research' paper? (1)

PhysicsPhil (880677) | more than 6 years ago | (#21493813)

Finding a paragraph like this in a research paper makes me call into question the motives and intentions of the 'researchers.' They seems sort of like the Jerry Springer of research (since he's just trying to help the families he has on his show...).

It's clear you didn't read the paper. To be sure, the quoted paragraph did appear in the paper, which of course was selected for the summary because it was the most interesting. The full paper is 24 pages of substantially heavier research and analysis. The paragraph in question was actually towards the end of the paper in a 'case study' section indicating what kind of information might be plausibly derived from an anonymity attack against the NetFlix database.

Also in the paper are one lemma, five theorems, a discussion thereof, a presentation and discussion of the the de-anonymizing algorithm, along with an interesting discussion of spareness within the original Netflix database (i.e. how similar are records from two different people).

Re:This is a 'research' paper? (1)

ioshhdflwuegfh (1067182) | more than 6 years ago | (#21494083)

[...]This is far from all we found about this one person, but having made our point, we will spare the reader further lurid details.
Also in the paper are one lemma, five theorems, a discussion thereof, a presentation and discussion of the the de-anonymizing algorithm, along with an interesting discussion of spareness within the original Netflix database (i.e. how similar are records from two different people).
What I want is not some tight point, nor an arrow pointing to this point, but exactly the lurid details, (and preferably but not necessarily how they link to all these theorems and the algorithm.)

Brokeback's decline (1)

spideyct (250045) | more than 6 years ago | (#21492331)

Wait - you mean if I enjoyed a movie with a gay theme, people are going to assume I'm gay?

Anyone think the IMDB rating of Brokeback Mountain is going to plummet dramatically. (It is 7.8 today)

And of course, if it does, we will be able to correlate the timing of the sudden drop with the publishing of this slashdot article, allowing us to link the slashdot readership with imdb users. Now we have your Netflix ratings, IMDB ratings, AND slashdot postings all correlated...

Better Description (1)

dlsmith (993896) | more than 6 years ago | (#21492637)

I think the real problem here is being buried in misguided analysis about the meaning of anonymity and associating movie preferences to political affiliation, etc.

Here's what's really been demonstrated: private information about users of some IMDB accounts who have rated movies on both IMDB and Netflix has be made public, despite Netflix's implicit assertion that releasing anonymous data is "safe." The user himself has not really been compromised -- nobody knows his address, phone number, names of family members, etc. -- but people now know more about the IMDB account than was intentionally published. When the user publicly posted his opinions about 5 (say) favorite movies, he did not expect his private opinions about 100 others, as expressed in Netflix, to also be publicly associated with that account.

The practical impact isn't clear. If the private information were conveniently published by IMDB, so that nobody had to work very hard to view it, it might sway how likely readers are to trust a certain reviewer. The impact of that change in trust doesn't seem very meaningful, though, and in any case, the private information *isn't* conveniently published. If, under similar circumstances, there were a correlation between private information and an eBay account, then there could be a real financial impact.

Another concern is that, if other factors have already made it possible to correlate an IMDB account with a real person, then someone can make the jump to associating all this private data with that individual. For example, I might link to my IMDB profile from my blog so all my coworkers can see my public reviews, not realizing that it's now possible for them to determine what movies I've privately watched.

wait until sometime 2008... (0)

Anonymous Coward | more than 6 years ago | (#21493005)

then we can have this discussion again over more prevalent movies that are controversial. whoever did this research paper really should have waited til Golden Compass comes out. I use GC as an example because the movie when it comes out is going to have a lot of people not pleased with it. Also Prince Caspian comes out in '08 as well. While totally off based, those two movies are going to be what defines the new IMDB in the given year.

I can say that whatever off beat movies that come out are going to have substantial rating as well, but Golden Compass is going to have the most impact, just because its going to be very disliked because of the whole history surrounding it(and yes there will more then likely be boycotts opening day)and its producers.

enough said? I think so.

what utter nonsense (1)

e-scetic (1003976) | more than 6 years ago | (#21493357)

If you think you can determine political affiliation based on how someone rates movies, especially in America, then you're just plain retarded.

To take an example, a left-winger might rate Michael Moore flicks poorly because one thing about Moore's stuff is he almost always seems to avoid more effective ways of making his points. They agree with the message, disagree with the methodology or style of film. On the other side of things, a libertarian, Goldwater Republican, "conservative", etc., might rate Moore's Sicko highly, because there is undoubtably something wrong and shameful with health care in America whether you believe in socialized medicine or not.

But you know what - it wouldn't surprised me if the day came when your movie ratings came back to haunt you. America and other countries do seem to be headed in that direction.

Old news: see "New Scientist" 11 November 2006 (1)

DavidHumus (725117) | more than 6 years ago | (#21494069)

Here ahref=http://technology.newscientist.com/article/mg19225776.800-has-netflix-given-away-the-answers-in-its-software-competition.htmlrel=url2html-4572 [slashdot.org] http://technology.newscientist.com/article/mg19225776.800-has-netflix-given-away-the-answers-in-its-software-competition.html> - sorry, subscription required for full text of the article, so I'll just reveal one piece of nonsense (not to diminish their accomplishment):

Competition judge Charles Elkan at the University of California, San Diego, agrees that the method could work if enough Netflix users also use IMDB, but he believes it will be possible to detect and disqualify cheats when they submit their computer code. Narayanan and Shmatikov aren't convinced. "There are techniques to obscure how data is introduced within the thousands of lines of code you'd submit," Shmatikov says.
Yeah, someone's going to hand over a million dollars after you tell them "here's the algorithm, just ignore that large chunk of hex over there."
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?