Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Recommendation Algorithm Wants To Show You Something New

ScuttleMonkey posted more than 4 years ago | from the try-it-you'll-like-it dept.

Programming 90

Several sources are reporting on a new metric that computer scientists are going after with respect to recommender systems — recommendation diversity. "In a paper that will be released by PNAS, a group of scientists are pushing the limits of recommendation systems, creating new algorithms that will make more tangential recommendations to users, which can help expand their interests, which will increase the longevity and utility of the recommendation system itself. Accuracy has long been the most prized measurement in recommending content, like movies, links, or music. However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer."

cancel ×

90 comments

Sorry! There are no comments related to the filter you selected.

So... (-1, Offtopic)

Jorl17 (1716772) | more than 4 years ago | (#31290062)

Soo, uhm....?


errr....



Wanna make out?!

That's called an "contextual ad engine". (3, Insightful)

Animats (122034) | more than 4 years ago | (#31290078)

creating new algorithms that will make more tangential recommendations to users, which can help expand their interests,

The advertising industry already has that technology. Their idea of "expand interests" usually involves shopping, of course.

Re:That's called an "contextual ad engine". (2, Insightful)

blahplusplus (757119) | more than 4 years ago | (#31290132)

The real issue is people's finite attention, I notice even with recommendation systems on amazon.com there is no way I could ever read everything they recommended to me and still have a life. It may be neat for movies but even then I'm sure there will be a list of recommended stuff that you simply can't get aroudn to.

I really think someone should add up all the hours required to experience every movie/game released in a year and compare it to the the average persons free time, a lot of stuff is over-produced and it would be good if someone was out there modelling how many products you could possibly want to experience over a yearly period.

Re:That's called an "contextual ad engine". (0, Offtopic)

wjousts (1529427) | more than 4 years ago | (#31290312)

The real issue is people's finite attention, I notice even with recommendation systems on amazon.com there is no way I could ever read everything they recommended to me and still have a life.

I think you're missing the point. The point is to try and sell you stuff. You're not obligated to read (or even buy) something just because Amazon recommended it.

Re:That's called an "contextual ad engine". (1)

SomeJoel (1061138) | more than 4 years ago | (#31290390)

I really think someone should add up all the hours required to experience every movie/game released in a year and compare it to the the average persons free time

Now there's a useless metric. Stuff is produced to satisfy the spectrum of interests. And the "average person" (who does not exist) would consume some small percentage of everything.

Re:That's called an "contextual ad engine". (0)

Anonymous Coward | more than 4 years ago | (#31297472)

Only useless to someone who is as clueless as you are.

Re:That's called an "contextual ad engine". (4, Insightful)

Kjella (173770) | more than 4 years ago | (#31290650)

I really think someone should add up all the hours required to experience every movie/game released in a year and compare it to the the average persons free time, a lot of stuff is over-produced and it would be good if someone was out there modelling how many products you could possibly want to experience over a yearly period.

But that makes good recommendations more important, not less. If you go into a library it's highly unlikely you'll be able to read every book in it, but does that matter? You just want to read the good books about things that interest you. If Spotify was on full shuffle and you could get everything from death metal to yodeling in the next song, you wouldn't want it - you'd go back to your own favorites. On the other hand, if everything is interpolated you only get more and more of the same. People don't work like that, you may have your favorite food but it's not something you want to perfect and have every day.

A good search helper should be something in between - keeping to things you're reasonably likely to like but on the other hand challenge you a little to explore and listen to things a little outside your normal repertoire. Yes of course I realize the marketing potential here in sending the masses to their new hit wonder but I don't think the concept is that unreasonable. Think about how your friends are influencing your music taste, they're not interpolating they're gently pulling in the direction they like. If they hit the right mix this would be a real asset because you go to that site because of the good recommendations and that's not such an easy thing to copy.

Re:That's called an "contextual ad engine". (2, Interesting)

Skreems (598317) | more than 4 years ago | (#31291002)

Last.fm used to have a setting that was basically like this... they gave you an "obscurity" slider which let you set a preference for stuff that was a good match (via the usual "people who like x also like" algo) but removed the stuff that was over a certain threshold of popularity. I found a ton of bands that I really like this way, but at some point they took away the feature, or hid it in some way.

Long story short, I have found systems like this very useful, and hope to see more of them.

Re:That's called an "contextual ad engine". (1)

tool462 (677306) | more than 4 years ago | (#31291036)

I kind of get this service by listening to XM. There are enough different types of stations, each fairly tightly focused, that I effectively tell it my preferences by which stations I program into my favorites. But then each station is run by DJs who are playing what they like within that format. I have found a TON of new music this way. I'd guess that, outside of my old "standards" (hs/college music) most of the new music I've purchased has been by bands I wouldn't have heard of if I didn't hear them on XM.

Re:That's called an "contextual ad engine". (1)

ottothecow (600101) | more than 4 years ago | (#31291500)

DJ's are great. I stream The Current [publicradio.org] (89.3 in Mpls/St Paul, streaming link here [publicradio.org] ) all of the time because they are a real independent radio station with DJ's who are free to play what they like and no corporately mandated playlist. Sure, there are artists/songs liked by many DJs that are heard frequently but the amount of stuff they pull out of their music collection is astounding. The benefit of having real DJ's with real opinions is that you can find ones who you like better than others and they will do a good job of guiding you to music you may like.

I particularly like the DJs who cover the afternoon and evening weekday shifts and have been introduced to a lot of music through them.

Re:That's called an "contextual ad engine". (1)

darthdavid (835069) | more than 4 years ago | (#31291634)

I think you'd like WVBR then... http://wvbr.com/ [wvbr.com] (93.5 in the Ithaca NY area, it's a Nonprofit owned and run mostly by Cornell students but it isn't directly affiliated with the University).

Re:That's called an "contextual ad engine". (1)

darthdavid (835069) | more than 4 years ago | (#31291696)

And in case it wasn't obvious from the fact that I've recommended them, they play some kickass tunes too :)

Re:That's called an "contextual ad engine". (1)

ottothecow (600101) | more than 4 years ago | (#31291770)

I'll check it out.

What I like about the current though is that it decidedly does not have that college radio vibe. It is a straight up professionally managed station (part of the MPR network) with lots of contact with artists for things like in-studio performances and concert engagements and a staff that has spent a long long time in the industry. One of the aforementioned DJ's used to DJ on a college station and I like his sets a lot better in the past 5 years since the current started and the other one is the sister of Paul Westerberg and has a long history of DJing on some great (albeit short-lived) stations.

Re:That's called an "contextual ad engine". (1)

darthdavid (835069) | more than 4 years ago | (#31293030)

Well WVBR gets some of that 'college radio vibe'. Mostly the good parts like the informal atmosphere. They play plenty of current music, as well as classic rock. They also play lots of concerts (though these are more or less all by local bands, plenty of good music just don't expect to hear anyone particularly famous).

Re:That's called an "contextual ad engine". (1)

nanoflower (1077145) | more than 4 years ago | (#31296670)

My own favorite is http://www.kcrw.com/ [kcrw.com] because of their live performances. Some great live performances in their studio in their Morning Becomes Eclectic archives

Re:That's called an "contextual ad engine". (1)

Doggabone (1025394) | more than 4 years ago | (#31299046)

I'm a regular listener of Radio Paradise http://www.radioparadise.com/ [radioparadise.com] , largely because they'll often put songs after each other that sound great in sequence, but are of completely different genres. Independently owned, not what I'd call a college vibe.

Re:That's called an "contextual ad engine". (1)

Jah-Wren Ryel (80510) | more than 4 years ago | (#31291780)

If you go into a library it's highly unlikely you'll be able to read every book in it, but does that matter? You just want to read the good books about things that interest you.

I think you could define the ideal recommendation system by adding just one word to your definition above:

You just want to read the good books about things that will interest you.

Re:That's called an "contextual ad engine". (1)

blahplusplus (757119) | more than 4 years ago | (#31294206)

"But that makes good recommendations more important, not less"

I disagree, I think most recommendation systems today are "good enough". Think about the time things are released today, most people today go for new stuff and merely want to keep track of recently released stuff, so it's not so much knowing exactly rather then getting X into the awareness of others and there are lots of avenues besides recommendation engines for this.

I mean lots of interesting things are often times recommended even NOW with current algorithms, I have shit tonne of books I've book marked through recommendations or looking at other peoples public lists, even though I know I can't get to them all.

I still think browsing other peoples lists (aka like delicious or Stumble upon) are also great options.

Re:That's called an "contextual ad engine". (2, Interesting)

martin-boundary (547041) | more than 4 years ago | (#31290756)

I like to use the [ and ] keys in mplayer to speed up time when watching movies. It's not difficult to keep up with the plot and dialog up to about 1.5x the speed, and this makes the typical 2 hour movie run in 1 hour 20 min, or the typical 40 min series run in 26 min. I'll speed up time more when the movie is crap, and less if I want to really enjoy everything.

The point is you have more than the binary watch/don't watch choice available to you, if you're willing to assert control over your own time in creative ways.

Re:That's called an "contextual ad engine". (1)

ooshna (1654125) | more than 4 years ago | (#31291022)

If I could find a video player that allowed me to speed up the film say 1.5x but still keep the sound the same pitch I would be very happy

Re:That's called an "contextual ad engine". (1)

Tekfactory (937086) | more than 4 years ago | (#31291156)

MythTV is supposed to have this capability

Re:That's called an "contextual ad engine". (1)

martin-boundary (547041) | more than 4 years ago | (#31291250)

mplayer -speed 1.5 -af scaletempo movie.avi

Re:That's called an "contextual ad engine". (2, Informative)

Tekfactory (937086) | more than 4 years ago | (#31291254)

Ok, replying to Parent instead of myself.

MythTV has a feature called Timestretch that does this

Supposedly VLC can also do this.

As well as WinAmp, Media Player classic with external filters, gomplayer, the list goes on and on.

http://www.tomshardware.com/forum/236916-49-timestretching-more-stretching-time-times [tomshardware.com]

I'm kind of hoping Tivo adds something like this eventually.

Re:That's called an "contextual ad engine". (2, Insightful)

Sir_Lewk (967686) | more than 4 years ago | (#31291084)

Dude, don't forget stop and smell the roses. Not everything in life is supposed to be a race.

Re:That's called an "contextual ad engine". (1)

bertok (226922) | more than 4 years ago | (#31292654)

Dude, don't forget stop and smell the roses. Not everything in life is supposed to be a race.

That's almost as bad as my parents. They'll miss the first 5 minutes of the movie, then sit down a little bit, stand up and go get themselves a drink, sit back down, stand up and go to the bathroom or whatever, and so on. They'll end up watching maybe 20% of a movie is small random pieces. I'm not just talking about over-the-air broadcast, but even DVDs, which they could pause, but don't.

Then the questions start: "who's that guy?"... "why are they shooting at each other?"...

I just don't show them movies any more. It doesn't matter how good it is, they'll ruin it for themselves, and piss me off until it's ruined for me too.

Re:That's called an "contextual ad engine". (1)

Doggabone (1025394) | more than 4 years ago | (#31299198)

Mine are like that - I watch movies with them I've already seen. They enjoy the movie - yes, I'd think it was ruined to watch a movie that way but they enjoy it. And I enjoy the time with them, since I'm not paying as much attention to the movie and can answer "Who's that guy? What'd I miss?" without missing the movie.Apparently they re-watch the ones they really like when I'm not around, and those times the movie isn't background to my visit - they settle down more, or pause/rewind the movie. When I have company, I don't pause the movie either.

Also, sometimes you can speed up a thing, get the story, but lose a lot by speeding up the pace. It depends a lot on what you're watching, but for some things the pace is as important as the dialogue. Other films are saved, or nearly so, by the time compression.

Re:That's called an "contextual ad engine". (1)

Danny Rathjens (8471) | more than 4 years ago | (#31293298)

This works great for the olympics. I speed up the between race silliness then drop down to normal speed for the race/event. :)
But for tv shows I just use } once or twice which is 1.1 or 1.2 times normal speed.
(and as mentione dbelow you need the af = scaletempo setting in your ~/.mplayer.config so that sounds don't get pitch-distorted at the different speeds.

Re:That's called an "contextual ad engine". (0)

Anonymous Coward | more than 4 years ago | (#31293618)

Yeah, I do that too. Wait... you're talking about how you watch porn, right?

Re:That's called an "contextual ad engine". (1)

LostCluster (625375) | more than 4 years ago | (#31290168)

Yep... I even use that occasionally on my little site. Come to me with Windows, and you might see an ad for VMWare's server products. Come to me with a Mac and might see an ad for VMWare Fusion. It's all in reading the user agent.

Re:That's called an "contextual ad engine". (2, Funny)

megamerican (1073936) | more than 4 years ago | (#31290186)

Amazon's recommendations have become so accurate lately that it typically asks me if I'm interested in buying things I've already purchased from them.

Re:That's called an "contextual ad engine". (1)

Tekfactory (937086) | more than 4 years ago | (#31290618)

I know you got the funny mod, but its true, and needs attention.

I own a Canon DSLR, I've bought a battery charger for the camera from Amazon, and it keeps recommending me more batteries, that's fine. It recommends me new lenses, and filters, that's fine. I add some of the filters and lensea to my "Wish List" and now it wants to sell me a Nikon. That's not Ok.

I'd like it not to offer me competing/slightly different goods for goods I already own, I have Dragon Age:Origins, Mass Effect, Knights of the Old Republic I & II as well as Jade Empire all for the PC. In addition to strategy guides, Mass Effect 2 and the upcoming expansion to Dragon Age, which are reasonable, it also wants to sell me the Xbox and PS3 versions of the games I already own, as well as the collector's/game of the year editions of games I already own. That's not Ok.

I'd really like an option to be "Not Interested" in a whole series, like 24 or Lost, if I tell it I don't want Season 3, don't offer me Seasons 1, 2, 4, and 5.

Re:That's called an "contextual ad engine". (1)

mejogid (1575619) | more than 4 years ago | (#31291014)

I'm not so sure - in the case of traditional advertising you get certain groups intentionally targeting users that match a given criteria such as views of a certain type of website or TV program. Even if web advertising then adjusts based on the effectiveness of links between given groups and given adverts, that's still fundamentally driven by a manually selected connection.

In the case of the system described in the article, to match you with similar uses and then apply a degree of randomness to provide more interesting results. In this case, the link is driven by automatically calculated connections based on usage rather than targeting. This means it can be applied not only to products, but to any other site that involves multiple users with varying interests.

This seems like a more interesting alternative with more potential than traditional targeted advertising.

Please disconnect this sytem from the network. (1)

Last_Available_Usern (756093) | more than 4 years ago | (#31290102)

How long before it has enough data to recommend we should be destroyed and acts on it?

Re:Please disconnect this sytem from the network. (2, Funny)

WrongSizeGlass (838941) | more than 4 years ago | (#31290178)

How long before it has enough data to recommend we should be destroyed and acts on it?

According to my calculations, about six days. On the seventh day it will rest.

10 Goto 10 (1)

LostCluster (625375) | more than 4 years ago | (#31290110)

So, first we start out with one TV in the house and mass appeal programs. Then, as we get more and more channels, each user watched a specific channel targeted to their demographics. Then we got more specific programs from podcasts, and recommendation systems told us what we'd like before we knew it existed. The problem was, then the content makers didn't know how to sell such small audiences, so we're going to have to muck up the recommendations systems to suit them... sure, good luck with that.

Re:10 Goto 10 (1)

Michael Kristopeit (1751814) | more than 4 years ago | (#31290332)

BRUN response

Netflix ain't no dummies (1)

BadAnalogyGuy (945258) | more than 4 years ago | (#31290148)

Even small percentage increases in per-order purchases can result in huge gains across the board. Netflix, with a comparatively paltry prize amount, has bought themselves an incredibly efficient revenue generating piece of software.

I'm surprised to see that it still relies on popularity ranking as a cornerstone of the algorithm, but the other areas, especially heat diffusion and random walk are very cool and I'd love to read more about it.

you knw where this really needs to be improved? (2, Insightful)

Em Emalb (452530) | more than 4 years ago | (#31290158)

Books. I am an avid reader of sci fi and fantasy, and man, most recommendations out there just BLOW.

Re:you knw where this really needs to be improved? (1)

AndersOSU (873247) | more than 4 years ago | (#31290282)

well to be honest... that's because most sci fi and fantasy books blow.

No, really, I say this as a fan of the genres.

Re:you knw where this really needs to be improved? (0)

Anonymous Coward | more than 4 years ago | (#31290412)

That's what recommendations are supposed to filter.

Re:you knw where this really needs to be improved? (1)

AndersOSU (873247) | more than 4 years ago | (#31290598)

that's not a recommendation, or at least not dynamic recommendations, it's more like a top 10 list. And you don't need a fancy algorithm to display the same recommendations for everyone.

Recommendations are supposed to say: hey, we have a bunch of well liked books, but your likes map to this person's likes - and they also liked this book you haven't read.

In SciFi, that gets corrupted to: these books all suck, but they're all we have in the genre, so this one sold better than the others.

Re:you knw where this really needs to be improved? (1)

Em Emalb (452530) | more than 4 years ago | (#31290502)

Exactly. That's why I'm saying that. I dunno how many books I've started to read then just put down because they are just brutal.

Of course, one mans junk is another man's gem so whatever. :-)

Re:you knw where this really needs to be improved? (0, Offtopic)

carolfromoz (1552209) | more than 4 years ago | (#31290628)

Books. I am an avid reader of sci fi and fantasy, and man, most recommendations out there just BLOW.

Hey do you fancy being a personal recommendation engine for a minute? I love Neal Stephenson - who else should I check out?

I'm living in a non-english speaking country at the moment so rely on amazon for book buying, and I'm waayyyy out of touch.

Re:you knw where this really needs to be improved? (1, Offtopic)

Em Emalb (452530) | more than 4 years ago | (#31290738)

Neil Gaimon has some good stuff. I actually like most everything I've read by Jim Butcher. Most are light and easy reads, but he writes pretty well. Those are time-passers.

Re:you knw where this really needs to be improved? (1)

carolfromoz (1552209) | more than 4 years ago | (#31290990)

Thanks - I'll check 'em out.

Re:you knw where this really needs to be improved? (0)

Anonymous Coward | more than 4 years ago | (#31291840)

If you liked Snow Crash, and if you haven't read William Gibson's classic cyberpunk trilogy (Neuromancer, Count Zero, and Mona Lisa Overdrive) I would recommend them. For a quick feel for what they're like, he has a compendium of shorts called Johnny Mnemonic (the awful Keanu Reaves movie was based on the titular short, which was much better the movie made it look.)

They're sometimes a bit dated (fencing 3MB of stolen RAM chips is laughable now), but for the most part the technology is just "there"; so when he doesn't go into the details of the tech, it's a lot less jarring. What I really like about him is that he makes me see the scenes that he paints -- the Sprawl, Chiba City, I saw them as if looking through a window.

Gibson's written quite a few novels since then, but none stuck with me the way those three did.

Re:you knw where this really needs to be improved? (1)

mcgrew (92797) | more than 4 years ago | (#31290742)

I've found that most recommendations blow, period. Like movies; if the reviewers pan it, I'm almost certain to like it. The exception, it seems, is recommendations from slasdot commenters; I discovered Cory Doctorow and Terry Pratchett here.

Re:you knw where this really needs to be improved? (1)

maxume (22995) | more than 4 years ago | (#31291496)

Doctorow is awful.

For movies, the trick is not to read 'the reviews', but to pick a critic and learn his style; then, you get an idea from his review if you will like the movie or not (bonus points for the critic if their likes line up with yours).

Re:you knw where this really needs to be improved? (1)

mcgrew (92797) | more than 4 years ago | (#31315802)

Doctorow is awful.

I'd read some of his magazine articles and had the same opinion, but his books are engrossing; of course, everyone's tastes are different.

Re:you knw where this really needs to be improved? (1)

thms (1339227) | more than 4 years ago | (#31292440)

Yes, these algorithms can finally release you from having to rely on the dictatorship of the masses wrt. recommendations, or investing too much time building your own favourite recommendations-authors.

Previously you selected a smaller supgroup from the general mass, e.g. move to only reading what the NYT recommended, but you were still in a rather big herd. Now you can pretty much "build your own crowd" as you read and rate what you liked/disliked.

I wonder if Netflix is really the only company which run these deeper algorithms. Are they that computationally expensive or just really complex? Wouldn't that be a killer feature for, say, a social network - I might even join for that feature!

Still, this article talks about improving such a system, but I haven't even seen one out in the wild yet!

Tangential recommendations couldn't be any worse (0)

Anonymous Coward | more than 4 years ago | (#31290162)

Most of the time when a site suggests something for me I have absolutely no interest in what they suggest. I don't see how a tangential approach could be any less effective.

Re:Tangential recommendations couldn't be any wors (1)

Michael Kristopeit (1751814) | more than 4 years ago | (#31290360)

they are both equally bad... it's just that the people with money chasing the solution are no longer sold on the standard approach... it's quantum now... if we all go for the blonde no one gets the blonde, so recommend hair dye.

Algorithm Pacino Says: (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#31290166)

Let me introduce you to my little recommendation.

Where are the recommendations and targeted ads? (1)

Kanel (1105463) | more than 4 years ago | (#31290250)

We've been both promised personal recommendations and been threathened with personalized advertisement, yet I hardly ever see any of it.

Take Youtube I thought there was something fancy behind it but now that it displays _why_ it's recommending a clip, you can tell that it's extremely simple. Being a practictioner in machine learning and AI myself, I must confess that most industry implementations in our field is 10% very simple stuff, with 90% boring database and infrastructure code around it.

No news websites allow personalization, Google has (supposedly) only minor tweaks for the individual user. There's the recommendation system at Amazon, true, but it stands out only because it's the only one worth mentioning. (May I have overlooked some music streaming sites here?)

Compared to what we can do with search engines, the state of the art and the implementations are dismal. Is it a Really Hard Problem (TM) ? Consider the Netflix competition. Several groups worked feverishly to improve on the inhouse Netflix recommendation system and did so, by only 10%. Can we really hope for a breakthrough?

Re:Where are the recommendations and targeted ads? (1)

megamerican (1073936) | more than 4 years ago | (#31290860)

That's because they are probably lying to you.

The example I'll use is google but it applies to most companies. Whenever they come out with a press release saying that they're now collecting this or that information it is only to serve you targeted ads, yet every ad I've ever seen while logged in to Google is directly related to my search terms or the e-mail I'm currently reading.

Here is an article from 2006 [slashdot.org] that states that Google is going to listen in to your microphone and webcam to serve ads. Where are these targeted ads?

Now it's 2010 and most people know about the PA school district [yahoo.com] dialing into kids' laptops to spy on them with webcams.

There is also a less known PBS documentary from last year called Digital Nation [pbs.org] which celebrates the fact that school administrators are spying on kids with laptops and webcams.

I'd love to know what these companies are really doing with the data and the sad fact is we probably aren't suspicious enough about it.

Re:Where are the recommendations and targeted ads? (1)

Kanel (1105463) | more than 4 years ago | (#31290948)

Actually, I think a lot of companies are collecting a lot of data because people _think_ that it will be useful in the future. If the data mining gurus says that a database of consumer behaviour is worth billions, then that's something a company can list as their assets. Drives up the perceived value of the company and the stock price.

PNAS (0, Funny)

Anonymous Coward | more than 4 years ago | (#31290262)

My PNAS wants to show you something too. Careful or it'll squirt you in the eye.

Tricky Business (3, Interesting)

miasmic (669645) | more than 4 years ago | (#31290284)

Every recommendation algorithm I've seen does one or both of two things. The first being staying extremely close to things I have already expressed an interest in - never broadening my horizons.

That, or it suggests really popular things, for example with music always getting a string of well known, popular bands and artists like Radiohead or Pink Floyd suggested as bands I might like - because many people who like similar sorts of music to me like Radiohead, the algorithm thinks I would like Radiohead too - they can't seem to figure that I would already know if I liked Radiohead or not at this point. I've never found a way to tell a recommendation algorithm that Pink Floyd is OK but I want something less popular...

Re:Tricky Business (1)

Michael Kristopeit (1751814) | more than 4 years ago | (#31290400)

I've never found a way to tell a recommendation algorithm that Pink Floyd is OK but I want something less popular...

don't buy anything. the first site that correctly recommends something you haven't heard of, but also really like, but it. that is how the invisible hand works.

Re:Tricky Business (1)

Michael Kristopeit (1751814) | more than 4 years ago | (#31290418)

pffffft. BUY it.

Re:Tricky Business (1)

mdm42 (244204) | more than 4 years ago | (#31295742)

Yup. Did that. Only problem was that the band that caught my attention is on a label that has never been heard of by stores where I live, so the only realistic way to get hold of their music was... BitTorrent. S'ok - the band puts all their releases on the 'net anyway, generally several weeks before the official release. As a direct consequence, I now own CD copies of every album they've ever done. (Well except the latest one which hasn't been officially released yet.)

The invisible recommendation hand that worked for me was last.fm...

Re:Tricky Business (1)

Kanel (1105463) | more than 4 years ago | (#31290422)

Most algorithms in machine learning and data mining are either very simple or complicated but very generic.

In your case the recommendations are based on a similarity metric not on the music itself but on those who like it. Really popular bands are useless for characterising a music lover since the group of people who like pink floyd will be so diverse. Because of that, it won't be able to map from pink floyd to users and back to bands similar to pink floyd :-(

Re:Tricky Business (1)

miasmic (669645) | more than 4 years ago | (#31291408)

For sure, it's a crappy way to do it.

The new ideas the article talks about with subsets are a great idea - if you could identify groups of users who might not have heard Pink Floyd, like teens or people who died before 1970, you could recommend it based on tastes among their peers just to them. Though I feel using such metrics is risky - leaving little room for non-conformism, or resting in peace for that matter.

Trencher and the Police are the best test cases (0)

Anonymous Coward | more than 4 years ago | (#31290462)

If I'm searching for digital issues of Kieth Giffen's Trencher comic, I am not interested in targeted ads for earth digging equipment. If I am searching for The Police, the band, I am not interested in law-enforcement training diplomas. These are good test cases to see if targeted advertising is any good or not - all of them I've ever seen fail these cases.

Re:Trencher and the Police are the best test cases (0)

Anonymous Coward | more than 4 years ago | (#31291076)

If every instance of targeted advertising fails your test cases, then how can they possibly be good test cases? They're useless as a basis of comparison, if the result is the same for everyone. I mean, I get that homonyms are confusing without context, but until a system comes along that is capable of accounting for context, there's little reason to point out that no system currently does.

Re:Trencher and the Police are the best test cases (1)

miasmic (669645) | more than 4 years ago | (#31291478)

The first example is good, but if you are searching for "The Police", it's unlikely any algorithm, or human observer would think you were searching for the band. If you searched for "The Police Band" (without quotes obviously) then I would say fair enough.

Otherwise it would be like searching for "big black cock" and being surprised that the results and ads were not about poultry.

Re:Tricky Business (1)

Paul Lamere (21149) | more than 4 years ago | (#31300180)

Every recommendation algorithm I've seen does one or both of two things. The first being staying extremely close to things I have already expressed an interest in - never broadening my horizons.

clearly you haven't used the wreckommender: http://wreckommender.com/ [wreckommender.com]

Re:Tricky Business (1)

miasmic (669645) | more than 3 years ago | (#31334560)

I just gave it a go with every wish for it to be good, but it didn't seem to recognise my test artists (some mildly obscure drum & bass). I then tried "Metallica" to cover the opposite end of the spectrum, but got a 500 internal server error - an easter egg perhaps? Finally I tried US noise metal band "Lightning Bolt" but it returned an Enya song as a reccomendation. That's when I decided it wasn't built for my tastes! Nice design though.

Prior art (0)

Anonymous Coward | more than 4 years ago | (#31290334)

I'm sure I posted this very idea to /. years ago, probably more than once. I'm also sure I'm not the only one who recognizes the utility of joggling the output of a recommendation system.

Not new (1)

wjousts (1529427) | more than 4 years ago | (#31290368)

This isn't really that new at all. I've seen several other groups doing something with diversity vs similarity in recommender systems.

How interesting (0)

Anonymous Coward | more than 4 years ago | (#31290420)

I happen to like fine wine, vintage automobiles, afternoon tea and Goatse. I am looking forward to meeting others with similar interests.

Re:How interesting (2, Funny)

MillionthMonkey (240664) | more than 4 years ago | (#31290496)

Have you ever been in a Turkish prison?

Group-sink. (1)

Ostracus (1354233) | more than 4 years ago | (#31290468)

"However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer.""

Slashdot: "I see you've subscribed to certain opinions. Here are some more recommendations."

Re:Group-sink. (1)

Kanel (1105463) | more than 4 years ago | (#31290672)

It's been a while since the viewpoints in Slashdot articles have challenged my opinions. A case of the chicken and egg question? :-o

To Find Something New (0)

Anonymous Coward | more than 4 years ago | (#31290474)

take
heroin [youtube.com] .

Losers.

Yours In Vilnius,
Kilgore T.

Linking Tangential Attributes (1)

decipher_saint (72686) | more than 4 years ago | (#31290528)

I understand the problem; the direct connection criteria between two different things might be completely indecipherable or insurmountably complex and subtle (let alone indirect relationships i.e. six degrees of Kevin Bacon). That means whatever you build has to account for trends to narrow the band of complexity which leads to the same old problem of only suggesting status quo.

A tool that can only suggest "obvious" or "random" things leads to undesirable results and at best can only fractionally provide you with "success".

I still think it's a cool project with a lot of opportunity for discovery but I just can't get past the idea that you either tell people what they want (advertising) or let people discover things on their own (interconnection).

We're damned if it asks us (1)

Kanel (1105463) | more than 4 years ago | (#31290788)

A system like this can work in two ways. Either the similarity measure computes a "distance" between the music by analyzing the sound and metadata, or it maps a band to a group of people that like it and then maps from this group back to other bands the group likes.

If the system is employing the latter, we have a problem. If we select only popular bands or pick bands randomly, there's no hidden wisdom of the crowd for the algorithm to extract. We can't blame the software, only ourselves

Re:Linking Tangential Attributes (1)

plover (150551) | more than 4 years ago | (#31292072)

I just can't get past the idea that you either tell people what they want (advertising) or let people discover things on their own (interconnection).

The best advertising is advertising misinterpreted as interconnection.

Remember the kerfluffle a year or two ago with the web site twittering your friends with your purchases, or some kind of plug in that monitored your purchases? I don't remember exactly which site, nor do I care because the details aren't important, but that was one step away from the goal here. Imagine if all your friends posted all their purchases all the time, and you got tweeted with all those purchases, and all those tweets flowed through a giant recommendation algorithm. Now imagine if I run that algorithm, and Nike pays me for advertising. I'll make sure that when your friend Joe buys Nike shoes the tweet will get through to you and all his friends; but when Mary buys Adidas and Carol buys Sketchers, nobody will see them.

To you, it's interconnection: your friends bought Nike shoes and apparently nothing else, so maybe you'll buy Nike shoes. To Nike, it's advertising. To me, it's profit.

Of course the drawback to these systems happens when they're discovered and revealed for what they truly are. History says they won't last beyond that point. But they're highly profitable right up until they are outed.

I recommend (1)

rinoid (451982) | more than 4 years ago | (#31290586)

I recommend getting out into the Big Blue Room and doing something real, tangible, and unique!

Re:I recommend (0)

Anonymous Coward | more than 4 years ago | (#31290992)

Wow, thanks. Your suggestion of "going outside" is totally new, and so relevant to recommendation algorithms! I'm sure nobody else would ever have thought of it. You are to be praised.

There's a recommendation algorithm in my town (1)

beefnog (718146) | more than 4 years ago | (#31290734)

...he always wants to show you something new, and it's always in the back of his van along with the puppies and candy.

Re:There's a recommendation algorithm in my town (1)

snspdaarf (1314399) | more than 4 years ago | (#31291362)

"Hey, Mister, there's no candy back here, and I think your puppy peed on the mattress."

IBM algorithm (1)

belthize (990217) | more than 4 years ago | (#31290774)

      They should use IBM's new algorithm, it's faster than the old one.

"RMSE" as a yardstick is one reason for this (1)

Sanity (1431) | more than 4 years ago | (#31290776)

One reason this kind of problem occurs is that many collaborative filtering algorithms are measured based on "root mean squared error", basically the square root of the mean of the differences between what was predicted and what the user actually did.

The problem with this metric? It doesn't account for a variety of important things, one of which is that most users value diversity. Another is that in most recommendation systems, what is important is the relative relevance of recommendations to each-other, whereas RMSE is an absolute measure of effectiveness. And a really tricky one is that the recommendation algorithm itself can impact user behavior. For example, the user may raise their standards if the algorithm does a better job.

The unfortunate answer is that the only rock-solid way to measure the effectiveness of recommendation algorithms is to test them with real users, perhaps splitting the user population between different algoritms, and seeing which does best.

I'm pretty familiar with this issue as my day job [sensearray.com] is building a behavioral ad targeting engine. We learned a long time ago that while RMSE has its uses, there is often limited correlation between an algorithm's ability to predict user behavior retrospectively (which ads they will click on and what products they will buy), and how much additional revenue the algorithm will generate in practice.

Our solution is to use RMSE as a first-blush indication of how good an algorithm is. Secondly, we take the top, say, 10% of ads with the best predictions, and see what the actual click or conversion rate is within this 10%. This requires a higher volume of data, but yields results that are closer to what we find in reality. Lastly, the algorithm then has to prove itself in the wild on a small subset of traffic. Only then can we really know if any algorithm is an improvement on any other.

Perhaps it can recommend... (1)

Tokerat (150341) | more than 4 years ago | (#31292472)

...some other research groups with names that make me giggle like an idiot.

w4nderlust (0)

Anonymous Coward | more than 4 years ago | (#31293412)

We working on quite the same topic approaching from a different point of view and trying to exploit the concept of serendipity to try to obtain similar resuts.

Our first publication on the topic:
http://www.computer.org/portal/web/csdl/doi/10.1109/HIS.2008.25

Piero Molino
http://www.di.uniba.it/~swap/index.php?n=Membri.Molino

Nothing new here! (1)

Tony Isaac (1301187) | more than 4 years ago | (#31294836)

Recommendation engines, such as the ones used by Netflix and Amazon.com, already recommend really random choices. Except in the simplest cases (you bought a nail gun, you might want some nails), current recommendation engines stink at figuring out what I want. Trouble is, the recommendations are interesting so few times that I don't even look at them any more. I'll bet these researchers actually used to work for Amazon.com creating their recommendation engine, but got fired. Not knowing what else to do, they wrote a paper to describe what they did!

Where is better search? (0)

Anonymous Coward | more than 4 years ago | (#31295198)

Screw recommendations, what I want is a better search and organizations. When you look for example at Amazaon they still are to stupid to put all the books in a comic book series together, you have to manually search for each of them or hope that the recommendations picked one of them up. Why don't they just show you a list of everything related to a series? How about a search for all comics in which Spider-Man appeared or movies where two actors played in together? Movies where an specific actor had more then 10min screen time and all that kind of detailed stuff. Why am I limited to a stupid text search over an items description, why can't I search the actual content? On top of that Amazon doesn't even seem to have their advertisment linked with their sales, they constantly recommend stuff hat I already bought from them.

Looking through torrents, release lists, Wikipedia and other third party information is much more informative then what the commercial companies manage to provide and without that extra help it would often be near impossible to find some items and actually buy them.

The Acronym... (1)

Rambone.ftw (1758978) | more than 4 years ago | (#31352296)

As someone who neither follows, nor particularly cares about this topic, did anyone have a lul or two when they noticed the acronym used in the first line of the quote? "PNAS"
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>