×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

"Long Tail Effect" Doesn't Work As Advertised, Say Wharton Researchers

timothy posted more than 4 years ago | from the econtalk-has-good-anderson-interview dept.

Businesses 82

Death Metal writes "In a working paper titled, 'Is Tom Cruise Threatened? Using Netflix Prize Data to Examine the Long Tail of Electronic Commerce,' Wharton Operations and Information Management professor Serguei Netessine and doctoral student Tom F. Tan pull information from the movie rental company Netflix to explore consumer demand for smash hits and lesser-known films. Netflix made its data available as part of a $1 million prize competition to encourage the development of new ways that will improve its ability to introduce customers to lesser-known titles they might find appealing." In short, the researchers say that the Long Tail effect described by Chris Anderson is much less important in the real world than popularly held. Says the article: "The key difference between the opinion of [Anderson's] book and the study by Wharton researchers is how they define 'hits' and 'niches.' In the book, Anderson focuses on the definition of hits in absolute terms such as the top 10 or top 1,000 products, while Netessine and Tan argue that, to take growing product variety into account, one has to define popularity in relative terms, such as the top 1% or top 10% of products, to properly assess the presence or absence of the Long Tail."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

82 comments

Missing the point (4, Insightful)

BadAnalogyGuy (945258) | more than 4 years ago | (#29481779)

The long tail doesn't threaten those at the top any more than it isolates those at the bottom. It only describes the shape of the market which necessarily has only a few specific market products which are used by the majority and the rest of the products with very few customers in the "long tail". It's a market definition, not a competition definition.

You can cut the tail off of a gecko at any point, but it doesn't mean that somehow the tail can exist without a fat end and a thin end. Since the tail is simply the appendage attached to the abdomen, wherever it is attached defines its fat end, and where it ends is the thin end. Even if you cut the tail off completely, all that you've done is stimulated the tail regrowth reflex.

Re:Missing the point (5, Informative)

JimboFBX (1097277) | more than 4 years ago | (#29481813)

The working title of the paper is misleading, since there is no mention of "threat", an "effect", or anything of that sort. In fact, all I got out of it was that they were just debating something rather trivial and inconsequential - the definition of a "hit" in a statistical model and how using "top 1000" or so is improper based on NetFlix data.

To be honest, this isn't really "news" worthy of a front page listing.

Re:Missing the point (4, Interesting)

mabhatter654 (561290) | more than 4 years ago | (#29482917)

But few retail stores stock more that 50 or 100 current titles, so I think the original idea is quite good. Movie theaters show only 12 or so films at a time, even in big cities. Opening a Brick and Mortar store or theater up to even 1% of "hits" say music/movies that actually made a popularity chart in the last 50 years would be an impressive achievement. Even blockbuster movies like Star Wars or Indiana Jones become "unpurchasable" in a relatively short amount of time.... They've already hit "bargin bin" status in most retail outlets.

Having a surefire way to get at that back catalog would be highly important. The real key is getting business to focus on marketing in "long tail" manner. Something like Netflix is interesting because they are a business that really pays no penalty for keeping extra DVDs in the warehouse.... but how do they get people interested in WATCHING them. I find Emusic to be a similar thing in that area, but again, the hardest part is matching up MORE stuff I'm interested in rather than what publishers are currently marketing. I think Disney has the best handle on it because they republish back catalog in a big way every 10 years or so... making it "new" again to a new group of people. How do you do that for general things like "Bing Crosby" movies or "Rogers and Hammerstein" musicals? Heck, even getting recent Anime published in English in a reasonable time is difficult, or finding material from Electronica/J-Pop scenes due to publishers only wanting to publish "top 10" material.. when the majority of people don't BUY that stuff.. but they all would buy a different part of the top 500 or so songs.

50 to 100? (0)

Anonymous Coward | more than 4 years ago | (#29486031)

Where do you get that 50 to 100 titles data from? I can line 50 DVD cases (spine out) on my tiny 3 foot wide desk. Surely a STORE has more shelf space than THAT!

Re:50 to 100? (0)

Anonymous Coward | more than 4 years ago | (#29487007)

So? What if the number is 1000 or 10,000? It's a fixed number, but the number of movies grows every year. So every year your fixed number of titles becomes a smaller and smaller and smaller percentage of the "universe of all movies".

Re:Missing the point (1)

MacWiz (665750) | more than 4 years ago | (#29488241)

Even blockbuster movies like Star Wars or Indiana Jones become "unpurchasable" in a relatively short amount of time...

This is artificial scarcity. They are only unpurchasable because the long tail isn't profitable enough for the big studios, who will stop making copies as soon as they can't sell a boatload of them anymore.

Disney has the best handle on it because they republish back catalog in a big way every 10 years or so... making it "new" again to a new group of people.

Disney is the worst manipulator of artificial scarcity. The only reason they republish "in a big way," is because they take their stuff off the market in an equally big way. There is a new crop of five-year-olds every year. You don't have to make Snow White or Pinocchio "new" again to please them.

The 10-year cycle (or whatever the actual length is -- I think it's longer) is designed to make sure new parents don't have a copy of whatever Disney is re-releasing when they start an advertising campaign to make their kids ask for it.

How do you do that for general things like "Bing Crosby" movies or "Rogers and Hammerstein" musicals?

The same kind of advertising budget might be able to do it (although as a kid, being forced to watching Rogers and Hammerstein musicals when they were new seemed like punishment). It would seem that re-releasing and injecting an advertising campaign brings the work back to the front of the line (or at least attempts to). That means it's no longer part of the long tail, which should (theoretically) work without it.

Re:Missing the point (1)

OrangeCatholic (1495411) | more than 4 years ago | (#29489021)

Having a surefire way to get at that back catalog would be highly important. The real key is getting business to focus on marketing in "long tail" manner. Something like Netflix is interesting because they are a business that really pays no penalty for keeping extra DVDs in the warehouse.... but how do they get people interested in WATCHING them.

You *don't.*

Star Wars and Indiana Jones are rightly bargain-bin items. That's because everyone has seen them in every imaginable permutation (in theaters, on re-release, on TV, on cable, on VHS, etc.)

The "long tail" has nothing to do with age. It has to do with obscurity. Titles like Bigger Stronger Faster [imdb.com] (a recent documentary about steroids) or God's Sandbox [imdb.com] (an Israeli film about tribal female circumcision) are better examples of the long tail. Or quality Anime imports. Or Adult Swim's output. Or foreign-language films.

Netflix has great ways of promoting obscure titles. Which is why I've seen the two that I mentioned.

Re:Missing the point (5, Interesting)

Anonymous Coward | more than 4 years ago | (#29481821)

It only describes the shape of the market

That's precisely the point. If the shape is such that a top movie gets only 1% of the market, top movies won't make enough profit to justify hiring Tom Cruise and it's a problem for him.

Re:Missing the point (1)

anagama (611277) | more than 4 years ago | (#29483147)

If the shape is such that a top movie gets only 1% of the market, top movies won't make enough profit to justify hiring Tom Cruise and it's a problem for him.

Tom Cruise is his own reason to justify not hiring Tom Cruise.

Re:Missing the point (1)

swilly (24960) | more than 4 years ago | (#29484895)

To be fair, he was awesome in Tropic Thunder. The highlight of the movie really. And his cameo in Austin Powers Goldmember was pretty funny too.

Re:Missing the point (5, Insightful)

mangu (126918) | more than 4 years ago | (#29481939)

The long tail doesn't threaten those at the top any more than it isolates those at the bottom. It only describes the shape of the market

But the shape of the market is exactly the point. In a competitive market profit margins are very thin and a relatively small difference may mean life or death to a company. In the entertainment industry we often see an effect where the biggest productions often seem to struggle to break even, while relatively small investments may bring huge profits.

Defining a "hit" as one of the top ten or top 1000 or any absolute number is stupid. It reminds me of a political joke in the Soviet Union, where the result of a race between two athletes, a Russian and an American, was reported in the press as "the Russian came in second while the American was next to the last". In electrical and electronics engineering threshold values are often defined as the point where the power is one half of the maximum, the so-called "-3 dB" points.

Re:Missing the point (1)

Loomismeister (1589505) | more than 4 years ago | (#29481987)

Biggest productions from my anecdotal experience seem to make a ton more money than the movie was made with.

Re:Missing the point (1)

digitig (1056110) | more than 4 years ago | (#29482445)

THIS (1)

wfolta (603698) | more than 4 years ago | (#29483017)

Parent has it right: What you read about is that the film cost X in production costs and raked in Y in box office receipts, where Y > X. But from what I understand, no film has ever officially made a profit: it gets eaten up by all kinds of Byzantine expenses. The music industry is slightly less corrupt -- really big bands actually get a profit -- but it's incredible to read about how little a smash hit will make for the actual musicians who created and performed it.

Of course, that has nothing to do with the discussion at hand. And as others have said, that debate boils down to traditional (limited shelf space, therefore absolute numbers) versus online (unlimited shelf space, therefore percentages) sales models.

Re:THIS (1)

onepoint (301486) | more than 4 years ago | (#29483405)

Film's and music make very good profit's, but it's called creative accounting. It can be very attractive to invest in movies where a know amount of tax losses will occur over a set period of time. The music industry has a different set of rules, don't forget that a band gets an advance to produce the music ( venture capital investment ), the returns on that investment must be paid before the band gets a dime ( I seem to recall the band called Boston, and how they got screwed ).

I have invested in movie deals, some have dumped me a great load of money, some broke even, and some have gone ka-put.

For those that invest in music, I keep repeating these words, I the investor make money on part of the song and part of the album sales, you the artist make the most money going on tour and merchandise.

Re:Missing the point (4, Insightful)

coaxial (28297) | more than 4 years ago | (#29482195)

Defining a "hit" as one of the top ten or top 1000 or any absolute number is stupid. It reminds me of a political joke in the Soviet Union, where the result of a race between two athletes, a Russian and an American, was reported in the press as "the Russian came in second while the American was next to the last". In electrical and electronics engineering threshold values are often defined as the point where the power is one half of the maximum, the so-called "-3 dB" points.

There are 17,770 movies in the Netflix Prize training set. 1000 movies account for 5.63% of movies for the entire dataset. This 5% account for 63% of all rentals. If you use your threshold of "half of the maximum," then you have the top 100 movies. More to the point, your threshold definition using decibels is predicated on the data being normally distributed, and this data conforms to a power distribution, most likely Zip-Ian.

Getting back to your point of defining a "hit" based on profitability, that too is poor way of defining it. It's much easier for a very cheap film to make multiples of it's budget in revenue, but still no one sees it. Number of viewers has always been the traditional way of defining a hit. Revenue is just proxy for that.

Re:Missing the point (1)

drinkypoo (153816) | more than 4 years ago | (#29482563)

Getting back to your point of defining a "hit" based on profitability, that too is poor way of defining it. It's much easier for a very cheap film to make multiples of it's budget in revenue

Its. Its budget. Anyway, forget profitability, Hollywood is now interested in profit. That's why they're making more direct-to-DVD movies (especially chick flicks, which lose nothing on the small screen) and the quality of effects in blockbusters like Wolverine or Watchmen is in the toilet. Trying to figure out the most popular movies of all time is vaguely interesting, but since Netflix assuredly already rents them, not actually very useful.

Re:Missing the point (4, Interesting)

khchung (462899) | more than 4 years ago | (#29482231)

Defining a "hit" as one of the top ten or top 1000 or any absolute number is stupid.

While it sounds stupid, using a top xx% is, in a way, validating the idea of long tail.

Why? Because before Amazon, when book stores are still only brick and mortar, there is only so much physical space to hold top 1000 or however many number of books. Note that this number is fixed, it won't grow because more kinds of books are published.

So having an absolute number of top 100 or top 1000 simply corresponds to the physical constraint that most bookstore can only put so many books on the shelf.

The advantage that Amazon has over physical book stores is that it can hold practically unlimited number of books. So only now, without the physical constrain, we can practically use top 10% instead. This, in fact, proved that there are many more profitable books outside top 1000 (or however many), and that physical bookstores are missing out many sales due to it.

Re:Missing the point (4, Interesting)

ukyoCE (106879) | more than 4 years ago | (#29482547)

This, in fact, proved that there are many more profitable books outside top 1000 (or however many), and that physical bookstores are missing out many sales due to it.

YES.

This is exactly why I've stopped using brick and mortar retailers almost entirely. They carry such a limited selection that it's often a wasted trip.

This goes for video rental stores once they consolidated (in my area) to a chain of "new release-only" stores. I switched to Netflix and have never been back, and have converted many friends to Netflix too.

Music stores, which in my area have never carried anything but the most popular overpriced crap. Now I buy from Amazon or direct from musicians' websites.

Groceries are one of the few markets left worth using brick and mortar stores for. Anything else is just a showroom for cheaper online stores, at this point.

Re:Missing the point (1)

onepoint (301486) | more than 4 years ago | (#29483359)

You my friend need to brows a used book store or a flea market.
Sometimes it's not the need that gets filled, but the desire of
fulfillment from finding the just perfect something that you want.

Re:Missing the point (1)

CptNerd (455084) | more than 4 years ago | (#29486411)

I have a used book store on Amazon, you insensitive clod!
Of course, I'm not allowed to post the link here, but still...

Re:Missing the point (1)

droptone (798379) | more than 4 years ago | (#29485709)

The advantage that Amazon has over physical book stores is that it can hold practically unlimited number of books. So only now, without the physical constrain, we can practically use top 10% instead.

I am pretty sure a minor but important point made in The Long Tail was that not only did you have to combine massive inventory with a good recommendation system to have a widely successful business. The recommendation system is key since people will tend to regress toward the hits without outside influence.

Re:Missing the point (3, Interesting)

Znork (31774) | more than 4 years ago | (#29482793)

In the entertainment industry we often see an effect where the biggest productions often seem to struggle to break even

It's called Hollywood accounting [wikipedia.org] and has very little to do with actual 'profits'. Small productions might also seem 'unprofitable', once they learn to have their Cayman Island subsidiary charge the project $500k for the producers porta-potty rental. Actual profits are simply funnelled to the desired destination by way of semi-internal charges for rents, distribution, marketing, consulting, etc, etc. As long as your costs are to yourself, nothing really has to make a profit...

And, of course, the monopoly rights industries are nothing like a competitive free market.

Either way I think it's premature to analyse the long tail to any extent. Copyright in itself drives the distortion of the market by encouraging excessive marketing and inordinate market control, creating an economic situation where it's better for the major players that people buy fewer products for more, than more products for less, and they do all they can, with a fair level of success, to push that market shape.

Re:Missing the point (1)

Shark (78448) | more than 4 years ago | (#29483529)

Very interesting point there... I wonder what use they might have for such low profit figures. Could it be used to justify draconian drm and lobbying against fair use, launching a crusade against p2p technologies, etc? Oh, I imagine there's a very nice tax incentive as well.

Re:Missing the point (1)

gobbo (567674) | more than 4 years ago | (#29484035)

I wonder what use they might have for such low profit figures

Every film production has three budgets: the studio budget (for investors), the taxman budget, and the real budget (producer eyes-only). There's a combination of outrageous above-the-line costs (producer and main actor fees) and, as the GP mentioned, dubious inflated line items, that allow these production budgets to play various stakeholders for chumps.

Re:Missing the point (1)

Sj0 (472011) | more than 4 years ago | (#29491885)

There was a story recently about Lord of the Rings, how the estate of J. R. R. Tolkien was to receive no royalties because of Hollywood accounting -- a movie that grossed 6 billion dollars worldwide had been put down on paper to show it hadn't turned a profit. Peter Jackson sued the studios as well, claiming their accounting methods understated revenues, and settled for an undisclosed sum.

Re:Missing the point (1)

Dog-Cow (21281) | more than 4 years ago | (#29493111)

It's a shame the estate was involved at all. I sincerely hope every member of the estate dies penniless and painfully. Hopefully over a couple of decades.

Re:Missing the point (1)

Sj0 (472011) | more than 4 years ago | (#29502299)

I agree, from a policy point of view. I don't think it's right that copyrights for works by long dead artists are still in effect. On the other hand, it IS their right under the law, and I'd rather hate on the copyright industry for lobbying for the laws and the congress for passing them than hating on the estate for simply following the law to their benefit.

Re:Missing the point (1)

tehcyder (746570) | more than 4 years ago | (#29492039)

Defining a "hit" as one of the top ten or top 1000 or any absolute number is stupid. It reminds me of a political joke in the Soviet Union, where the result of a race between two athletes, a Russian and an American, was reported in the press as "the Russian came in second while the American was next to the last".

In Soviet Russia jokes laugh at you.

you're missing the point (1)

speedtux (1307149) | more than 4 years ago | (#29482537)

All the recent brouhaha about the "long tail" doesn't merely relate to the shape of the distribution (which has been known for a long time); it's about inferences people draw from that shape.

A lot of the inferences I have seen are unwarranted, and papers like this come to the same conclusion.

Re:Missing the point (1)

Geoffrey.landis (926948) | more than 4 years ago | (#29482665)

Yes, it looks to me as if they are arguing about the relative shape of the tail, not about whether the long tail exists.

The existence of a long tail doesn't really negate the presence of a fat head as well.

Re:Missing the point (1)

Blink Tag (944716) | more than 4 years ago | (#29482985)

It only describes the shape of the market ... It's a market definition, not a competition definition.

Not wholly accurate. The "long tail" phenomenon as used in this context describes the belief that the revenue gained from the long tail exceeds that gained from the top selling items. The idea encourages production and sales of items that appeal to niche markets.

This idea has not yet been born out by the marketplace. While it's true that those who frequent Slashdot (and let's be honest, most here are probably close to being outliers in any normal distribution) may enjoy the less popular media the populates the long tail, most of the media sold in the U.S. is still sold by Walmart, which keeps primarily the most recent stuff in stock.

Just a couple of days ago I listened to a presentation by researchers from Emory who came to the same conclusion with music album sales. In order for the long tail to work, prices have to be low enough that sampling behavior is enabled--but there is still a significant herd mentality when purchasing music. (So it's not much of a stretch to extrapolate this to video, to align w/ the article.) A quick peek at the seasonal distribution of sales explains this: for album sales, the Christmas season sells significantly more than at the rest of the year, suggesting album sales are mostly gifts, where purchases are likely to be risk-averse. (So, no long tail effect there.) The same research has not been done at the _track_ level, where sampling is more likely, but it the very un-bundling of tracks from albums that is leading to revenue declines in the music industry.

Other research suggests that Netflix would be *more* likely to display the long tail effect than other media outlets. Its very nature is sampling (rentals at very low cost), so it's a good place to look for the long tail. Decision psychologists refer to the "want self" and the "should self", the competing faces of our internal dialogue. A service like Netflix is more likely to trigger the "should self" because gratification (watching a DVD) is delayed from the initial decision, leading to a higher likelihood of "art film"-style selections than is found at, say, a $1-night DVD vending machine. Thus, the findings of these researchers showing no long-tail effect from Netflix data are additionally convincing.

Re:Missing the point (1)

OrangeCatholic (1495411) | more than 4 years ago | (#29489081)

>The "long tail" phenomenon as used in this context describes the belief that the revenue gained from the long tail exceeds that gained from the top selling items.

That's idiotic. The purpose of the long tail is to ensure the survivability of the business versus competition. The reason why the tail is "long" (and thin) is because it doesn't generate a lot of interest or revenue.

What the tail does, is to provide depth to an otherwise-shallow business. 7-11 convenience stores are an example of a business with absolutely no tail. They do make money. But it's all cups of coffee and bags of chips - the stores themselves are unpleasant, the labor mind-numbing, and nobody goes in there and drops $100 for any reason.

The best line in the article is: "Nobody in the business world is confused about this, thankfully," Anderson adds.

And the stupidest is: For any company marketing a physical product, such as Netflix's DVDs or Amazon's books, managers must weigh the costs of stocking an item against the likelihood that it will generate revenue. "If you stock a lot of products that nobody consumes, then you have a problem," says Netessine.

Hahaha. Really? Products must be chosen wisely to generate sales? These Wharton guys are little baby children. I hope UPenn pays them $4 yearly salary to write this crap and they can go spend it at the tail-less convenience store where four bucks is considered a big sale.

Re:Missing the point (1)

Blink Tag (944716) | more than 4 years ago | (#29490885)

>The "long tail" phenomenon as used in this context describes the belief that the revenue gained from the long tail exceeds that gained from the top selling items.

That's idiotic. The purpose of the long tail is to ensure the survivability of the business versus competition. The reason why the tail is "long" (and thin) is because it doesn't generate a lot of interest or revenue.

I didn't say I agreed with the claim. It's an idea that (as you point out) is not intended for to brick-and-mortars; the long tail is given as a reason why web business are superior to physical stores.

The term was coined and perpetuated by Chris Anderson, author of The Long Tail and, more recently, Free.

Increasingly, long-tail claims are being disproved by researchers.

This is completely moronic (3, Insightful)

SevenSpirits (1266138) | more than 4 years ago | (#29481799)

If you add an insignificant product to the end of the tail, it obviously increases the proportion of market share of the first X% of products. That's simple math!* If your model of the "long tail" completely fails in the simple case of adding a once-purchased product, maybe your model sucks and Chris Anderson's model was more useful. * Yeah, there's a small requirement of proportional market share of the Xth percentile product vs the insignificant one, but no need to nitpick that.

Re:This is completely moronic (3, Interesting)

OrangeCatholic (1495411) | more than 4 years ago | (#29481881)

>If you add an insignificant product to the end of the tail,
>it obviously increases the proportion of market share of the first X% of products.

Yeah, but add a product because it is important by itself. Consider adding Dewar's to a store that has no blended scotch whiskey.

Will Dewar's increase niche sales? No. Will it increase single-malt sales? No. However, with a gaping hole in the scotch section, can you let Dewars die on its own?

Of course not. You must have it. Now that you have Dewar's represented, should you also add Dickface Brand for half the price? That's really the trick of it.

Re:This is completely moronic (4, Funny)

Dogtanian (588974) | more than 4 years ago | (#29482161)

Of course not. You must have it. Now that you have Dewar's represented, should you also add Dickface Brand for half the price?

You have *no* idea what you're talking about.

Everyone knows that Dickface Brand is a bourbon, not a scotch.

Re:This is completely moronic (1, Insightful)

Anonymous Coward | more than 4 years ago | (#29482049)

If you add an insignificant product to the end of the tail, it obviously increases the proportion of market share of the first X% of products. That's simple math!* If your model of the "long tail" completely fails in the simple case of adding a once-purchased product, maybe your model sucks and Chris Anderson's model was more useful.

But that DOES matter if you are buying into CA's conclusion that the "Long Tail" is important from a business perspective. Nobody I've read disputes that the total sales of more obscure DVDs (for example) may be quite significant in aggregate compared to the sales of blockbuster titles. However in order to apply a "Long Tail" business model you are going to have a truly vast quantity of inventory that will mostly move very, very slowly (maybe one or two copies per year of each SKU). You will have much greater warehousing and finance costs between the date you purchase any particular item and when it gets sold.

In order to make a successful, profitable business it is important that the "Long Tail" not only exist but be sufficiently "heavy" to offset these increased costs. Talking about the relative market share of the top X% and the rest is relevant to what sort of business model you would want to choose. Talking about the "top 10" vs the rest is irrelevant if 10 is not a realistic number in that marketplace.

Re:This is completely moronic (2, Insightful)

Anonymous Coward | more than 4 years ago | (#29482175)

Please don't use DVDs as an example of long tail economics. Having a physical copy of every item in the tail will in most cases completely negate any kind of profit. The whole point is that with virtual products (or print on demand) you can have much, much higher storage density with limited cost. Only then can you afford to only sell two items a year.

Re:This is completely moronic (1)

DarkOx (621550) | more than 4 years ago | (#29482313)

True, but if you are making product available that is not generally available elsewhere you can charge higher margins. Consider record stores for example usually carry a much wider catalog than say a Best Buy, Target, or Boarders Books. They usually charge $19 for anything that is not this weeks mega hit as well. If you could find it at Best Buy it would probably be $16 but you can't find it there. You pay extra because the Record store made it available.

The Record store has two problems, first they have make you aware of the product (mostly where they fail) and second they have find a price point consumers will pay over the norm that gives them enough margin to cover the higher costs and still make enough sales to justify the activity in the first place.

So the long tail can be made heavier to some degree if the competition of the niche market you are entering is light enough and the elasticity of the demand is low enough..

Re:This is completely moronic (1)

OrangeCatholic (1495411) | more than 4 years ago | (#29489125)

That's exactly how it works. You pay a slight premium for selection. Selection=knowledge. You are always selling knowledge.

The long tail is unpleasant and difficult to maintain, but in business, the effort put into maintaining an appearance is the main source of competition. The business that is working more diligently should win.

Of course, whether customers are savvy enough to recognize the winner is always a gamble. Sometimes pure drivel wins. Sometimes, businesses deliberately dumb themselves down, to make their jobs easier (!) even though they make less money.

I've seen it done. Everyone is searching for his personal work/profit ratio. A long tail maximizes profit. By having every single movie, Netflix will get every single eyeball. Fact: There will be no eyeballs left for anyone else. Netflix wins. But the long tail does increase labor costs, and at some point, Netflix may say, We're big enough already.

Heavy tail distributions are dangerous beasts (5, Interesting)

Anonymous Coward | more than 4 years ago | (#29481923)

OK, I am not a mathematician, but this paper makes me deeply skeptical.

If the input data is indeed heavy tail (non-existing higher moments) or quasi-heavy tail (existing, but extremely large higher moments) how on earth they can use variance, R^2 and other measures? They may not even exist! And if the input is quasi-heavy tail, then of course they exist, but the convergence time could be arbitrarily long!

I had the unpleasure to work with quasi-heavy-tailed data, and it is really enlightening. You watch the evolution of some metric (e.g.: avg) as the function of incoming data, and you see of course convergence. At least for a while. And then in sudden an extreme outlier comes in, and the avg takes a huge jump! Now if your input is heavy tailed enough, you can be never sure that your measure finally came to rest (converged), or the next jump is just over the corner!

I hope a more educated person clarifies this, I am just an engineer.

Re:Heavy tail distributions are dangerous beasts (2, Insightful)

Ambitwistor (1041236) | more than 4 years ago | (#29482671)

Generally they're talking about Pareto or power-law distributions, which aren't quite as degenerate as, say, the Cauchy distribution with no higher-order moments. You're right that they should check for the existence of the moments they care about; I presume they do, but I don't know.

Re:Heavy tail distributions are dangerous beasts (1)

NonSequor (230139) | more than 4 years ago | (#29482839)

The thing is that when you're just looking at data points as they come in, you can't tell if a distribution is well-behaved. It's possible for the data to look like it's coming from a well-behaved distribution, but then you run into an outlier that's way too far out.

Those power-law distributions have cut-offs for the shape parameter for the higher order moments to exist. If the shape parameter is near the boundary, then I would imagine that there's a good chance that you could end up with a sample that looks like all of your analysis methods are working properly, but really comes from a distribution that infrequently produces severe outliers.

Re:Heavy tail distributions are dangerous beasts (1)

Ambitwistor (1041236) | more than 4 years ago | (#29491243)

The thing is that when you're just looking at data points as they come in, you can't tell if a distribution is well-behaved.

If you make distributional assumptions (e.g., "it's a power law"), you can put probabilistic bounds on the shape parameters. If you're near a boundary, yes, there's going to be a high probability that the true parameter is on the "bad" side of the boundary. If you're far from a boundary, then there's a low probability. This is obvious.

If you don't make distributional assumptions, there's little you can conclude about the nature of any distribution.

Re:Heavy tail distributions are dangerous beasts (1)

NonSequor (230139) | more than 4 years ago | (#29492683)

If you don't make distributional assumptions, there's little you can conclude about the nature of any distribution.

I guess really my issue is that I think distributional assumptions are dangerous, particularly when you're modeling the behavior of crowds.

These sorts of systems can shift behaviors if they run into a constraint that hasn't revealed itself yet.

Re:Heavy tail distributions are dangerous beasts (1)

frank_adrian314159 (469671) | more than 4 years ago | (#29493095)

I think distributional assumptions are dangerous, particularly when you're modeling the behavior of crowds.

Yeah, that's true. But if you can model the behavior of the individuals as a simple enough Markov model (i.e., linear weight transitions) you can actually solve for exact moments. Scary. On the other hand, I have this odd feeling that most individuals aren't very linear.

I run Gentoo (0)

Anonymous Coward | more than 4 years ago | (#29481935)

how does this effect me?

Re:I run Gentoo (0)

Anonymous Coward | more than 4 years ago | (#29481953)

how does this effect me?

RTFA. It says there your kernel may take forever to compile.

Re:I run Gentoo (0)

Anonymous Coward | more than 4 years ago | (#29482187)

It will make you go bald faster. Then, chicks won't flock around you anymore.

Re:I run Gentoo (0)

Anonymous Coward | more than 4 years ago | (#29482203)

If he's running Linux, it's impossible for them to be flocking around him any less!

Re:I run Gentoo (0)

Anonymous Coward | more than 4 years ago | (#29483533)

If he's running Linux, it's impossible for them to be flocking around him any less!

I don't get it. What do advisory file locks have to do with any of this? Gentoo will support it no more and no less than any other POSIX implementation!

Re:I run Gentoo (1)

Hal_Porter (817932) | more than 4 years ago | (#29482675)

On a typical 3 inch by 2 inch graph of the long tail the data point representing you is about 17.45 feet to the right of the y axis and thus invisible.

Re:I run Gentoo (0)

Anonymous Coward | more than 4 years ago | (#29484309)

You're right at the top of the curve. But only if the chart is titled "Mandrake users looking for a way to get made fun of even more."

Long Tail Wikipedia Link (2, Insightful)

Bacon Bits (926911) | more than 4 years ago | (#29481967)

I just want to thank the submitter/editor for providing the link to Wikipedia for those of us who don't know what's meant by the Long Tail. As it happens, I do know what the "long tail" is, but one of the more tiring aspects of SlashDot is the number of narrow articles that hit the front page that wholly lack any sort of description.

Agreed (1)

Fantastic Lad (198284) | more than 4 years ago | (#29483683)

Well put.

--My 'favorite' instances are when an article is loaded up with acronyms with no explanation. It happens everywhere, not just Slashdot, that I find myself scanning an article in reverse looking for a bloody definition --and about as often as not, never finding one. Something about that just makes me steam.

Of course I can always wiki a definition myself, but the nice thing about having one linked directly from an article description is that I can be reasonably assured that the author is using the SAME definition as the one linked.

Ask ten people to define a relatively simple noun and you'll get ten totally different definitions, half of which are so incompatible that ridiculous flame wars can erupt over different readings of the exact same sentence.

Cheers!

-FL

Re:Long Tail Wikipedia Link (1)

LakeSolon (699033) | more than 4 years ago | (#29499537)

My feeling about Slashdot is that the submissions (and posts) should be heavy on links to references (such as Wikipedia where appropriate).

And that descriptions of terms can therefore be left out since this isn't print, and we may as well save the space (and mental bandwidth) since we have this handy new hyperlink feature.

Unfortunately far too many submissions are just the leading paragraph of the story copy/pasted, and painfully obviously not meant for this audience. Why these are even accepted (or, god forbid, not rewritten by the editors) is incomprehensible to me.

And slightly more apropos of the article: by random coincidence I've just received, while typing the above, an IM from a not-particularly-technical friend of mine who is telling me she's watching the back-catalog of Lost on Netflix instant-view. The long tail [wikipedia.org] indeed.

If I understand the Pareto distribution correctly (2, Informative)

Sique (173459) | more than 4 years ago | (#29481983)

Then the 80/20-rule is just a good rule of thumb.

If we have a simple hyperbolic distribution (which is a special case of Pareto), then adding more elements to the set and waiting for the distribution to renormalize as hyperbolic increases the relative weight of the top 20%. So if you have a big online retailer like Amazon with more titles than a conventional bookstore, then you can expect the top 20% sellers on Amazon generating a bigger part of all sales of Amazon than the top 20% of a bookstore in relation to all sales of said bookstore.

Re:If I understand the Pareto distribution correct (5, Informative)

ZombieWomble (893157) | more than 4 years ago | (#29482085)

What you posted describes the Pareto distribution, yes. However, the Pareto distribution is exactly the opposite of what the "Long Tail" model suggested by Andersen describes.

The crux of Andersen's argument is that, while Amazon et al have the same demand for big-name titles, their tail is longer and higher than a traditional bookstore, and by defining a cut at a certain point (say, those with less than 5% of the peak sales, those outside the top 10% or whatever is appropriate) it can be seen that the low-volume sales represent a larger fraction of the total sales due to the extreme length of the tail.

Quoting from the Wikipedia article on the topic:

In the graph shown above, Amazon's book sales or Netflix's movie rentals would be represented along the vertical axis, while the book or movie ranks are along the horizontal axis. The total volume of low popularity items exceeds the volume of high popularity items.

Andersen was suggesting that, in the limit of infinite items to sell and negligable stocking costs, much more profit is to be derived from the large number of items that sell a few copies than the few items that sell many copies.

Indeed, it went further than that, suggesting that as people got used to having more choice, they would begin to shun the "popular" items in favour of more obscure titles, further fattening the tail. But that's even more speculative and somewhat independent of the other economic predictions.

Shifted exponential (3, Informative)

AlpineR (32307) | more than 4 years ago | (#29482193)

I'm picturing the demand curve as an exponential, shifted so that it intercepts both the x and y axes. There's a lot of demand for the most popular items, and declining demand for less and less popular ones. By definition, of course, but the shape of the curve matters. No matter how far out you go, there's always somebody who'll want it (given a large enough population).

For a traditional bookstore, the x axis hits the curve pretty high. There's a substantial cost to stock each book, say $2.00/year. There's also a fairly small local demand, say 200 copies a week for a John Grisham novel. Only a few thousand titles sell fast enough to make a profit before that $2.00/year eats up the sale price minus wholesale price.

For a mail-order/online bookstore, the cost to stock each book is lower since you only need a warehouse instead of reading stacks + comfy chairs + cashiers + parking. The cost to stock each book could drop to $0.50/year. The demand is now national, so that same John Grisham novel sells 20,000 copies a week. And a title that sold once a year in a traditional store now sells twice a week. So, many more titles can beat the clock and turn a profit.

The shape of demand didn't change. In both cases it's an exponential cut off at the point of profitability. But that point is now much farther out along the x axis. So the online retailer can make money selling stuff that would never survive in a traditional store. And customers can find stuff online that they'd be lucky to ever see locally.

Re:Shifted exponential (1)

enrevanche (953125) | more than 4 years ago | (#29485011)

There are other issues that improve this type of retailers sales

  • A customer may buy an obscure item related to the popular one because it is both available and discoverable.
  • A customer may buy the popular items from Amazon because they also have the obscure ones, i.e they usually don't have to search for another source, this precludes competitors.
  • Amazon becomes the default supplier to most customers because of their ability to both provide diversity and methods for discovery.
  • Due to increasing concern over identity theft, customers become comfortable with a smaller number of suppliers. I.e. they do not have have to worry about finding something or paying for it safely.

There are some problems with this approach

  • It gives Amazon immense power to determine whether an item is ever sold because a few companies can become the only viable market. (This may be alleviated with things like Amazon Marketplace if they do not regulate what a seller may sell, especially in regard to media)
  • Many corporations are cowards and will cave to external pressure to ban something. (I do not know if Amazon ever did this).

Disclaimer: I buy a lot of stuff from Amazon (including from the US, Canada, France and occasionally the UK and Japan)

They redefined the terms and broke the model (4, Interesting)

petes_PoV (912422) | more than 4 years ago | (#29482217)

Since these guys aren't using the same definitions that the original book used to describe hits, niches and long-tails it's really no surprise that they get different results - they've interpreted the data in different ways!

The thing that's always struck me about the long-tail effect is that you've got to work it, to get value from it. Just having all the books or films by a particular author / actor isn't enough. You have to use that information and have the intelligent algorithms to guide your website visitors (or maybe "entice" would be a better word) to consider those alternate products. Just saying "Uuh, here's all the other stuff that guy's done" isn't enough, it needs enthusiasm and some knowledge of *why* a visitor might like a particular past work. That's where the gold lies: not in the long tail itself, but how you utilitise it.

Re:They redefined the terms and broke the model (1)

An Onerous Coward (222037) | more than 4 years ago | (#29483111)

Absolutely. The big hits are the big hits because they are (usually unlike anything in the long tail) known quantities. People you know have read and recommended them, or they base their recommendations on previous, related work. You tell people that District 9 is a movie about alien apartheid, you may not get much of a response. Tell them that it's a Peter Jackson movie, and interest skyrockets, because people have enjoyed his work before.

People want to know that their time and money is about to be well spent, and that's tough to do without guidance. Say an online book store has thirty million books in their catalogue, and no real information about the lower 29M, aside from maybe a description and a few sample pages. Wandering at random, and assuming that only a few percent of the books will seem promising given the information given, you'll probably have to read about fifty pages or so just to find a book worth buying, which is a pretty pricey investment.

Re:They redefined the terms and broke the model (1)

coaxial (28297) | more than 4 years ago | (#29483225)

You tell people that District 9 is a movie about alien apartheid, you may not get much of a response. Tell them that it's a Peter Jackson movie, and interest skyrockets, because people have enjoyed his work before.

Which is really a shame since it's really a Neill Blomkamp movie. Same thing with 9 and and Shane Acker. Watching the previews, you'd think Tim Burton did it. Granted, in this case promoting the exec producer is giving a boost to the film, but there's no reason to give the actual writer-director the short shrift, when plenty of movies are billed "From executive producer Uncle Moneybags, comes the latest film by director Mr. Filmmaker..."

Re:They redefined the terms and broke the model (1)

Flambergius (55153) | more than 4 years ago | (#29484059)

Yup. Logged on to say the exactly same thing. They might well have done some interesting research and have nice data, but they aren't by no means disproving Anderson's conjecture, just because they redefined the issue.

Movies? (2, Interesting)

Caesar Tjalbo (1010523) | more than 4 years ago | (#29482311)

From my own experience, I sometimes get an obscure book because I have a particular reason to get that specific title (be it for the subject or the writer). I listen to obscure music because it sits somewhere in my playlist and the player is on when I'm doing something else.

Reading a novel takes time but I find it no problem to put a book down while I prefer to watch a movie from start to finish in one go. Books on a particular subject I read when I have the time and the interest, or else use them as reference material for when I need to know something specific. Watching a movie requires my time and attention, it's something I plan rather than just listen to some music or read a chapter because I haven't got something better to do.

When I do sit down to watch a movie, I tend to stay on the safe side, try to get the highest chance of being entertained. That may well be with an obscure movie but more often it's with a reasonably wellknown title. But then, I can't remember when I last bought or rented a movie in stead of downloading so my consumption won't show up in any shop's statistic at all.

first pos]t (-1, Redundant)

Anonymous Coward | more than 4 years ago | (#29482393)

these challenges a previously gloves, condoms However I don't

Am I Following This Correctly? (2, Interesting)

Bob9113 (14996) | more than 4 years ago | (#29483261)

In the book, Anderson focuses on the definition of hits in absolute terms such as the top 10 or top 1,000 products, while Netessine and Tan argue that, to take growing product variety into account, one has to define popularity in relative terms, such as the top 1% or top 10% of products, to properly assess the presence or absence of the Long Tail.

So let me see if I get this:

Anderson says, "Assume a bell curve with a sufficiently large ordinal scale X axis. Select a sufficiently narrow segment in the center. The area under the curve not included in that segment will be greater than the area under the curve included in that segment."

Netissine and Tan respond, "However: Select a segment in the center whose width is a fixed percentage of the ordinal scale of the X axis. Select a percentage which is sufficiently large. Now the area under the curve in the included segment will be greater than the area not included."

For this, Anderson gets a best selling book, and two Wharton academics get a paper out of stating the counterpoint?!?

My brother went to Wharton. He is extremely smart. I know many of his friends from school -- they too are very intelligent folks. Wharton is not a kidding around school, it is pretty hard-core.

My brother and his friends from school mostly make stunning amounts of money. Many of them work for banks in the sorts of positions which, particularly given the recent total failure of banks at their primary wealth-creating task (risk management), might lead one to wonder if they are really responsible for enough sustainable GDP growth to justify their extraordinary compensation. To correct the first sentence in this paragraph, my brother used to make stunning amounts of money. A few years ago this very conundrum led my brother to retire, because he could not live with the disproportion between his production and his compensation. Most of his friends from school are not so infected with ethics.

Seeing this article, and the startling inconsequence of these supposed shining stars of business academia, I am inclined to agree with my brother's conclusion. And to reinforce my belief that we have, over the past 40 years, skewed the distribution of wealth toward the supposed best and brightest business thinkers, and away from all other areas of production, too heavily. Whether we, in chanting the mantra of ensuring that the business analysts and risk managers get fully compensated and motivated regardless of how outsized their compensation may appear to we mere mortals, have pushed the system much too far in that direction at the expense of compensation and motivation for those who are not business analysts and risk managers. Whether there may be forces at work which already influence cashflow in their direction, and our supposed levelling of the tax code has instead removed the normalizing force that was preventing an unhealthy portion of GDP from flowing to those with abnormally high influence on the flow of GDP -- but who, recent evidence suggests, are not really so extraordinary in their contribution to it.

Re:Am I Following This Correctly? (1)

Fantastic Lad (198284) | more than 4 years ago | (#29483603)

I think of the human brain in two respects. Software and hardware.

IQ tests are like hardware benchmarks. And sure, I've known my own share of not-kidding-around-smarts kind of people. (Who practically need cooling fans on their heads.) --But the software many of them were running was basically, "Donkey Kong". Which is to say, I've known more clued-out 'smart' people than I'd be comfortable shaking a stick at.

Sounds like your brother was one of the very few who had, "Wisdom 1.0" installed.

It's an unfortunate indicator that he decided to leave the banking industry, allowing those running Donkey Kong to continue managing our economy.

-FL

Re:Am I Following This Correctly? (0)

Anonymous Coward | more than 4 years ago | (#29484049)

Other than the fact you have the curve wrong, you are pretty close. But the curve matters a lot.

Revenue tends to follow a power law distribution. In a simple power law distribution the value of the n'th most valuable item is 1/n times the value of the most valuable item. (There are families of power law distributions in which this relation doesn't hold exactly, but it tends to be reasonably accurate.) Therefore the combined value of the first n items is about k*log(n) where k is the value of the most valuable item. Since log grows very slowly, the bulk of the value is in the first few items. These are your "hits", and their disproportionate value is so extreme that everyone is aware of them.

Anderson's insight was that the aggregate value of the tail becomes huge for large n, and by making n larger the value of the hits goes down.

This paper has the insight that as n grows the proportionate value of the top 5% grows. You can see this in the simple distribution we described. If aggregate value out to the n'th element grows like k*log(n), then the aggregate value out to 0.05*n should be about k*(log(n) - log(20)). Which means that the aggregate value of the last 95% should be a fixed multiple of the value of the hits, namely k*log(20). Which is not too far off from the value of the top 10. So as n grows the relative value of the top 10 and the bottom 95% diminish relative to the total at about the same rate!

So by slicing the same curve in different ways they get different answers. The question is which answer is more appropriate. I would side with Anderson. The business problem of stocking and marketing your top 10 products is very different than stocking and marketing the top 5% of products out of a very large selection. A company that can solve the latter problem can make much more than a company that only solves the first one. You obviously can't disregard your top 10 products. But you can't stop there either.

Re:Am I Following This Correctly? (1)

Keith Henson (1588543) | more than 4 years ago | (#29507587)

"and away from all other areas of production, too heavily."

Truer words were never spoken. There are no lack of things that could be done, but very little interest in them.

Google Oil Drum Henson for an example. If you want, get in touch.

Keith Henson hkhenson at rogers dot com

discredited a long time ago as a commercial model (1)

speculatrix (678524) | more than 4 years ago | (#29485397)

I thought the long tail was significantly discredited a while ago. Lets check google [google.co.uk]. hmm, not entirely. however, the guy who wrote it keeps coming up with new ideas which get a lot of attention and even praise before cooler heads actually think about it properly. The FT took a look at Chris Anderson's book freemium and, well, read John Gapper and his [ft.com]follow-up questions [ft.com].

Visual search helps find Long Tail of Travel (1)

Coastal_Ron (1617229) | more than 4 years ago | (#29487293)

No matter how large a catalog of content that Netflix and Amazon have, the real challenge is giving their users the tools to find the content. For movies, you can search for one or more parameters like categories, actors, awards, directors and even member favorites. Apple has tried to add some assistance for music with their "Genius" service, but each movie, song or book is like a painting - each one unique, and each can be identified using many descriptions.

For travel, you have the advantage of narrowing your choices by the general area you'll be visiting. If you're traveling to New York City, you don't have to consider any hotels that are in Los Angeles. Still, New York City has lots of hotels to choose from, and when you're looking on Expedia or Travelocity, they can show you page after page of potential places. Even when you narrow your choices, you still have to look and see where the hotels are located, and see if that fits in with the rest of your trip.

We created Where's URL [wheresurl.com] to address the location problem. We show you where everything is, and give you the ability to filter by category (Lodging, Food/Drink, Attractions, etc.) and sub-category (hotel, B&B, art museum, etc.). By letting you choose places by their location, we have evened the playing field for those businesses on the tail-end of the Long Tail of Travel. When you search for hotels in New York, you can easily find the ones that are across from Central Park, or near SoHo.

The Long Tail in Music (1)

ocelma (922639) | more than 4 years ago | (#29500133)

There's always been a long tail in the demand curve, and it always will.
What we need are useful recommendations that guide us from the head to the hidden treasures located along the tail area.

Also, I find very disappointing that none (Wharton, Anderson, etc.) uses the Long Tail model proposed by Kalevi Kilkki ( http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1832/1716 [firstmonday.org] ). This model has a better formal definition of the Head, Mid and Tail parts of the curve (not based neither on absolute nor on %), based on how to split the (log) x-axis.

[shameless plug] I did a PhD named "Music Recommendation and Discovery in the Long Tail" http://www.iua.upf.es/~ocelma/PhD/index.html [iua.upf.es]
So, I also did some boring analyses about the Long Tail in the music (recommendation) domain.

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...