Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

eBay Makes Huge Gains In Parallel Efficiency

kdawson posted more than 5 years ago | from the free-servers-and-a-pony dept.

Supercomputing 47

CurtMonash writes "Parallel Efficiency is a simple metric that divides the actual work your parallel CPUs do by the sum of their total capacity. If you can get your parallel efficiency up, it's like getting free servers, free floor space, and some free power as well. eBay reports that it amazed even itself by increasing overall PE from 50% to 80% in about 6 months — across tens of thousands of servers. The secret sauce was data warehouse-based analytics. I.e., eBay instrumented its own network to do minute-by-minute status checks, then crunched the resulting data to find bottlenecks that needed removing. Obviously, savings are in the many millions of dollars. eBay has been offering some glimpses into its analytic efforts this year, and the PE savings are one of the most concrete examples they're offering to validate all this analytic cleverness."

cancel ×

47 comments

Finance (3, Interesting)

Exanon (1277926) | more than 5 years ago | (#25430483)

Huh, that should hopefully make their stock price go up a little bit. Whatever mitigates the financial crisis is great...

Re:Finance (1)

Colin Smith (2679) | more than 5 years ago | (#25430801)

Whatever mitigates the financial crisis is great...

Why would stock prices go up if the value of a dollar is in the process of doubling?

The values of the stock markets have almost nothing to do with anything real. Far more important are the numbers of real and imaginary dollars in existence.

Or put another way. Want the stock markets to go up? Well, persuade your bank to stop taking dollars out and shooting them.
 

Re:Finance (3, Interesting)

Anonymous Coward | more than 5 years ago | (#25430985)

Why would stock prices go up if the value of a dollar is in the process of doubling?

Doubling? Against what benchmark has the value of a dollar doubled? The euro? Gold? UK pound? Canadian dollar? Swiss franc? The price of oil has dropped dramatically, but that has more to do with the oil market realizing that a world recession is going to reduce demand for oil.

The values of the stock markets have almost nothing to do with anything real.

Slightly true. A share is a real, small fractional ownership of a real company. Companies that have assets, sales, products, services, employees, and (hopefully) profits. The intrinsic value of that share can be determined by looking at the underlying business and calculating its value.

But then you get interesting things. If you look at the value of the New York Times, they only have one real asset: their massive headquarters on prime real estate in Manhattan. The value of the New York Times company is almost the same as the value of that building. All the other assets (printing presses, distribution channels, the brand recognition of a famous newspaper, advertising, sales, other publications, etc) are valued by the market at next to nothing. Is the market wrong? Maybe. The business must be worth something, right?

On the other hand, the New York Times is controlled by one family and will be very difficult to take over. The family isn't having much success in growing the business. All newspapers are having trouble with circulation numbers, and they are getting squeezed for advertising revenue by other sources (like online ads). To remain a successful business, they need to cut costs and/or increase revenue. That's why the market valuation of the New York Times is next to nothing compared to the hard assets.

Far more important are the numbers of real and imaginary dollars in existence.

Imaginary dollars? Care to explain? What are these magical imaginary dollars?

Despite your low slashdotID, you sound a bit like a kook.

Re:Finance (1)

eat here_get gas (907110) | more than 5 years ago | (#25431457)

[quote]...The values of the stock markets have almost nothing to do with anything real....[/quote]

i call bullshit! the worth of the stock market tonight is what everyone is worth tomorrow. i don't have a single dollar in the stock market, but i can see how it effects the few i do have in my wallet...

Re:Finance (1)

Brad Eleven (165911) | more than 5 years ago | (#25447163)

The values of the stock markets have almost nothing to do with anything real.

Exactly. Even if eBay has made real efficiency improvements, few investors will comprehend the meaning of the claim in terms of "anything real," e.g., power consumption, productivity, seller satisfaction, etc.

They'll just think, "That's a good place to put my money."

Right after that, they'll think, "Isn't it?"

Of course their PE has gone up (1)

hackingbear (988354) | more than 5 years ago | (#25433027)

They have just successfully raised their PE simply by sending their per share stock price from 32 to 16. What an achievement!

Good for them! (3, Interesting)

vintagepc (1388833) | more than 5 years ago | (#25430497)

Kudos to them for taking the initiative to be more efficient rather than to just buy more servers to increase capacity. On a side note, I wonder if this makes their servers able to process their data faster... If so, it means more getting sniped (and possibly snipers getting sniped) for the average joe.

But where's the beef? (5, Insightful)

TheLink (130905) | more than 5 years ago | (#25431119)

They talk about improving efficiencies, 1.6X etc, but ok so what got improved?

What interesting things did they learn? What were they doing wrong before and what did they change?

I don't see any hard facts or much useful info.

Car analogy: it's like Ford says we've improved engine efficiencies 1.6X.

But you don't even get new MPG figures, no comparison of 0-60 before and after (to show whether there was any impact on performance), no torque curves, not even a mention of "high intensity electric fields reducing viscosity".

So to me it's as good as some PR firm bullshit and should not be on Slashdot.

Heck it's about as much useful news as programmer productivity improving because Slashdot went down for a day.

Re:But where's the beef? (1)

beav007 (746004) | more than 5 years ago | (#25437021)

So to me it's as good as some PR firm bullshit and should not be on Slashdot.

This is called a slashvertisment, and is not uncommon in these parts.

To paraphrase: You must be new here...

Re:But where's the beef? (1)

TheLink (130905) | more than 5 years ago | (#25449461)

Just because it's not uncommon doesn't mean it should be here.

Huge ebay hole (-1, Troll)

Anonymous Coward | more than 5 years ago | (#25430503)

For sale [goatse.cz]

Re:Huge ebay hole (0, Funny)

Anonymous Coward | more than 5 years ago | (#25430553)

Looks a little worn out and used to me. Is the price negotiable?

Re:Huge ebay hole (0)

Anonymous Coward | more than 5 years ago | (#25436489)

It goes to the highest bidder

Of course (4, Funny)

transporter_ii (986545) | more than 5 years ago | (#25430509)

With the extra millions saved, they are going to lower their fees...as well as spend some time trying to figure out how to quit pissing everyone off...right?

Transporter_ii

Re:Of course (0)

Anonymous Coward | more than 5 years ago | (#25430677)

With the extra millions saved, they are going to lower their fees...as well as spend some time trying to figure out how to quit pissing everyone off...right?

Transporter_ii

OMG! Ponies!

Re:Of course (0)

Anonymous Coward | more than 5 years ago | (#25431977)

The first thing that entered my mind was how this savings will NOT be passed on to its customers

Re:Of course (0)

Anonymous Coward | more than 5 years ago | (#25436851)

With the extra millions saved, they are going to lower their fees...as well as spend some time trying to figure out how to quit pissing everyone off...right?

Not when they have critical mass - people buy at eBay because they have everything, and people sell on eBay because buyers come there. That, and there isn't many payment options other than Paypal for some random person to accept credit cards - getting a merchant account is difficult if you're not doing some minimum amount of money every month.

eBay learned a long time ago they can piss people off and they'll still come back...

One bottleneck begets another (1)

lamapper (1343009) | more than 5 years ago | (#25430533)

WOW, way to go IT techs for ebay! I find it fascinating reading articles on the removal of bottle necks. Its interesting the tools and methods that are used to monitor specific parts of the company's IT System (servers, network, applications and more).

It is interesting how one bottle neck is overcome only to find yet another bottleneck. Very cool.

News from the Future... (2, Funny)

Anita Coney (648748) | more than 5 years ago | (#25430571)

Today, ebay announced that it obtained a patent on using warehouse-based analytics to increase parallel efficiency in server operations.

Re:News from the Future... (3, Interesting)

Kris_J (10111) | more than 5 years ago | (#25430671)

And if you want to deal with them, you have to use their warehouse-based analytics.

Oh, and PayPal and Ebay fees will be going up next week.

Re:News from the Future... (1)

Cylix (55374) | more than 5 years ago | (#25430811)

Today, ebay announced that it obtained a patent on host monitoring.

Seriously, doesn't everyone already monitor every tiny metric and application hook?

I know one company that had an awesome metric accounting system. Tons of data, down to the second, and extremely efficient.

Impressive Gain (1)

Apple Acolyte (517892) | more than 5 years ago | (#25430601)

That is an impressive gain eBay received through those analytics. I have to wonder, though, why it took so long for their network team to remove bottlenecks. Was there some breakthrough technology that just recently allowed for this kind analysis, or were the engineers lazy all these years?

Re:Impressive Gain (1)

Colin Smith (2679) | more than 5 years ago | (#25430771)

Excellent troll. 7/10

 

Re:Impressive Gain (1)

WeblionX (675030) | more than 5 years ago | (#25434849)

L@@K! Make impressive gain in your parallel efficiency! 3 months supply! L@@K!

Or maybe this was done to a mgmt goal? (4, Insightful)

bboxman (1342573) | more than 5 years ago | (#25430621)

I can imagine this: Some eBay hotshot comes up and says: "we only use 50% of our servers, we've got to do better here". So:
1) They don't buy new servers. Workload increases, better utilization, no analytics involved.
2) Or, someone got clever, and added an idling process to each idle server. Presto, we've improved our PE -- and we've got a nice yearly bonus as well.

The article actually says nothing, besides claiming a supposed 1.6x improvement, besides a very vauge refrence to analytics. This ./ post is actually meant to promote www.xlmpp.com.

Re:Or maybe this was done to a mgmt goal? (1)

rhizome (115711) | more than 5 years ago | (#25432519)

I can imagine this: Some eBay hotshot comes up and says: "we only use 50% of our servers, we've got to do better here". So:
1) They don't buy new servers. Workload increases, better utilization, no analytics involved.
2) Or, someone got clever, and added an idling process to each idle server. Presto, we've improved our PE -- and we've got a nice yearly bonus as well.

I'm guessing their improvement figure of 50% -> 80% doesn't include the machines and processing overhead necessary to run the datawarehouse and status checks. In other words, rearranging deck chairs, hiding the salami.

Re:Or maybe this was done to a mgmt goal? (0)

Anonymous Coward | more than 5 years ago | (#25433481)

The article actually says nothing, besides claiming a supposed 1.6x improvement

Well, it's right in there where they got it.

eBay said that in "about six months" they increased their parallel efficency from 50% to 80%.

80% / 50% == 1.6 factor of improvement.

But overall I agree with you: this was one article where just reading the summary was all you need.

Nice timing for this story... (5, Interesting)

Adult film producer (866485) | more than 5 years ago | (#25430633)

Right now is the time to soothe investor fears caused by their recent tapping of a $1 Billion line of credit..

http://www.reuters.com/article/ousiv/idUSTRE49G7L420081017 [reuters.com]

Analyst forecast lower revenue for Ebay in coming quarters, DOH.

http://tinyurl.com/5e69mt [tinyurl.com]

Ebay credibility (1)

syousef (465911) | more than 5 years ago | (#25430657)

...and Paypal makes all your transactions safe and worry free.

Pull the other one.

erlang (0)

Anonymous Coward | more than 5 years ago | (#25430705)

The erlang programming model makes it very easy to write applications that scale to "n" cores!

self promotion (2, Insightful)

jcookeman (843136) | more than 5 years ago | (#25430747)

Is xlmpp.com a site for three egoists to drown in their own self promotion?

Sellers Leaving in Droves, and Many Glitches (2, Interesting)

Ron Bennett (14590) | more than 5 years ago | (#25430851)

eBay today isn't the same type of place as 6 months ago. So much has changed; it's essentially a just facade of its former self.

eBay sellers have been leaving in droves, and there have been more glitches, some quite serious, on both eBay and PayPal lately.

It would have been more interesting to see such an article discussing parallel efficiency gain at say Amazon or some other large retailer whose business model / activity level had remained similar during the time period being measured.

Ron

leaving to where? (1)

Trepidity (597) | more than 5 years ago | (#25435543)

eBay has been pissing people off for years, but the main problem is that a functioning marketplace requires a critical mass of buyers and sellers, and none of the competitors (e.g. Yahoo Auctions) have managed to build nearly enough of a critical mass. Just about the only exceptions are in product-specific areas, for example eBay has never really owned the used book market, where Amazon Marketplace et al do a brisk business.

Slashvertisement (4, Insightful)

bendodge (998616) | more than 5 years ago | (#25430911)

Yes, we know eBay is trying to boost it's stock value and xlmpp.com wants more traffic.
Tag it what it is an move on. There's not much real info here.

Teradata?? (0)

Anonymous Coward | more than 5 years ago | (#25431125)

Actually they use Teradata... which has always been parallel everything... disclosure -I work for Teradata and "would know!"

Oh, good! (1)

penginkun (585807) | more than 5 years ago | (#25431393)

This means the scammers can rip buyers off even more efficiently. Good job, eBay!

Ah, this explains it then! (1)

pandrijeczko (588093) | more than 5 years ago | (#25432075)

This of course explains why eBay fees have increased yet again - as sellers we're paying for increased eBay efficiency meaning that we will get higher prices for our stuff and will not be answered by a useless eBay tosser after sitting in a call queue for 30 minutes.

Cool!

Self-optimizing (0)

Anonymous Coward | more than 5 years ago | (#25432139)

It's called self-optimizing. All companies and governments should use it. It is how the Japanese have excelled in autos, electronics, cameras, etc. A few US companies like Harley-Davidson use it, and of course, Linux and GNU development are examples of this. http://en.wikipedia.org/wiki/W._Edwards_Deming/ [wikipedia.org]

Oh great - now they're laying off machines too? (1)

schwaang (667808) | more than 5 years ago | (#25432611)

I mean, they're already increasing the "parallel efficiency" of their human workforce [mercurynews.com] .

Servers of the world, unite!

Makes sense.... (1)

MostAwesomeDude (980382) | more than 5 years ago | (#25433209)

After all, with all the stolen goods from TSA showing up on eBay, it makes sense that they'd need to beef up their servers. :3

What does ebay do anyway? (-1, Troll)

Anonymous Coward | more than 5 years ago | (#25433563)

What on earth does Ebay do? I mean they have no logistics, no inventory to manage. Its mostly people posting items, and managing the progress of a whole lot of auctions. There are some financial and anti-fraud measures on the backend...but give me a break.

Ebay hearts java and I think it is the use of slow ass languages and crappy programmers that makes having soo many systems necessary.

I would be very surprised if a handful of quad core PCs you can pick up at your nearest electronic store would not be capable of handling all of ebays transactional requirements if written properly.

Wait, let me guess. (1)

ReedYoung (1282222) | more than 5 years ago | (#25434381)

"The secret sauce was data warehouse-based analytics. I.e., eBay instrumented its own network to do minute-by-minute status checks, then crunched the resulting data to find bottlenecks that needed removing."

Performing this number-crunching on idle CPUs/cores is responsible for 90% or more of the improvement from 50% to 80% Parallel Efficiency?

Now I've read the article, and wish I hadn't. (1)

ReedYoung (1282222) | more than 5 years ago | (#25434979)

It's just free advertising for the corporation mentioned in the second link with no useful information, ie no "News for Nerds", about the process they used beyond the vague "data warehouse" in the summary. If /. was always like this, everybody would stop reading the articles.

Re:Wait, let me guess. (1)

yukk (638002) | more than 5 years ago | (#25441385)

Performing this number-crunching on idle CPUs/cores is responsible for 90% or more of the improvement from 50% to 80% Parallel Efficiency?

Nah, they just realised that by turning off all that heavy monitoring they could suddenly get more performance out of the original servers.

Parallel Efficiency Explained (0)

Anonymous Coward | more than 5 years ago | (#25435023)

They managed to squeeze in another 1U server on top of their existing boxes.

Good job..ebay (0)

Anonymous Coward | more than 5 years ago | (#25437799)

No matter what, but by saving some power they are doing good for all of us by reducing their energy consumption footprints for better environment. It is not necessary to always have benefit from what you do.. selfish hunters.. think outside of well..

Big Surprise! (1)

DragonWriter (970822) | more than 5 years ago | (#25460977)

If you actually look for where the performance bottlenecks are and remove them, you get better efficiency. Who would ever have guessed?

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...