Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

The Art of Scalability

samzenpus posted more than 4 years ago | from the read-all-about-it dept.

Book Reviews 63

Martijn de Boer writes "Creating high performance growing networks is really a special skill managers and network architects should possess to be ready for the future. The Art of Scalability is a book written for these kinds of functions, and prepares you for the present and the imminent future. Scalability is achieved by principles that work on many levels within enterprises, whether it's processes, organizational structure or setting up your project, this book covers it all." Read on for the rest of Martijn's review.The sub-title for this book is "Scalable Web Architecture, Processes and Organizations for the Modern Enterprise" which is basically a catchy title for project managers and decision makers. It's just catchy enough to grab the book from a shelf and start digging into the table of contents, which is exactly the spot where you get hooked and decide to get the book. The book is written by two experts on the subject; both of them have a strong background in large corporations dealing with scalability. Throughout the book the authors shine a good light on tools and cases used for making a project scalable.

The book is divided into four sections which follow the process of starting a scalable project. First off you will need to know everything about strategy, organizational structure, the kind of people your organization needs and how to manage your team. The second part focuses on the how's and why's of scalability, aspects of planning for continuity, crisis and incident management. The later chapters in this second part talk about risks, how to value risk and the importance of testing to make educated conclusions about how far to scale your project, a notable mention here is chapter 19 which focuses on trade-offs in development speed or doing things the right way, often an underestimated point in these businesses. Architecting scalable solutions is the third part of the book, mainly taking a more technical approach to the matter. This part talks about containing faults and breaking up applications so they scale well. This goes all the way to scaling the database backend for your application to caching objects and making synchronous versus asynchronous calls to your application.

While most of the book is good for preparation, I think the fourth part of the book crosses the boundary from preparation to being very useful during the process as a fallback to your knowledge. It's titled accordingly "Solving other issues and challenges" and mainly focuses on things you will come across during the first and perhaps even later stages, providing solutions for unforeseen costs of data storage to monitoring your application for user experience, speed and the processes you will have to implement for this.

If you have read this far, and are thinking about how complex scalability really is, then you have found the right book. The clear writing style and detailed writing of numerous pitfalls for such big projects make this book a valuable asset to your knowledge base. It's also written in a challenging way, and between the lines there is also some humor (evident for instance in the reference to Rain Man in chapter 18), making it a fun way to learn or build upon a great skill set needed by organizations in the not so distant future. I feel very positively about the writing style of the book, but despite there being some illustrations there could be some more diagrams of organizational structure and the architecture of projects as I didn't find the illustrations adding much to the context.

The short appendixes to the book are mostly about calculating capacity for cpu power, bandwidth, etcetera. These appendixes actually provide sample calculations which are useful to backup your analysis when you need to make a bill of materials for your superiors and can also be used to better grasp the size of a project. Numbers mostly provide a context for the people above you, and I think these extra pieces of content could have deserved another part in the book, but perhaps it's better off as some food for thought.

Before starting with this book, I had not much prior knowledge about scaling projects other then some technical ideas of implementation. I always hate it when I know I'm just scratching the surface of something, and need to satisfy my urge to learn more about the subject. The Art of Scalability really helped me accomplish that, and provided much more background information then I expected. I was really surprised by the pieces about cloud computing, it's such a buzzword nowadays but the texts give it a real context.

If you are looking to set up a project or are generally interested by the concepts of scalability, then this is the right book for you. It's an appropriate recommendation for most businesses, because this knowledge can only be used to your advantage and just takes a little bit of time to read trough. It's a heavy subject, but once you finish the book you will be able to make a decision about architecture based on a good foundation of background information rooted in real situations.

You can purchase The Art of Scalability from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Sorry! There are no comments related to the filter you selected.

Ferst Past (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31024910)

HI all.

this book covers it all (3, Funny)

djupedal (584558) | more than 4 years ago | (#31024938)

Didn't we just go thru this with the slime mold that grows perfect networks?

http://science.slashdot.org/article.pl?sid=10/01/22/1715229 [slashdot.org]

So if this is so hard for humans and they need new books to work it out, why can slime mold do it?

Is there a slime mold chapter in this book or is that coming in next release?

Re: this book covers it all (4, Funny)

fm6 (162816) | more than 4 years ago | (#31025366)

The problem is finding a network administrator who's smarter than a slime mold.

Re: this book covers it all (1)

Monkeedude1212 (1560403) | more than 4 years ago | (#31026500)

You mean finding one who is as smart as slime mold.

Re: this book covers it all (1)

pclminion (145572) | more than 4 years ago | (#31025604)

Maybe because in some ways, slime molds and fungi (slime mold isn't a true fungus) actually have far more advanced physiologies than our own. Don't confuse the primitive appearance of the organism with its biological complexity and resilience. If the fungi had ever evolved locomotion, they would be dominating the planet right now.

Re: this book covers it all (0)

Anonymous Coward | more than 4 years ago | (#31025616)

Fungi are capable of rudimentary locomotion.

Re: this book covers it all (0)

Anonymous Coward | more than 4 years ago | (#31029848)

they are dominating the planet right now.....

Re: this book covers it all (1)

pclminion (145572) | more than 4 years ago | (#31030728)

There's no denying that the fungi are incredibly successful, but by "dominating" I meant more of the "causing other species to tremble in fear" sort of thing.

How Marketable Will That Skill Be? (1)

damn_registrars (1103043) | more than 4 years ago | (#31024974)

I'm not sure that scalability will be all that relevant as long as Moore's Law continues to hold. IIRC it has taken on average less than 10 years for the computing power of a top10 supercomuter to be available in a consumer laptop. As computational resources on the market continue to expand, I would suspect that we could soon come to see the opposite of scalability becoming more important - getting more done with less space and energy.

That said, any admin worth their salt who focused on scalability should still be an excellent sys admin anyways, so they shouldn't be heading towards the bread line. I'm just not convinced that focusing on learning skills for network scaling is the best idea unless you want to build clusters for research groups.

Re:How Marketable Will That Skill Be? (3, Insightful)

PhrostyMcByte (589271) | more than 4 years ago | (#31025092)

As the number of cores in your computer increases, and with things like NUMA starting to take hold, I suspect many of the skills learned in horizontal scaling will start to become very important.

Re:How Marketable Will That Skill Be? (2, Insightful)

Maximum Prophet (716608) | more than 4 years ago | (#31025158)

Intel et. al. are achieving Moore's law by adding more cores to existing chips, not making them much faster for single threaded processes. In ten years, if you really want your code to run fast, it may have to scale across 10s of processors just to run on any given machine.

Re:How Marketable Will That Skill Be? (2, Insightful)

Anonymous Coward | more than 4 years ago | (#31025358)

Did you even look at the title of the book?

"Scalable Web Architecture, Processes and Organizations for the Modern Enterprise"

This has more to do than just processing power. Good Grief...

Re:How Marketable Will That Skill Be? (3, Insightful)

Blakey Rat (99501) | more than 4 years ago | (#31025478)

I'm not sure that scalability will be all that relevant as long as Moore's Law continues to hold.

Ask Twitter or Facebook.

Moore's Law is helping them, I'm sure, but for really successful new websites, it's nothing compared to the influx of new customers to deal with. (And yes, it's Slashdot so we all must hate Twitter, but you have to admit they grew damned fast.)

I think you're overestimating the value of CPU power here. The real bottleneck is always data-- storing data in such a way that access to it is fast, but keeping it in a consistent state at the same time. Nobody would use Facebook or Twitter if it took a couple minutes for your status update to be visible by your friend.

Re:How Marketable Will That Skill Be? (1)

PopeRatzo (965947) | more than 4 years ago | (#31027506)

.. we all must hate Twitter, but you have to admit they grew damned fast.

Like asian carp in the great lakes, and just about as welcome.

Re:How Marketable Will That Skill Be? (0, Offtopic)

Blakey Rat (99501) | more than 4 years ago | (#31027530)

Wow, despite the disclaimer somebody still posts the "OMG I hate Twitterz so much!" post. I guess there's just no avoiding it on this site. Yes, we get it. You hate Twitter. We. Get. It. Stop posting it. Please.

Re:How Marketable Will That Skill Be? (2, Funny)

cerberusss (660701) | more than 4 years ago | (#31028500)

CHAPEL HILL, NC–Area resident PopeRatzo does not use Twitter, a fact he repeatedly points out to friends, family, and coworkers–as well as to his mailman, neighborhood convenience-store clerks, and the man who cleans the hallways in his apartment building.

"I, personally, would rather spend my time doing something useful than tweeting," PopeRatzo told a random woman Monday at the Suds 'N' Duds Laundromat, noticing the establishment's wall-mounted TV. "I don't even own an account."

Re:How Marketable Will That Skill Be? (2, Funny)

PopeRatzo (965947) | more than 4 years ago | (#31029514)

I'm glad to see that press agent I hired is earning his money.

Re:How Marketable Will That Skill Be? (1)

PopeRatzo (965947) | more than 4 years ago | (#31029504)

You hate Twitter. We. Get. It. Stop posting it. Please.

Sorry, friend, but the answer is "no".

You can have my twitter-hatred posts when you wrest them from my cold, dead hands.

Are you also fed up and sick and tired of the Windows-hating posts? Or IE hating posts? Or Apple-hating posts? I must have missed your impassioned pleas for those to stop.

Interesting that with all the griping and carping that goes on here, with all the rants against this product or this technology or that company, the one thing you decide to stand up and protect is...Twitter.

Interesting indeed.

Re:How Marketable Will That Skill Be? (2, Insightful)

Blakey Rat (99501) | more than 4 years ago | (#31029694)

Are you also fed up and sick and tired of the Windows-hating posts?

Yes.

Or IE hating posts?

Only the ones that don't acknowledge IE7 and IE8 exist, which is most of them. So... yup.

Or Apple-hating posts?

Yah.

I must have missed your impassioned pleas for those to stop.

Well, ok. I did a Windows one just a little bit ago... here it is: http://slashdot.org/comments.pl?sid=1531200&cid=30967046 [slashdot.org]

Is it really that wrong to want the level of discourse to rise above "moron" levels on this forum? Is it really so wrong that I object to reading the exact same comment a hundred times a week, that people get modded up for adding absolutely *nothing* we haven't seen a million times before?

Interesting that with all the griping and carping that goes on here, with all the rants against this product or this technology or that company, the one thing you decide to stand up and protect is...Twitter.

Maybe you should have waited for the answer to your questions before making assumptions. Because now you just look like an idiot.

Re:How Marketable Will That Skill Be? (1)

PopeRatzo (965947) | more than 4 years ago | (#31030616)

OK, you're passion to defend twitter from my hatred is greater than my desire to continue to declare my disgust.

I will never, ever again express my distaste for Twitter in any way. Not on the Internet and not personally. Nor will I cast aspersions on Windows, Microsoft, Sony or Apple. Yet I reserve the right to express negative opinions of users thereof (especially Apple, but only for the entertainment derived from their outraged reactions).

You have convinced me of the error of my ways, friend.

Because now you just look like an idiot.

Just "now"? You, sir, have not been paying attention.

If I may however, I do believe the callous use of the word "idiot" which for decades was the technical term for people who are mentally challenged, warrants an apology. You are above such degrading terms, and the hundreds of thousands of mentally challenged people and their families deserve better. I believe there is some boilerplate recently made available for such an apology.

Re:How Marketable Will That Skill Be? (1)

inKubus (199753) | more than 4 years ago | (#31031990)

Unfortunately, they have multiple datacenters and they are running cached mysql with some magic glue so it often does take several minutes for changes to propagate from coast to coast, become cached, etc. That's why the default feed is not live, but you can click it to get it--they would be brought to their knees doing all live updates.

They do use a technique called sharding which keeps the row counts down. Basically they break up all the users into different tables, databases or even servers. Then they try to keep users who talk to each other frequently in the same "shard". This minimizes the joins and of course enables more horizontal scalability. Personally, I think they could probably do more with a mainframe style of system but they don't have the money for that. Not that I hate LAMP but really the site is shoddy. Unpredictable at best, and really it doesn't do much. Just holds some information from almost every person in the U.S.

So what? TransUnion and Experian have held our entire credit histories and can not only retrieve them but score them with a complex algorithm in a few milliseconds. Which goes to show (and I've been trying to tell my boss this with some success) is that it's not just the data or the complexity but the interactivity that really intensifies the need for speed. Ajax and stuff is cool but it's only cool when it works fast. Otherwise it sucks.

So now we're dealing with hundreds of tiny requests rather than one big one. Apache was designed to handle those big requests. So you run into lots of issues there. Yeah, there's lighty and nginx which are going to be the new kings of that stuff. I'm thinking about probably moving my web services to a separate web server from my big MVC framework pages.

Re:How Marketable Will That Skill Be? (1)

z-j-y (1056250) | more than 4 years ago | (#31040040)

twitter wasn't scalable at all even long after it became famous.

don't buy this book. only 12.5 people on the earth need to know scalability, and they don't need to read this book either.

Very Marketable (1)

TiggertheMad (556308) | more than 4 years ago | (#31025516)

As computational resources on the market continue to expand, I would suspect that we could soon come to see the opposite of scalability becoming more important - getting more done with less space and energy.

If you think the solution to scaling issues is to 'throw more silicone' at the problem, you are about the worse programmer in the universe. This talent is incredibly handy, as any site shut down by Slashdot's ravening hordes would attest. What you are suggesting is akin to 'Bubble sorts are a great algorithm if you do them an a fast CPU'. Any tech company that hopes to achieve more than a small audience will need people with this skill set.

With databases hitting petabyte sizes, and a world wide Internet audience in the hundreds of millions and growing, I very much doubt that scaling issues will be resolved by simple hardware solutions anytime in the foreseeable future.

Re:Very Marketable (0)

Anonymous Coward | more than 4 years ago | (#31026080)

hay! i like bubble sorts!

Re:Very Marketable (1)

emt377 (610337) | more than 4 years ago | (#31026224)

What you are suggesting is akin to 'Bubble sorts are a great algorithm if you do them an a fast CPU'.

This is an interesting analogy, because the fastest possible way to sort 3 elements is bubble sort. It's a good example why reductionism often yields poor algorithms. It's also relevant to scalability of implementation versus scalability of architecture. The former is more about software design, running MT hot, clustering, db schemas, etc, while the latter is more about partitioning the problem and processes. Many business processes are much too big for any one person to comprehend in depth, and it's important to partition it so everyone can understand the overarching purpose and structure, but only have to be expert on one part. These can then further partitioned if big enough. Scalability of architecture in the really comes down to modularity and an ability to identify and eliminate bottlenecks - an implementation may have bottlenecks, but with a good architecture individual parts can be reimplemented or provisioned as needed to eliminate them. It's also common to architect for the biggest realistic scope but only implement what's actually needed; this guarantees that things like policy management goes in place early even if it's mostly a skeleton. Since policy needs to be managed at the highest abstract level it needs to be part of the architecture, and is one of those things that's virtually impossible to bolt on after the fact. Anyway, somehow I wanted to convey that good architecture can't be reduced to good implementation and while good implementation is important it's not actually essential. What's essential is that good implementation is possible. (This is also why if you can't design and implement software at high level of proficiency you can't architect, because you will invariably paint yourself into a corner.)

Re:Very Marketable (1)

psycho_eddy (681884) | more than 4 years ago | (#31026450)

If you think the solution to scaling issues is to 'throw more silicone' at the problem, you are about the worse programmer in the universe.

throwing more silicone at the problem has helped many an aspiring playmate. i know this is unintentional, but i got a chuckle out it :-) emphasis mine.

Re:Very Marketable (0)

damn_registrars (1103043) | more than 4 years ago | (#31027060)

As computational resources on the market continue to expand, I would suspect that we could soon come to see the opposite of scalability becoming more important - getting more done with less space and energy.

If you think the solution to scaling issues is to 'throw more silicone' at the problem,

I have no idea how you could have possibly derived that from what I said. Indeed I was saying the opposite of that - I was questioning whether scaling itself is relevant.

you are about the worse programmer in the universe

I am not a programmer by profession, nor do I play one on TV.

Re:How Marketable Will That Skill Be? (1)

ClosedSource (238333) | more than 4 years ago | (#31026534)

Remember Moore's law is about number of transistors. Historically there was a correlation between Moore's law and processor performance. A mulitcore approach breaks that correlation.

Re:How Marketable Will That Skill Be? (1)

khallow (566160) | more than 4 years ago | (#31026620)

I'm not sure that scalability will be all that relevant as long as Moore's Law continues to hold.

If that were going to be true, it would have happened long ago. Remember the traffic for a popular website can double a lot faster than once every two years.

Re:How Marketable Will That Skill Be? (0)

Anonymous Coward | more than 4 years ago | (#31026812)

As long as there is a limit on what a single item can do, there'll be the people whose needs go beyond that, and thus end up needing more than one of said item to handle their workload.

But sure, when the day comes that we have a thing that'll let us transfer unlimited bits over an unlimited distance and use unlimited cpu cycles and has unlimited memory and unlimited storage all at once, then yes, you're right, scaling won't be necessary. But until then?

Re:How Marketable Will That Skill Be? (1)

liquiddark (719647) | more than 4 years ago | (#31027412)

Moore's Law 1) isn't about speed in particular and 2) may or may not hold for much longer. It was only predicted to last 10-20 years, and that was almost 50 years ago. We have about 5 halvings to go before we've saturated silicon at the atomic level, and although there are other techniques for preserving the growth curve, we're far enough away from Grand Computing Goals that it would seem likely we're going to see a dramatic slowing in the growth of computer power per dollar spent in the reasonably near future.

It's definitely well past time that anyone who wants to work in IT start concerning themselves with scalable solutions. Relying on morebetterfastercheaper hardware stopped being wise about the same time processor clock speeds stopped rising, which was several years ago.

First two sections (3, Interesting)

Curlsman (1041022) | more than 4 years ago | (#31025124)

are not going to happen in large organizations.
Nobody ever knows everything about the strategy or organizational structure, and the people and managers change places and responsibilities faster than any project can proceed.
And continuity, crisis and incident management are only longed for when the systems are down and eveybody is expected to work continuously until they're fixed, but never when it's time to pay for it.

Conway's law (4, Funny)

gmuslera (3436) | more than 4 years ago | (#31025356)

In any organization, there will always be one person who knows what's going on. That person must be fired

Re:First two sections (1)

nullchar (446050) | more than 4 years ago | (#31027484)

First two sections are not going to happen in large organizations.

What you say is sad but true in my experience at the companies I have worked for.

The first two sections are essentially: (1) the business strategy for your particular product, and knowing your human resources (team); (2) /really/ thinking about the future, not just how you will sell it; why you would need to scale.

The following question is somewhat on topic as this book appears to be targeted to decision makers at the company, which usually means high-level product/project managers and "directors" or "VPs" of technology/software engineering.

Question: Why do company decision makers not rely on pooling the collective input of their subordinates or teams, and instead make rapid and unfaltering business and technical decisions that are not forward-thinking?

Perhaps they are marking wise decisions for the company, so they can make money in the short term and grow the business, which allows them to fund new projects and so on. But I read on Slashdot all too often where management's short-sightedness to choose one specific technology, seemingly without consent of other knowledgeable employees (yet lower on the decision chain), cause headaches for the actual implementors down the chain.

Say these decision makers dictate no application caching and choose a commercial database offering for vertical performance scaling, instead of adding caching and partitioning data on free databases for horizontal scaling. Yet if they would have asked the collective software engineers and low-level management for input, the decision would have been reversed. Will a book like this even help those types of high-level decision makers, say VPs of Tech: brought in from outside the company to grow the business away from the previous [multiple] VPs' mistakes?

Eerie (1, Offtopic)

fulldecent (598482) | more than 4 years ago | (#31025206)

This reminds me of the original Zelda where you go in the cave and pay money...

Gates: Give us all your personal information and we will use this to make recommendations for you
User: /me gives
Gates: We recommend that you be less gullible

Try again (1)

rockNme2349 (1414329) | more than 4 years ago | (#31025256)

ERRRRRR

Sorry buddy, better luck next time. Wrong thread.

Re:Eerie (1)

getSalled (1331585) | more than 4 years ago | (#31025508)

Talk about the metaphor of all metaphors....

Warning (4, Insightful)

bjourne (1034822) | more than 4 years ago | (#31026178)

Unless you are actually working on a web site that needs to be scalable, thinking about scalability is both a waste of time and harmful. A totally unscalabe web app where each page view takes 1 second, can sustain 86k page hits per day or 2.5 million page hits per month. Will your web site see that much traffic? Really? If not, your main objective should be to get the site out of the door as soon as possible to keep development costs down. Hopefully, users flock to your site, melting the servers under huge loads of traffic. Good for you, you'll have a very profitable job designing the next scalable version of the site.

Or you get no traffic because the site idea was stupid to begin with. That sucks, but much less so than if you had spent months of effort making it scalable. "Scalable architecture" is merely a different form of premature optimization, and it is just as harmful.

Re:Warning (0)

Anonymous Coward | more than 4 years ago | (#31026640)

"A totally unscalabe web app where each page view takes 1 second, can sustain 86k page hits per day or 2.5 million page hits per month."

Your simplified model only holds true if all of those one second requests appear strictly one after the other. If all your eighty six thousand requests land over lunchtime, you might need to think about scalability.

Re:Warning (3, Insightful)

godrik (1287354) | more than 4 years ago | (#31027030)

"Your simplified model only holds true if all of those one second requests appear strictly one after the other. If all your eighty six thousand requests land over lunchtime, you might need to think about scalability."
A fairly good definition of "being slashdotted"

Re:Warning (1)

Firehed (942385) | more than 4 years ago | (#31030924)

Talk about missing the point.

In the tech world (and even more so in the land of Web 2.0 and web apps), getting stuff out the door quickly and improving it is way more important than having it work perfectly from day one. Because if the idea is any good, someone else is also working on it, and the first one out the door tends to win. It's widely acknowledged that with web startups, if you want to take on an existing market, you need to be 10x better than your competition. So if you're first to market with something decent, the second guy with a better, faster, sleeker product is going to still lose to you unless his product is WAY better than yours.

Scalability only matters if you have lots of users. Not scaling well is a good problem to have, because it means you have lots of users. And no, getting slashdotted over lunchtime does not qualify as a scalability issue (that typically qualifies as a server misconfiguration or really bad code issue). It sucks for a couple of hours, but you don't get slashdotted every day. If you ARE seeing that kind of traffic burst every day, then congratulations - you have users!

Re:Warning (2, Insightful)

BitZtream (692029) | more than 4 years ago | (#31027002)

Wrong, because 80k of those hits actually come during a 8-12 portion of the day, sometimes much smaller depending on the focus of the site.

Yes, its fine if you're never going to get anywhere near that kind of load, but load planning doesn't work in 'days' it works in hourly peaks and valleys if you want to look at it from a real rough perspective.

Ask some sites that get slashdotted how well capacity planning works when you get 120k hits in the first hour its on slashdot, and 10k for the rest of the day after it has moved down the list on the main page. Sure they could have easily handled the load with a little delay if they had built an app that loads in 1 second, IF the load was perfectly timed to send one request per second, which simply isn't the way it works.

Re:Warning (1)

bjourne (1034822) | more than 4 years ago | (#31032822)

I should have known slashdotters always have trouble getting the point of simplified calculating examples. To put it another way, a simple, shitty $20/month VPS with a standard LAMP stack can easily handle 4 hits/second, which translates to 14k hits/hour.

There are millions of sites on the internet, thousands are added each day. The chance of your site getting on slashdot is extremely small. You'd be a fool to waste time, money and effort planning for that.

throughput vs latency (2, Informative)

Colin Smith (2679) | more than 4 years ago | (#31027016)

A totally unscalabe web app where each page view takes 1 second

That's latency, not throughput. Scalability is throughput. Latency is how shitty it is to use.

I can scale your 1 second web app by adding 1000 processes per server and 1000 servers. That's a million page views per second throughput. But it'll still be a 1 second latency piece of shit application.

 

Re:throughput vs latency (1)

emurphy42 (631808) | more than 4 years ago | (#31029436)

Latency is how shitty it is to use.

IINM latency is how bad it starts out, throughput is how many simultaneous users it can handle before it gets worse.

Re:throughput vs latency (1)

kisanth88 (593283) | more than 4 years ago | (#31030082)

Latency, Throughput and NO NO NO NO. Are you guys on crack? Latency is network based. Host to host. It takes 45ms for my packet to get from my computer to your computer. Throughput is network based. Host to host. I can achieve from my computer to your computer 1000 kilobits per second in throughput. Server level "latency" is some other crap, I'm sorry but us networkers REFUSE to accept your junky server problems as either latency or throughput, find new words server monkeys! Make em up! Here are some suggestions, processor bound response time, PBRT (latency for server geeks), processor bound data transfer PBDT (throughput for server geeks). Damn it server monkeys, get off my network! And stop blaming it for your latency and throughput whines! *Shakes cane* JB

Re:throughput vs latency (1)

Firehed (942385) | more than 4 years ago | (#31030954)

He's obviously talking about the page rendering time, not hosting a fast app from a dial-up connection. We're not serving static pages anymore (and if it takes a full second to serve a static page, you're doing something seriously wrong).

Re:Warning (3, Interesting)

Jeppe Salvesen (101622) | more than 4 years ago | (#31027076)

Once you know how, making the solution fairly scalable is usually not a whole lot of extra work. Furthermore, if you don't bother learning how to do stuff right, you'll continue to make subpar products until you get your shot and then you waste that shot because your solution wasn't up to the task. Ahh.

Re:Warning (1)

bjourne (1034822) | more than 4 years ago | (#31032866)

Yes it is, and that is why books like "The Art of Scalability" are written. For proof, take a look at Google App Engine [google.com] and try and implement a large application on top of it. It's a lot of fun to play with, but no one who has used it can deny that it takes a lot more effort than a plain old "unscalable" LAMP app.

Re:Warning (1)

gbjbaanb (229885) | more than 4 years ago | (#31027656)

Unless you are actually working on a web site that needs to be scalable

I'm sure they wrote Facebook thinking they'd only get a few hundred hits per month... few years and several billion dollars later, guess what - they're writing PHP compilers to fix the problems with the initial design.

Take a little time to make yourself a better developer, learn the necessary knowledge and apply it to all you do, and you'll be known as the guy who writes good systems. Keep thinking you can just hack it all together and you'll be the guy who wrote those shitty systems that need to be replaced. I know which guy I'd prefer to be. The biggest thing is - you don't need to spend more time at all on designing the better system once you've become good, doing a good job will become second nature.

Re:Warning (1)

Firehed (942385) | more than 4 years ago | (#31031040)

When your website has 350M+ users and 150M+ uniques/day, you're well past the point of traditional scaling solutions. If Facebook was a country, it would be the third largest in the world, so to say the amount of data they have to deal with is monolithic is a terrific understatement. At this point, I'm surprised they're not getting custom silicon; custom compilers seems like they should have come a hundred million users ago.

Don't get me wrong - I'm sure Facebook's original code was awful. Do you think they're still using it? Do you think they haven't hand-tuned every line of code on the site to save themselves entire datacenters worth of servers? No, I'm NOT advocating writing terrible code - but some scaling techniques simply aren't practical when a site is small, and may simply not work at all, and the best code in the world that's made for a single-server site just won't work when you need the raw horsepower of an entire rack of servers.

Re:Warning (0)

Jay L (74152) | more than 4 years ago | (#31028122)

Unless you are actually a contender for the Olympics, thinking about being stronger and faster is both a waste of time and harmful. Will you actually be a finalist? Really? If not, your main objective should be to fit out the door. Hopefully, sponsors flock to you. Good for you, you'll have your work cut out for you training in the pre-season.

Or you watch the Olympics on TV like the rest of us. That sucks, but much less so than if you had spent months of effort cutting your times by a hundredth of a second. "Training" is merely a different form of premature optimization, and it is just as harmful.

Re:Warning (0)

Anonymous Coward | more than 4 years ago | (#31028434)

Well, while I do agree with you, there are little things that you can do to make it *easier* to scale later. For example, you can design your architecture around lots of small non-coupled components---so if you ever need to, you can run them in parallel, etc.

Martijn (1)

dingen (958134) | more than 4 years ago | (#31026370)

Now there's a cool name.

Re:Martijn (1)

tommeke100 (755660) | more than 4 years ago | (#31027552)

Martijn is a very common Dutch name, but then again, "dingen" means "stuff" in Dutch as well

Re:Martijn (1)

dingen (958134) | more than 4 years ago | (#31028352)

Oh really?

Re:Martijn (1)

sexybiggetje (1664029) | more than 4 years ago | (#31032170)

It actually means "stuff" but it's close :).

Re:Martijn (1)

sexybiggetje (1664029) | more than 4 years ago | (#31032176)

It actually means "stuff" but it's close :).

Ofcourse I meant "things", gah! posting this early on a friday when the coffee still is boiling..

Scalable? (1)

frank_adrian314159 (469671) | more than 4 years ago | (#31026622)

Creating high performance growing networks is really a special skill managers and network architects should posses to be ready for the future.

Most software/network designers will never work on any project that has so much traffic that it needs to scale to the levels discussed in this book. In fact, most (note I say most) businesses don't actually need the amount of scalability currently built into their current out-of-the-box hardware and software solutions. Just saying...

"This book is much more than you may think it is" (1)

weston (16146) | more than 4 years ago | (#31027908)

"This book is much more than you may think it is. Scale is not just about designing Web sites that don't crash when lots of users show up. It is about designing your company so that it doesn't crash when your business needs to grow. These guys have been there on the front lines of some of the most successful Internet companies of our time, and they share the good, the bad, and the ugly about how to not just survive, but thrive." -- Marty Cagan, Founder, Silicon Valley Product Group

From the somehow omitted site for the book [theartofscalability.com] .

(Also... is there some kind of Martin conspiracy going on here? The author is named Martin, the reviewer is named Martijn, the guy I just quoted is presumably a Martin...)

Book cover (0)

Anonymous Coward | more than 4 years ago | (#31029460)

Ok, let me get this straight. You wrote a book about scalability, and for the cover image you picked... a bonsai? Really?

Does that mean that all of the book's advice boils down to a paraphrasing of "If you want your tree to grow large, then stop cutting off its roots, asshole!"

Re:Book cover... Copycat (1)

mrjb (547783) | more than 4 years ago | (#31034444)

There's nothing wrong with having a bonsai tree on the cover of a book about software development. But he's still a copycat ;)
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?