How Far Can Large Commercial Applications Scale?
In my experience, the custom applications I deal with seem to be built with not just incorrect assumptions regarding load, but *no* assumptions regarding load. When I first fired up one particular application in a production environment, we were seeing 6000 incoming messages per second. I asked the lead developer what we should be expecting to see. He had no idea.
This is caused by short sighted project management, which translates into short sighted programming. The necessary questions about throughput aren't asked, because it all works fine on the developers' PC with a test load. In our case, we eventually got the application running OK, but changes that have been made since have not taken into account anything to do with I/O, so the fact that our CPU usage is not maxing out seems to indicate to the development team that we are not bound by the server performance, and hence have not reached any scalability thresholds.
Obviously this is madness. If one was to investigate the scalability of this application properly, one should be looking at where I/O happens, where interprocess communication happens, where object creation and destruction happens, and so on... There is no other way to scale an application -- you have to define what the "load" is, find what happens when you increase it, work out where any bottleneck is, and how parallelisable this bottleneck is. Anything less is no more than buzzwords.