×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

524 comments

After a long night of coding once (2, Funny)

teamhasnoi (554944) | more than 11 years ago | (#4668336)

I looked down in suprise to find a leaky abstraction. Had to change pants.

Re:After a long night of coding once (1, Offtopic)

JediTrainer (314273) | more than 11 years ago | (#4668374)

I looked down in suprise to find a leaky abstraction. Had to change pants.

Stop looking at porn sites, and you might be able to avoid that problem.

Re:After a long night of coding once (3, Funny)

ERJ (600451) | more than 11 years ago | (#4668430)

+3 Informative? Trust me, this is information that I don't need...

Re:After a long night of coding once (0, Offtopic)

ERJ (600451) | more than 11 years ago | (#4668452)

And, of course, as soon as my first post got sinked it became +4 funny....

The Fall of Christina (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668343)

The release of Christina Aguilera's new album "Stripped" has brought about a good deal of controversy, and with good reason as Aguilera's overnight transformation is an understandable shock to those who have extensively followed the work of this emerging artist.
I, personally, am extremely disappointed with Christina's departure from her pronounced classical music roots. As a young, underground artist, Ms. Aguilera created a very progressive, yet accessible sound, which will undoubtedly be heralded for many years to come. Aguilera's self-titled first endeavor was a musical Odyssey that exhibited Christina's classical groundings, but also exuded Christina's inured pop instinct that is paramount to divas none other than Tina Turner and Aretha Franklin.
"Stripped", however, represents little more than the commoditization of this young talent. In an effort to make her work more accessible to the mainstream audience, "Stripped" asserts a water-downed, musically inhibited Christina that abandons much of the technical prowess that defined her earlier compositions.
It is extremely depressing that an artist with the talent and aesthetic appeal to propel the music industry into a new, musically sensible era past all of the packaged music groups with their plastic bubble gum hits. Unfortunately, Ms. Aguilera has just joined the pack and the loss of her potential is, undoubtedly, one of the greatest musical tragedies of our time.

Michael, the Slashdot squatter. (-1, Flamebait)

Anonymous Coward | more than 11 years ago | (#4668457)

Really, think about it: Slashdot editor Michael, a defender of people's rights online, stoops as low [censorware.org] as people who run www.whitehouse.com [whitehouse.com]. That, my friends is the true definition of hypocrisy.

Michael: Stop you ridiculous squatting. You can keep you immature stalkedbyseth.com [stalkedbyseth.com] -domain. We know you don't get any hits because of that domain. The reason your page gets any hits at all is because you squat on the domain censorware.org [censorware.org].

Abstraction: (3, Funny)

ewithrow (409712) | more than 11 years ago | (#4668348)

Because the first step to solving any problem is always to create more problems.

-E

even for non-programmers (3, Funny)

Anonymous Coward | more than 11 years ago | (#4668354)

Great! I'll print off a hardcopy and stick it on my refrigerator! I'm sure my wife will love it!

Re:even for non-programmers (0)

Anonymous Coward | more than 11 years ago | (#4668359)

Good one!

Re:even for non-programmers (0)

Anonymous Coward | more than 11 years ago | (#4668390)

Great! I'll print off a hardcopy and stick it on my refrigerator! I'm sure my wife will love it!

Bob Saget called:

Even he doesn't think you're funny.

Re:even for non-programmers (4, Interesting)

Nexus7 (2919) | more than 11 years ago | (#4668508)

I guess a "liberal arts" major would be considered the quintessential "non-programmer". Certainly these people profess a non-concern for most technology, and of course, computing. I don't mean that they wouldn't know about Macs and PCs and Word, but we can agree that is a very superficial view of computing. But appreciating an article such as this "leaky abstractions" required some understanding of the way the networks work, even if there isn't any heavy math in it. In other words, the non-programmer wouldn't understand what the fuss is about.

But that isn't how it's supposed to be. Liberal arts people are supposed to be interested in precisely this kind of thing, because it takes a higher level view of something that is usually presented in a way that only a CS major would find interesting or useful, and generalizes an idea to be applicable beyond the specific subject, networking.

That is, engineers are today's liberal arts majors. It's time to get the so called "liberal arts" people out from politics, humanities, governance, management and other fields of importance because they just aren't trained to have or look for the conceptual basis of decision making and correctly apply it.

Re:even for non-programmers (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668644)

Nice troll.

got modded up =)

timeout (1, Informative)

czaby (93380) | more than 11 years ago | (#4668365)

There is no magic. TCP is reliable even using IP because there are timeouts. If your data cannot be transmitted for a certain time limit, you will get timeout wich is perfectly legal result.

Re:timeout (3, Insightful)

binaryDigit (557647) | more than 11 years ago | (#4668438)

Well I wouldn't say that it's reliable "because there are timeouts". AAMOF, timeouts just compicate things. So you timeout waiting for packet N, you request a resend of it, and in the interim, guess what, packet N shows up, now you have two N's. Your code is now more complex in having to deal with this situation. Timeouts are just another parameter used adjust the behaviour of the algorithms that control the protocol. Getting deterministic results from an undeterministic foundation involves making observations, accepting some compromises, making some simplifying assumptions, and then writing code that takes all those things into account to come up with something that usually works.

TCP for the bored (5, Insightful)

mekkab (133181) | more than 11 years ago | (#4668606)

FIne it's relaibale becasue of acks, timeouts, adaptive re-transmit timeouts that take statistical averages of RTT times, exponential back-off and slow start, window acks which keep track of what bytes are received, etc.

So in your case of timing out N, re-tx'ing N, and then getting the repsonse to the first N back after sending the second N, you do two things:
1) Good! You got yr packet!
2) keep track of how many bytes you have received thsu far (TCP is not sending messages, it is sending a stream)
3) when you get the response from your second request, discard it, becuase you already received those bytes from the stream.
4) since you timed out, DON'T use the Round TRip Time for that reponse: slow down your expected RTT time, and THEN start measuring.

And guess what? If I unplug the NIC of the other machine, there is no reliable way of transmitting that data (assuming your destination machine isn't dual homed)- so I keep streaming bytes to a TCP socket and I don't find out my peer is gone for approx. 2 minutes.
WOW. There's nothing reliable about that boundary condition!

my point is TCP is reliable ENOUGH. But I wouldn't equate it with a Maytag warranty. It is not a panacea. Infact, for a closed homogenous network I wouldn't even consider it the best option. But if the boundary conditions fall within the acceptible fudge range (remember Real Time human grade systems are not 100% reliable, only 99.99999% and much of that is achieved through redundancy) your leaks are ok.

Re:timeout (2)

Isle (95215) | more than 11 years ago | (#4668652)

I dont think you understand TCP, you dont request a resend. TCP does that for you, if you timeout it means the connection is broken. Feel free to try again later, but trying again means opening a new connection, and no packages from an old connection will confuse the new one.

Re:timeout (4, Informative)

Minna Kirai (624281) | more than 11 years ago | (#4668496)

"Reliable" means "always works", it doesn't mean "always obeys the spec". (Unless you use a circular definition)

A timeout is a legal result by the TCP specification, but it's not reliable, because your data didn't make it through.

By the IP specification, your data might not make it either- and that's a legal result because the spec allows it to drop packets for any reason at all. That doesn't mean IP is reliable, just that it obeys its own definition.

Of course, no real protocol can ever meet this restrictive definition of reliable. Some maniac can always cut through your wires or incinerate your CPUs. Calling TCP a "reliable protocol" is just a shorthand for "as much more reliable than the underlying protocols as we could manage"

The timeout you mention does make TCP more reliable than IP, because it alerts you to the data loss, where the application can possibly take steps to retransmit it sometime in the future.
But its not as if TCP could ever achieve the perfect reliablity that the simplest, most abstract description of it would imply. Which is why, as the author says, those who rely on the abstractions can get bitten later.

Re:timeout (1)

Jerdie (516662) | more than 11 years ago | (#4668686)

I would think that reliable would mean, if it says it gets there, it got there, and if says it didn't, then it didn't. I don't believe that reliable means "always works", it means that it does what it says it will do.

Re:timeout (2)

Isle (95215) | more than 11 years ago | (#4668716)

No calling TCP reliable, means that it provides certain features you can rely on:
1. Data either arrives intact or dont arrive at all.
2. Data always arrives in the order it was sent
3. Data is never duplicated.

You can safely rely on all of these facts. Calling TCP a garantie that you packages arrive is leaky, but TCP doesnt claim that.

So if you dont abstract TCP more than you are meant to, it is a non-leaky abstraction.

Informative (5, Insightful)

Omkar (618823) | more than 11 years ago | (#4668375)

Although I used to program as a hobby, my eyes bugged out when I saw this article. It's actually quite interesting; I finally realize why the hell people program in lower level languages.

One point that I think could be addressed is backward compatibilty. I really know nothing about this, but don't the versions of the abstractions have to be fairly compatible with each other, especially on a large, distributed system? This extra abstraction of an abstraction has to be orders of magnitude more leaky. The best example I can think of is Windows.

Re:Informative (2, Informative)

Jamey12345 (41681) | more than 11 years ago | (#4668395)

Com and it's decendants are supposed to take care of this. In reality they work relativly well, but also lead to larger and larger libraries. The simple reason is because Com has to remain backwards compatable, by way of leaving in the old functions and methods.

Re:Informative (4, Interesting)

Bastian (66383) | more than 11 years ago | (#4668462)

I think backward(really more slantwise or sideways) compatibility is almost certainly one of the reasons behind why C++ treats string literals as arrays of characters.

I program in C++, but link to C libraries all the time. I also pass string literals into functions that have char* parameters. If C++ didn't treat string literals as char*, that would be impossible.

Re:Informative (5, Interesting)

binaryDigit (557647) | more than 11 years ago | (#4668524)

I think it's a mistake to simply say that "high level languages make for buggier/bloated code". After all, many abstractions are created to solve common problems. If you don't have a string class then you'll either roll your own or have code that is complex and bug prone from calling 6 different functions to append a string. I don't think anyone would agree that it's better to write your own line drawing algorithm and have to program directly to the video card, vs calling one OpenGL method to do the same (well unless you need the absolute last word in performance, but that's another topic).

Joel is ignoring market factors (2)

Ars-Fartsica (166957) | more than 11 years ago | (#4668597)

Yes it would be nice to get back to 'first principles' and address machine resources directly, but its impossible to deliver a product to the marketplace in a meaningul timeframe using this method, particularly when Moore's law blurs the gains anyway - crap runs fast enough.

The underlying problem with programming (5, Insightful)

Jack Wagner (444727) | more than 11 years ago | (#4668384)

I'm of the idea that the whole premise that high-level tools and high level abstraction coupled with encasulation are the biggest bane of the software industry. We have these high level tools which most programmers really don't understand and are taught that they don't need to understand in order to build these sophisticated products.

Yet, when something goes wrong with the underlying technology they are unable to properly fix their product because all they know is some basic java or VB and they don't understand anything about sockets or big-endian/little endian byte alignment issues. It's no wonder todays software is huge and slow and doesn't work as advertised.

The one shining example of this is FreeBSD, which is based totally on low level C programs and they stress using legacy program methodologies in place of the fancy schmancy new ones which are faulty. The proof is in the pudding, as they say, when you look at the speed and quality if FreeBSD, as opposed to some of the slow ponderous OS's like Windows XP or Mac OSX.

Warmest regards,
--Jack

Re:The underlying problem with programming (-1, Offtopic)

mickwd (196449) | more than 11 years ago | (#4668460)

"The one shining example of this is FreeBSD, which is based totally on low level C programs and they stress using legacy program methodologies in place of the fancy schmancy new ones which are faulty."

Lol, good trolling - I see you're still getting mod points, too. Some moderators will never learn.

Re:The underlying problem with programming (4, Insightful)

binaryDigit (557647) | more than 11 years ago | (#4668482)

Well I'd agree up to a point. The fact is that FreeBSD is trying to solve a different problem/attract a different audience than XP/OSX. If FreeBSD was forced to add all the "features" of the other two in an attempt to compete in that space, then it would suffer mightily. You also have to take into account the level/type of programmers working on the these projects. While FreeBSD might have a core group of seasoned programmers working on it, the other two have a great range of programming experience working on it. A few guys who know what they're doing working on a smaller featureset would always produce better stuff than a large group of loosely coupled and widely differing talents working on a monsterous feature set.

Re:The underlying problem with programming (5, Insightful)

jorleif (447241) | more than 11 years ago | (#4668579)

The real problem is not the existance of high-level abstractions, but the fact that many programmers are unwilling or unable to understand the abstraction.

So you say "let's get rid of encapsulation". But that doesn't solve this problem, because this problem is one of laziness or incompetence rather than not being allowed to touch what's inside the box. Encapsulation solves an entirely different problem, that is the one of modularity. If we abolish encapsulation the same clueless programmers will just produce code that is totally dependent on some obscure property in a specific version of a library. They still won't understand what the library does, so we're in a worse position than when we started.

Re:The underlying problem with programming (2)

Tom7 (102298) | more than 11 years ago | (#4668581)

I guess you are a troll (I hope you don't really believe what you're saying!!), but you're missing an important point: FreeBSD has security holes frequently found in it (ie, buffer overflows, heap overflows, format string attacks); bugs that would be impossible to make in a language like (say) Java. Security holes are the most salient example, but there are many perils to trying to do things "manually" and by programmer brute-force in a language like C.

Java's not my favorite language, but programs written in it tend to be more robust than their C counterparts.

Re:The underlying problem with programming (1)

Viol8 (599362) | more than 11 years ago | (#4668654)

I assume that by robust you're not taking the reliability of the VM into account , because reliable they aint. And they're slow , and they hog the CPU , and not all java code is portable, and Java is just C++ with some of its powerful features removed (though they're slowly adding them back). Apart from that , yeah , Java is a great language compared to C/C++.

Re:The underlying problem with programming (1)

captainwacky (532611) | more than 11 years ago | (#4668637)

--The proof is in the pudding, as they say,

The proof of the pudding is in the eating and FreeBSD is finger-lickin' good (or so I've been told).

Time to market is the factor, not elegance (5, Insightful)

Ars-Fartsica (166957) | more than 11 years ago | (#4668647)

This argument is so tired. The downfall of programming is now due to people who can't/don't write C. Twenty years before that the downfall of programming was C programmers who couldn't/wouldn't write assembler.

The market rewards abstractions because they help create high level tools that get products on the market faster. Classic case in point is WordPerfect. They couldn't get their early assembler-based product out on a competitive schedule with Word or other C based programs.

Re:The underlying problem with programming (2)

jilles (20976) | more than 11 years ago | (#4668678)

The problem is that you need all these high level abstractions to reduce the workload of creating large systems. There's just no way you could have those VB monkeys be productive in C and there's just no way you are going to replace them with competent C programmers. Besides, competent programmers are more productive using high abstraction level tools.

BSD is just a kernel + a small toolset. As soon as you start running all the regular stuff on top of it performance is comparable to a full blown linux/mac os X/windows desktop. Proof: mac os X, remove the none BSD stuff and see what's left: no ui, no friendly tools, no easy access to all connected devices.

Re:The underlying problem with programming (2)

dfn5 (524972) | more than 11 years ago | (#4668707)

I'm of the idea that the whole premise that high-level tools and high level abstraction coupled with encasulation are the biggest bane of the software industry.

The problem with programming at the lower level, like Xlib [x.org], is that it takes 2 years to get the first version of your program out. Then you move on to Xt and now it only takes only 1 year. Then you move on to Motif [lesstif.org] and it only takes you 6 months. Then you move on to Qt [trolltech.com] and it only takes 3 hours. Of course you want it to look slick so you use kdelibs [kde.org].

The leaky abstraction of spam (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668391)

We use TCP for many things like fetching web pages and sending email. The reliability of TCP is why every exciting email from embezzling East Africans arrives in letter-perfect condition. O joy.
...
If you send a bunch of messages with IP, don't be surprised if only half of them arrive, and some of those are in a different order than the order in which they were sent, and some of them have been replaced by alternate messages, perhaps containing pictures of adorable baby orangutans, or more likely just a lot of unreadable garbage that looks like the subject line of Taiwanese spam.


Hmm, Mr Spolsky, you got mail.

Re:The leaky abstraction of spam (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668435)

"email from embezzling East Africans"

Except Nigeria is in West Africa.

Seriously though, is this article worth reading. I read a little and it was like "Badly explained programming metaphors for idiots.". Is there any point, or would I just be wasting my time? (I can code, by the way).

What? (1)

HogGeek (456673) | more than 11 years ago | (#4668426)

By comparison, there is another method of transmitting data called IP which is unreliable.

I thought that was called UDP? Which is also built on top of IP...

Re:What? (2)

Bastian (66383) | more than 11 years ago | (#4668511)

UDP is also unreliable, but UDP, like TCP, is built on top of IP. It doesn't really work for the "relabile protocol running over an unreliable protocol" example.

Re:What? (4, Insightful)

Minna Kirai (624281) | more than 11 years ago | (#4668533)

The author wanted to make the point that TCP (fairly reliable) is built on IP (absolutely unreliable).

UPD is also build on IP, but adds almost nothing to it (aside from the ability to target individual port numbers instead of an entire computer), so its not any more reliable than IP.

The internet's developers could've built TCP on top of UDP, but for unknown reasons they didn't. Maybe they felt that a whole extra layer of abstraction would've been useless overhead.

Great examples (5, Interesting)

Ratface (21117) | more than 11 years ago | (#4668427)

I loved the articel - great examples, especially this favourite bugbear of mine:

"Problem: the ASP.NET designers needed to hide the fact that in HTML, there's no way to submit a form from a hyperlink. They do this by generating a few lines of JavaScript and attaching an onclick handler to the hyperlink. The abstraction leaks, though. If the end-user has JavaScript disabled, the ASP.NET application doesn't work correctly, and if the programmer doesn't understand what ASP.NET was abstracting away, they simply won't have any clue what is wrong."

This is a perfect illustration of why I will never hire a web developer who cannot program HTML by hand. The kids who are going to university nowadays and building everything using Visual Studio or Dreamweaver will never be able to ensure that a site runs in an acceptable cross-platform way.

Another favourite bugbear that this example reminds me of is people building page layouts using 4 levels of embedded tables where they reallly only needed 1 or 2.

Great article!

Re:Great examples (1)

venomkid (624425) | more than 11 years ago | (#4668499)

The problem isn't HTML by hand, in this case, it's the fact that HTML itself has limitations which javascript is used to overcome. Which then your high level guy is going to know crap about.

I hate this strategy by the way, because it doesn't allow me to open new windows on links. All Music Guide [allmusic.com] recently switched to javascript links, and it's infuriating. Not to mention broken if JS is disabled.

But as far as those tables go, that's a design issue, again not html. I can do tighter tables in dreamweaver faster than any hand coder can, because I know the tool and how to do the tables minimally *and* visually.

I think what you really want is someone who's been doing HTML long enough to know how to use it efficiently, and if they know how to do it by hand, odds are that's true.

Re:Great examples (1, Informative)

Anonymous Coward | more than 11 years ago | (#4668517)

Just a note, some people who use Dreamweaver do actually understand HTML. For example, I use Dreamweaver, and I can state that one of my favorite bugbears is people building page layouts using tables, when they should be using divs, with CSS for the positioning.

Re:Great examples (2)

Bastian (66383) | more than 11 years ago | (#4668618)

And my favorite bugbear is people who don't realize that some of us use tables rather than divs and CSS because support for that stuff isn't so hot on a lot of browsers.

Re:Great examples (2)

tshak (173364) | more than 11 years ago | (#4668534)

I would just like to second this. Although I use VS.NET, I don't use it to build my HTML. Sure, I still allow ASP.NET to insert javascript (you can disable that), but I build Web Applications and a Web Application demands a UI that requires javascript to create. Nevertheless, I would never hire a dev who didn't understand the implications. We may need to one day create a simple web site in ASP.NET and I don't want to sit their explaining to him why a "LinkButton" won't render properly in all browsers.

Mirrored Here (2, Redundant)

CFN (114345) | more than 11 years ago | (#4668432)

It looked like that site was about to be /.ed, so I copied the text below.

By Joel Spolsky
November 11, 2002
Printer Friendly Version

There's a key piece of magic in the engineering of the Internet which you rely on every single day. It happens in the TCP protocol, one of the fundamental building blocks of the Internet.

TCP is a way to transmit data that is reliable. By this I mean: if you send a message over a network using TCP, it will arrive, and it won't be garbled or corrupted.

We use TCP for many things like fetching web pages and sending email. The reliability of TCP is why every exciting email from embezzling East Africans arrives in letter-perfect condition. O joy.

By comparison, there is another method of transmitting data called IP which is unreliable. Nobody promises that your data will arrive, and it might get messed up before it arrives. If you send a bunch of messages with IP, don't be surprised if only half of them arrive, and some of those are in a different order than the order in which they were sent, and some of them have been replaced by alternate messages, perhaps containing pictures of adorable baby orangutans, or more likely just a lot of unreadable garbage that looks like the subject line of Taiwanese spam.

Here's the magic part: TCP is built on top of IP. In other words, TCP is obliged to somehow send data reliably using only an unreliable tool.

To illustrate why this is magic, consider the following morally equivalent, though somewhat ludicrous, scenario from the real world.

Imagine that we had a way of sending actors from Broadway to Hollywood that involved putting them in cars and driving them across the country. Some of these cars crashed, killing the poor actors. Sometimes the actors got drunk on the way and shaved their heads or got nasal tattoos, thus becoming too ugly to work in Hollywood, and frequently the actors arrived in a different order than they had set out, because they all took different routes. Now imagine a new service called Hollywood Express, which delivered actors to Hollywood, guaranteeing that they would (a) arrive (b) in order (c) in perfect condition. The magic part is that Hollywood Express doesn't have any method of delivering the actors, other than the unreliable method of putting them in cars and driving them across the country. Hollywood Express works by checking that each actor arrives in perfect condition, and, if he doesn't, calling up the home office and requesting that the actor's identical twin be sent instead. If the actors arrive in the wrong order Hollywood Express rearranges them. If a large UFO on its way to Area 51 crashes on the highway in Nevada, rendering it impassable, all the actors that went that way are rerouted via Arizona and Hollywood Express doesn't even tell the movie directors in California what happened. To them, it just looks like the actors are arriving a little bit more slowly than usual, and they never even hear about the UFO crash.

That is, approximately, the magic of TCP. It is what computer scientists like to call an abstraction: a simplification of something much more complicated that is going on under the covers. As it turns out, a lot of computer programming consists of building abstractions. What is a string library? It's a way to pretend that computers can manipulate strings just as easily as they can manipulate numbers. What is a file system? It's a way to pretend that a hard drive isn't really a bunch of spinning magnetic platters that can store bits at certain locations, but rather a hierarchical system of folders-within-folders containing individual files that in turn consist of one or more strings of bytes.

Back to TCP. Earlier for the sake of simplicity I told a little fib, and some of you have steam coming out of your ears by now because this fib is driving you crazy. I said that TCP guarantees that your message will arrive. It doesn't, actually. If your pet snake has chewed through the network cable leading to your computer, and no IP packets can get through, then TCP can't do anything about it and your message doesn't arrive. If you were curt with the system administrators in your company and they punished you by plugging you into an overloaded hub, only some of your IP packets will get through, and TCP will work, but everything will be really slow.

This is what I call a leaky abstraction. TCP attempts to provide a complete abstraction of an underlying unreliable network, but sometimes, the network leaks through the abstraction and you feel the things that the abstraction can't quite protect you from. This is but one example of what I've dubbed the Law of Leaky Abstractions:

All non-trivial abstractions, to some degree, are leaky.

Abstractions fail. Sometimes a little, sometimes a lot. There's leakage. Things go wrong. It happens all over the place when you have abstractions. Here are some examples.

Something as simple as iterating over a large two-dimensional array can have radically different performance if you do it horizontally rather than vertically, depending on the "grain of the wood" -- one direction may result in vastly more page faults than the other direction, and page faults are slow. Even assembly programmers are supposed to be allowed to pretend that they have a big flat address space, but virtual memory means it's really just an abstraction, which leaks when there's a page fault and certain memory fetches take way more nanoseconds than other memory fetches.
The SQL language is meant to abstract away the procedural steps that are needed to query a database, instead allowing you to define merely what you want and let the database figure out the procedural steps to query it. But in some cases, certain SQL queries are thousands of times slower than other logically equivalent queries. A famous example of this is that some SQL servers are dramatically faster if you specify "where a=b and b=c and a=c" than if you only specify "where a=b and b=c" even though the result set is the same. You're not supposed to have to care about the procedure, only the specification. But sometimes the abstraction leaks and causes horrible performance and you have to break out the query plan analyzer and study what it did wrong, and figure out how to make your query run faster.
Even though network libraries like NFS and SMB let you treat files on remote machines "as if" they were local, sometimes the connection becomes very slow or goes down, and the file stops acting like it was local, and as a programmer you have to write code to deal with this. The abstraction of "remote file is the same as local file" leaks. Here's a concrete example for Unix sysadmins. If you put users' home directories on NFS-mounted drives (one abstraction), and your users create .forward files to forward all their email somewhere else (another abstraction), and the NFS server goes down while new email is arriving, the messages will not be forwarded because the .forward file will not be found. The leak in the abstraction actually caused a few messages to be dropped on the floor.
C++ string classes are supposed to let you pretend that strings are first-class data. They try to abstract away the fact that strings are hard and let you act as if they were as easy as integers. Almost all C++ string classes overload the + operator so you can write s + "bar" to concatenate. But you know what? No matter how hard they try, there is no C++ string class on Earth that will let you type "foo" + "bar", because string literals in C++ are always char*'s, never strings. The abstraction has sprung a leak that the language doesn't let you plug. (Amusingly, the history of the evolution of C++ over time can be described as a history of trying to plug the leaks in the string abstraction. Why they couldn't just add a native string class to the language itself eludes me at the moment.)
And you can't drive as fast when it's raining, even though your car has windshield wipers and headlights and a roof and a heater, all of which protect you from caring about the fact that it's raining (they abstract away the weather), but lo, you have to worry about hydroplaning (or aquaplaning in England) and sometimes the rain is so strong you can't see very far ahead so you go slower in the rain, because the weather can never be completely abstracted away, because of the law of leaky abstractions.
One reason the law of leaky abstractions is problematic is that it means that abstractions do not really simplify our lives as much as they were meant to. When I'm training someone to be a C++ programmer, it would be nice if I never had to teach them about char*'s and pointer arithmetic. It would be nice if I could go straight to STL strings. But one day they'll write the code "foo" + "bar", and truly bizarre things will happen, and then I'll have to stop and teach them all about char*'s anyway. Or one day they'll be trying to call a Windows API function that is documented as having an OUT LPTSTR argument and they won't be able to understand how to call it until they learn about char*'s, and pointers, and Unicode, and wchar_t's, and the TCHAR header files, and all that stuff that leaks up.

In teaching someone about COM programming, it would be nice if I could just teach them how to use the Visual Studio wizards and all the code generation features, but if anything goes wrong, they will not have the vaguest idea what happened or how to debug it and recover from it. I'm going to have to teach them all about IUnknown and CLSIDs and ProgIDS and ... oh, the humanity!

In teaching someone about ASP.NET programming, it would be nice if I could just teach them that they can double-click on things and then write code that runs on the server when the user clicks on those things. Indeed ASP.NET abstracts away the difference between writing the HTML code to handle clicking on a hyperlink () and the code to handle clicking on a button. Problem: the ASP.NET designers needed to hide the fact that in HTML, there's no way to submit a form from a hyperlink. They do this by generating a few lines of JavaScript and attaching an onclick handler to the hyperlink. The abstraction leaks, though. If the end-user has JavaScript disabled, the ASP.NET application doesn't work correctly, and if the programmer doesn't understand what ASP.NET was abstracting away, they simply won't have any clue what is wrong.

The law of leaky abstractions means that whenever somebody comes up with a wizzy new code-generation tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying "learn how to do it manually first, then use the wizzy tool to save time." Code generation tools which pretend to abstract out something, like all abstractions, leak, and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting. So the abstractions save us time working, but they don't save us time learning.

And all this means that paradoxically, even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.

During my first Microsoft internship, I wrote string libraries to run on the Macintosh. A typical assignment: write a version of strcat that returns a pointer to the end of the new string. A few lines of C code. Everything I did was right from K&R -- one thin book about the C programming language.

Today, to work on CityDesk, I need to know Visual Basic, COM, ATL, C++, InnoSetup, Internet Explorer internals, regular expressions, DOM, HTML, CSS, and XML. All high level tools compared to the old K&R stuff, but I still have to know the K&R stuff or I'm toast.

Ten years ago, we might have imagined that new programming paradigms would have made programming easier by now. Indeed, the abstractions we've created over the years do allow us to deal with new orders of complexity in software development that we didn't have to deal with ten or fifteen years ago, like GUI programming and network programming. And while these great tools, like modern OO forms-based languages, let us get a lot of work done incredibly quickly, suddenly one day we need to figure out a problem where the abstraction leaked, and it takes 2 weeks. And when you need to hire a programmer to do mostly VB programming, it's not good enough to hire a VB programmer, because they will get completely stuck in tar every time the VB abstraction leaks.

The Law of Leaky Abstractions is dragging us down.

My company, Fog Creek Software, has just released FogBUGZ 3.0, the latest version of our innovative system for managing the software development process. Check it out now!

MOD PARENT DOWN (-1, Troll)

Anonymous Coward | more than 11 years ago | (#4668453)

Mirror another site, or like to the document, dammit!

Re:Mirrored Here (1)

Viol8 (599362) | more than 11 years ago | (#4668694)

"Sometimes the actors got drunk on the way and shaved their heads or got nasal tattoos, thus becoming too ugly to work in Hollywood" ?? How the hell did Vin Diesel get a job there then? :)

Really? (2, Informative)

CowboyMeal (614487) | more than 11 years ago | (#4668434)

Here's the magic part: TCP is built on top of IP. In other words, TCP is obliged to somehow send data reliably using only an unreliable tool.

Seems to have worked pretty well so far...

Give me a break. (0)

matp (42758) | more than 11 years ago | (#4668450)

Hey, and we'll all be better off if we only invest in your company's software tool.

Been around the blogsphere for days (1, Insightful)

Anonymous Coward | more than 11 years ago | (#4668454)

This is old news to bloggers, everyone and their mother has been linking to this article for days, although I don't find it terribly insightful.
An abstraction by definition is not 100% fidelistic to the source. All abstractions, like any technology, simultaneously extend and constrain certain aspects of experience. A car extends our range of transportation, but simultaneously constrains our other abilities while driving, hence so many accidents when not completely focused on the road. The only question is, is the abstraction worth the price of its leaks? Most of the time the answer is yes.

Good Points.... (3, Insightful)

Cap'n Canuck (622106) | more than 11 years ago | (#4668461)

This is a well written article that explains many things!

The more layers, the slower things get.
As computers speed up, it allows more feasible abstraction layers and next generation languages.

Optimizing routines is still important.
This holds true for ALL layers, as users are always expecting faster and faster apps.

It's still important to know all the layers
This allows old-timers to look down their noses at those whippersnappers.

Re:Good Points.... (2, Funny)

Anonymous Cow Hoard (611504) | more than 11 years ago | (#4668562)

This allows old-timers to look down their noses at those whippersnappers.

And my first-year computer scientist coworker wonders why I think VB and .NET development tools are the work of the devil :P When they break (note the "when" not "if"), I and my boss are the only ones that can figure out why because we're used to C, C++, and all the nifty "old" languages that make you abstract on your own, gosh durn it! Now, where's my medication...?

Re:Good Points.... (2, Funny)

Dimitri-san (169146) | more than 11 years ago | (#4668640)

The more layers, the slower things get.

You can tell I'm tired. I read that line like this:

The more lawyers, the slower things get.

Well, it's true...

Re:Good Points.... (2)

Codex The Sloth (93427) | more than 11 years ago | (#4668713)

It's still important to know all the layers
This allows old-timers to look down their noses at those whippersnappers.


Uhhh yeah, right. What are you doing to do when it doesn't work as it should. You'll have to go get dad to help you put it together.

This all reminds me of the Star Trek episode where people have become so reliant on machines that they are totally helpless when they break.

Old News ...and Not that Interesting (1, Troll)

dbretton (242493) | more than 11 years ago | (#4668467)

1. This came out about a week ago.

2. Basically, what Joel professes here is that non-strawman abstractions contain some sort of fallacy in their implementation that precludes you from the joyful ignorance of implementation details, such as array dereferencing, strings, etc.
In all seriousness, this is really nothing new, and should be more than apparent to all.

I was quite surprised at this article, as usually I find Joel's postings and ramblings to be rather insightful. Perhaps it's been a slow month. :)

Slashdot Editors Are Idiots (Again) (1, Offtopic)

Lethyos (408045) | more than 11 years ago | (#4668468)

I figured I'd share this with you guys, regardless of what /.'s idiot editor(s) think(s).

2002-11-14 15:19:42 Bill To Jail Hackers For Life Is Being Well Suppor (yro,news) (rejected)

This article [com.com] talks about Cyber Security Enhancement Act [loc.gov] (heh) that seeks life-time prison sentences for "hackers". There's a good summary at the end of the article. The bill also includes extended wire-tapping without court authorization and other abominations.

Something important for us all to be aware of.

YES THEY ARE! POST THIS ON FRONT PAGE! (1)

dbretton (242493) | more than 11 years ago | (#4668494)

This is the *important* stuff, not abstractions on diarrhetics.
I cannot believe that a bill like this would pass!

Re:Slashdot Editors Are Idiots (Again) (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668528)

Oh, you mean like this [slashdot.org].

Re:Slashdot Editors Are Idiots (Again) (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668537)

Read the article!

<<Specif(ies) that an existing ban on the "advertisement" of any device that is used primarily for surreptitious electronic surveillance applies to online ads.>>

If this passes, all those X10 camera ads would be illegal!

Hell, I'm emailing my congressmonkey right now!

Re:Slashdot Editors Are Idiots (Again) (-1, Offtopic)

Anonymous Coward | more than 11 years ago | (#4668658)

dear god, that's awful. life sentences for hacking in a manner that recklessly puts lives at risk? why that's... well, it's actually pretty fair.

This bill is a good thing.. It allows for appropriate punishment for people who try to use their computer to kill people. It lets authorities track an IP address related to an ongoing attack without waiting for a court order. It also allows ISPs to divulge user behaviour, if that behaviour might physically kill or maim somebody. It creates a new office of Science and Technology to do some useful research on things like personalized firearms, and a bunch of other good stuff.

Honestly, have you even read the law that you're complaining about? The only reason it could be considered news for nerds is that we should write our legislators to express our support for the bill.

Good article - how to solve? (5, Insightful)

cjustus (601772) | more than 11 years ago | (#4668475)

This article does a great job of describing problems in development environments today... But how do we solve it?

Hire VB programmers with assembly language experience that are network admins and are familiar with assembly language? No - the solution is not to hire people just with skill XYZ, but to hire people that are capable of thinking for themselves, doing research, problem solving, and RTFM...

It's a shame that so many companies hiring today are looking for skill X, Y, and Z... so some moron with X, Y, and Z on his resume gets hired, while someone that knows X - could learn Y and Z, and could outperform the moron, gets overlooked...

Yet I see it happen at my IT Staffing company almost every day...

Re:Good article - how to solve? (1)

xutopia (469129) | more than 11 years ago | (#4668540)

That reminds me of some company asking for someone with 25 years of experience in Java. I answered I was only 24 but danced Java and the mumbo ever since I was 2! :) Morons.

Re:Good article - how to solve? (1)

ovapositor (79434) | more than 11 years ago | (#4668574)

Amen brother!
That is exactly the problem I am having now. I have an MSEE and 10 years or programming and management experience. Can't even get an interview!

Too bad I concentrated on telecom. I will say that the good years were pretty good. I can't wait for the pendulum to swing back over to my side again.

Re:Good article - how to solve? (1)

broken_bones (307900) | more than 11 years ago | (#4668576)

Quoting cjustus:
the solution is not to hire people just with skill XYZ, but to hire people that are capable of thinking for themselves, doing research, problem solving, and RTFM...

Right on. All to often I think we look for schools/degree programs/industry certifications to teach us all we need to know. The reality is that no program can teach you everything (though some will claim they can). Instead we should expect to learn basics and critical thinking skills so that we can apply them later. Unfortunately learning basics sometimes isn't very glamorous and people want to skip straight to the fancy, flashy, high level stuff.

I guess it just goes to show that in whatever you do having a good grasp of the foundational basics is always worth it.

Re:Good article - how to solve? (5, Interesting)

Bastian (66383) | more than 11 years ago | (#4668695)

I got into that problem at a carreer fair the other day. Someone asked me if I had VB experience, and I said I didn't because I can't afford a copy of VB, but I was familiar with other flavors of BASIC, event-driven programming, and other RAD kits, so I would probably be able to learn VB in an afternoon.

He just looked at me as if I had said someting in Klingon.

I've been getting the distinct impression that the good programmer who knows his shit but doesn't have skills X, Y, and Z from the start is fucked because the people who do the hiring are clueless enough about programming that all they can do is watch for X, Y, and Z on a resume and fail to notice anything else.

No, cheapest adequate solution wins (2)

Ars-Fartsica (166957) | more than 11 years ago | (#4668699)

No - the solution is not to hire people just with skill XYZ, but to hire people that are capable of thinking for themselves, doing research, problem solving, and RTFM...

...and of course all of these highly skilled "classical" eningeers want to work on your VB product, right?

Maybe, if you are willing to pay them twice the going rate for the minimally competent VB coder...and at the end of the day someone who is not analytical by nature but just knows VB itself really well will probably produce a better product. Critical thinking is useful, but knowing your tool very well is probably twice as valuable.

In any case as the barrier to entry into programming lowers, wages drop, and the cheapest adequate solution wins.

leaky abstractions need to be patched up (2, Insightful)

smd4985 (203677) | more than 11 years ago | (#4668479)

though joel sometimes thinks he is cooler than he is, this article he wrote was great. i think the points he make are valid.

i think the solution is that we need to have CERTIFIED software components. to really evolve the art of software engineering, we need to stop having to deal with low-level ideas and increasingly interact with higher-level abstractions. but joel makes the point that any abstraction we may use is leaky, forcing us to understand how it works! so we need to start having certification processes for abstractions - i should be able to pick up a component/abstraction and believe in its specifications 100%. that way i can stop worrying about low-level details and start using components to build even more interesting systems / applications.

Re:leaky abstractions need to be patched up (2)

binaryDigit (557647) | more than 11 years ago | (#4668590)

Well it's a nice idea in concept, but what software company would want to go throught the hassle of certifying a component? To avoid the "leaky abstraction", you are basically saying that it would have to be utterly bug free and that it would cover every conceivable use of the tool. Take the example of a c++ string class. Someone else (was it the author, I can't remember now) mentioned that the abstraction becomes leaky as soon as the novice C++ programmer tries "A" + "B", which would seem perfectly fine given instrinsic string support.

Re:leaky abstractions need to be patched up (0)

Anonymous Coward | more than 11 years ago | (#4668602)

i should be able to pick up a component/abstraction and believe in its specifications 100%. that way i can stop worrying about low-level details
You've missed the point. Joel says that you can never believe a component/abstraction 100%, and you'll always run the risk of having to go down to a lower level and look at the details. Read the article again.

Re:leaky abstractions need to be patched up (2)

smd4985 (203677) | more than 11 years ago | (#4668703)

"Joel says that you can never believe a component/abstraction 100%"

oh man, i forgot - when joe says something we must follow and not question him. two internships at microsoft and the man is a messiah!

but seriously, take hardware engineering for example. when boeing builds a plane, it can say with a high confidence rate that 1) it won't crash due to mechanical failure, and 2) it'll last such and such years. hardware components are often 'certified' - until we evolve the software engineering process to be similarly reliable we'll be stuck with a status quo of buggy software. regardless of what joe says, this CAN be done. take NASA software for spacecraft - they RIGOROUSLY test every possibility and guarantee their specifications (they have no choice but to do so - if they don't follow orders, then people die). some have even suggested enacting 'lemon laws' for software - basically, if software fails on you, then you can sue the creator.

as technology becomes more of a support structure for our daily existence, we'll increasingly have to ensure its reliability.

Re:leaky abstractions need to be patched up (3, Insightful)

Minna Kirai (624281) | more than 11 years ago | (#4668663)

so we need to start having certification processes for abstractions

Sure, that'd be nice I guess. And it'd be impossible, except for simple cases. Joel's example of the motorcar in the rain can't be fixed 100% unless you've got a weather-genie flying along above you.

Leaky abstractions have 2 main causes:
  • Laws of Physics ("acts of god"). These you can never fix, because someday the implementation of the abstraction will encounter a situation beyond its ability. When that happens, the users can either give up and go home, or learn about the mechanisms underlying the abstraction and resolve it directly (or call in a specialist to do it).
  • Backwards compatibility ("acts of man"). Things like the ASP example of hyperlinks to submit forms, or the C++ string class (mostly). These we could possibly fix, but to do so is often prohibitively expensive, or just won't pay off fast enough. The goal of 100% specification confidence is nice, but today people aren't usually willing to make the sacrifices.


Wow... (1)

RAMMS+EIN (578166) | more than 11 years ago | (#4668483)

This guy made my day. He's a good writer, and even people who don't know the Way Things Work with computers can understand his article. Moreover, he explains why the world needs and will always need wizards, which gives me an argument for why employers should hire me. Thanks man!

Hmm... (1, Offtopic)

Xformer (595973) | more than 11 years ago | (#4668484)

Imagine that we had a way of sending actors from Broadway to Hollywood that involved putting them in cars and driving them across the country. Some of these cars crashed, killing the poor actors.

Unlike TCP, though, we don't have the ability to re-send copies of Hollyweird actors. Or do we? [sonypictures.com]

Duh! (2)

LordNimon (85072) | more than 11 years ago | (#4668490)

In other words, TCP is obliged to somehow send data reliably using only an unreliable tool.

How is this news? All technologies, on some level, are inherently unreliable. Therefore, in order to obtain reliability, it is always by adding some kind of redundancy to an unreliable tool.

I've never seen a technology touted as "reliable" that didn't achieve that reliability without some kind of self-checking or redundancy somewhere. Maybe that's the author's point, but he makes it sound as TCP/IP is unique in this regard.

This is what programming is all about. It seems pretty obvious to me.

Re:Duh! (2)

binaryDigit (557647) | more than 11 years ago | (#4668558)

but he makes it sound as TCP/IP is unique in this regard

I think he's just using it as an example that almost anyone can relate to in the internet age. And while it is obvious to those of us who code/administer/tinker with such things, but his use of the "hollywood actors" analogy would seem to point to the fact that his audience is not us.

WTF? (-1, Troll)

pete-classic (75983) | more than 11 years ago | (#4668530)

Is this story a troll?

TCP is a way to transmit data that is reliable.


[snip]

By comparison, there is another method of transmitting data called IP which is unreliable.

[snip]

Here's the magic part: TCP is built on top of IP. In other words, TCP is obliged to somehow send data reliably using only an unreliable tool.


Wah? Is this serious? Does he mean UDP?


Imagine that we had a way of sending actors from Broadway to Hollywood that involved putting them in cars and driving them across the country. Some of these cars crashed, killing the poor actors.

[snip]

Now imagine a new service called Hollywood Express, which delivered actors to Hollywood, guaranteeing that they would (a) arrive (b) in order (c) in perfect condition. The magic part is that Hollywood Express doesn't have any method of delivering the actors, other than the unreliable method of putting them in cars and driving them across the country.


Huh? You can't retransmit cabbages or actors or hard copies of badly researched essays . . . but you can retransmit freaking TCP packets!

While his point may be valid (and I'm not conceding the point) the essay is crap.

-Peter

Re:WTF? (2)

kurisuto (165784) | more than 11 years ago | (#4668628)

> Huh? You can't retransmit cabbages or actors or hard copies of badly researched
> essays . . . but you can retransmit freaking TCP packets!

He does go on to say that "retransmission" in this case means sending an identical twin of the actor. Your criticism really isn't fair in ignoring this. The metaphor of identical twins for copies of information is not perfect, but as a teaching tool for explaining TCP to a non-technical type, I'm not sure I could come up with another metaphor which matches this one for its intuitive value.

Re:WTF? (0)

Anonymous Coward | more than 11 years ago | (#4668639)

you can send ip packets without using the udp or tcp protocol nyet?

Re:WTF? (1)

godefroi (52421) | more than 11 years ago | (#4668666)


"Wah? Is this serious? Does he mean UDP?"

No, read the book again. TCP is a reliable protocol that is built on top of IP, which is an unreliable protocol. UDP, which is also an unreliable protocol, is also built on top of IP.

Are YOU a troll?

(hmm. Am I?)

Re:WTF? (1)

milesbparty (527555) | more than 11 years ago | (#4668676)

This author obviously doesn't understand TCP/IP.

It doesn't, actually. If your pet snake has chewed through the network cable leading to your computer, and no IP packets can get through, then TCP can't do anything about it and your message doesn't arrive.

What an idodic statement...different layers.

Re:WTF? (0)

Anonymous Coward | more than 11 years ago | (#4668687)

Go refresh your TCP and IP knowledge.

TCP is implemented on top of IP
UDP is implemented on top of IP

The comparison between TCP and IP is a valid one in this article. He does not mean UDP.

You cannot retransmit TCP packets. You can retransmit TCP segments or the IP packets comprising these TCP segments.

It seems that your response was more crap than the essay.

Cheers

Re:WTF? (0)

Anonymous Coward | more than 11 years ago | (#4668711)

Wah? Is this serious? Does he mean UDP?

No, he really (and rightly) means "IP".

The ultimate leaky abstraction (5, Insightful)

nounderscores (246517) | more than 11 years ago | (#4668542)

Is our own bodies.

I'm studying to be a bioinformatics guy with the university of melbourne and have just had the misfortune of looking into the enzymatic reactions that control oxygen based metabolism in the human body.

I tried to do a worst case complexity analysis and gave up about half way through the krebs cycle. [virginia.edu]

When you think about it, most of basic science, some religeon and all of medicine has been about removing layers of abstraction to try and fix things when they go wrong.

saving time with abstractions (1)

agurkan (523320) | more than 11 years ago | (#4668545)

So the abstractions save us time working, but they don't save us time learning.

At last I know what is really bothering me about all these new kids learning C++ and leaving plain C out. It is not that I am lazy or against change. Next time I give advice to the undergrads working in my office I'll make sure to quote this.

Thank you very much...

This reenforces the important point... (2, Insightful)

venomkid (624425) | more than 11 years ago | (#4668548)

...to start with, or at least be competent with, the basics.

Any good programmer I've ever known started with the lower level stuff and was successful for this reason. Or at least plowed hard into the lower level stuff and learned it well when the time came, but the first scenario is preferable.

Throwing dreamweaver in some HTML kiddie's lap, as much as I love dreamweaver, is not going to get you a reliable Internet DB app.

This is true of almost any engineering profession (2, Interesting)

today (27810) | more than 11 years ago | (#4668556)

The mechanical, electrical, chemical, etc, engineering fields all have various degrees of abstractions via object hiding. It just isn't called "object hiding" because these are in fact real objects and there is no need to call them objects because it is natural to think of them that way. When debugging a design in any of these fields, it is not unusual to have to strip down layers and layers of "abstraction" (ie, pry into physical objects) to get to the bottom of a real tough problem. Those engineers with the broadest skills are usually the best at dealing with such problems. There isn't really anything new in the article.

Abstractions don't kill systems, people kill... (2, Interesting)

redfiche (621966) | more than 11 years ago | (#4668570)

Abstractions are good things, they help people understand systems better. An abstraction is a model, and if you use a model, you need to understand its limitations. High level languages have allowed a tremendous increase in programming productivity, with a price. But just as you cannot be really good at calculus without a thorough understanding of algebra, you cannot be a really good coder if you don't know what's going on underneath your abstractions.

Great article, but don't throw out the high level tools and go back to coding Assembler.

Leaky slashdotted server... (3, Funny)

dereklam (621517) | more than 11 years ago | (#4668572)

I said that TCP guarantees that your message will arrive. It doesn't, actually. If your pet snake has chewed through the network cable leading to your computer, and no IP packets can get through, then TCP can't do anything about it and your message doesn't arrive.

Unfortunately, his Slashdotted server is proving that to us right now.

Moore's law buries Leaky Abstractions law (2, Insightful)

Anonymous Coward | more than 11 years ago | (#4668573)

Hiding ugliness has its penalties. Over time processor performance buries these penalties. What Joel doesn't tell you is that abstraction can buy you productivity and simply put, make programming easier and open it up to larger audiences.

Maybe someone out there prefers to program without any abstraction layers at all, but they inherit so much complexity that it will be impossible for them to deliver a meaningful product in a reasonable time.

Visual Basic and abstraction breakdown (5, Interesting)

RobertB-DC (622190) | more than 11 years ago | (#4668582)

As a VB programmer, I've *lived* leaky abstractions. Nowhere has it been more obvious than in the gigantic VB app our team is responsible for maintaining. 262 .frm files, 36 .bas modules, 25 .cls classes, and a handful of .ctl's.

Much of our troubles, though, come from a single abstraction leak: the Sheridan (now called Infragistics [infragistics.com]) Grid control.

Like most VB controls, the Sheridan Grid is designed to be a drop-in, no-code way to display database information. It's designed to be bound to a data control, which itself is a drop-in no-code connection to a database using ODBC (or whatever the flavor of the month happens to be).

The first leak comes in to play because we don't use the data control. We generate SQL on the fly because we need to do things with our queries that go beyond the capabilities of the control, and we don't save to the database until the client clicks "OK". Right away, we've broken the Sheridan Grid's paradigm, and the abstraction started to leak. So we put in buckets -- bucketfuls of code in obscure control events to buffer up changes to be written when the form closes.

Just when things were running smoothly, Sheridan decided to take that kid with his finger in the dike and send him to an orphanage. They "upgraded" the control. The upgrade was designed to make the control more efficient, of course... but we don't use the data control! It completely broke all our code. Every single grid control in the application -- at least one and usually more in each of 200+ forms -- had to have all-new buckets installed to catch the leaks.

You may be wondering by now why we haven't switched to a better grid control. Sure enough, there are controls out there now that would meet 95% of our needs... but 1) that 5% has high client visibility and 2) the rest of the code works, by golly! No way we're going to rip it out unless we're absolutely forced to.

By the way, our application now compiles to a svelte 16.9 MEG...

A truly excellent essay (2)

dcollins (135727) | more than 11 years ago | (#4668585)

This is one the best essays on software engineering I've read in a while. As a programmer and CS educator, it's really served to crystallize for me why (a) it seems so much harder for students to learn programming these days, and (b) why I've grown unhappy over the years with the series of new engineering paradigms that are in use. Extremely helpful for putting my own thoughts in order.

The law statement itself, "all non-trivial abstractions, to some degree, are leaky" may possibly get included in my personal "top 10" aphorisms manifesto.

Bad term choosen IMO (2, Interesting)

shadowtramp (573751) | more than 11 years ago | (#4668611)

Don't know how the term leak fills to mentioned nonprogrammers but in programmers' slang the word leak has distinct meaning. And it does diverge from what Joel use it for.

I think that term ooze would suite better in this case. It's possesses a kind of dirtiness to itself and the fealing the word 'ooze' gives me fits good with the matter of described problem. :o)

Back to the article. To be serious, i think that Joel mixed all things as examples of 'Leaky abstraction' to no purpose. Too different situations make concept to fall apart. Here what i mean:
In case of tcp/ip it denotes limits of abstraction. And regardless of programmer background every sane man should now those limits do exist.
In case of page faults it's a matter of competence - there is no abstraction at all. You either do know how your code is compiled and executed or you don't. It's the same when you know what the phrase in a given language do realy mean or you don't. I simplify here.
In the case of C++ strings i saw the only good example. What in my opinion the experience of STL and string class usage tells in this case is: one should understand the underlying mechanics fully before rely on abstraction behaviour.

In programming it is realy simple to tell will the given 'abstraction' present you with an easter egg or not: if you can imagine FSM for the abstraction you will definitely know when to use it.

other leaks (2)

mr_gerbik (122036) | more than 11 years ago | (#4668630)

Joel should write an article about his leaky hosting company... or maybe his leaky colo-ed box.

Since I can't get to the site and read the article, I'll tell some jokes.

"Leaky Abstractions?! Is this guy talking about Proctology??"

"Leaky Abstractions?! Someone get this guy a plumber!"

"Leaky Abstractions?! I knew we should have used the pill!"

-gerbik

Advertisement or real issue? (1)

redragon (161901) | more than 11 years ago | (#4668645)

Ok, the last line of the article:

My company, Fog Creek Software, has just released FogBUGZ 3.0, the latest version of our innovative system for managing the software development process. Check it out now!

I don't really see that as helping the issue of having problems with "leaky abstractions."

The issues really seems to be several fold:

1.) Our current abstractions are not good enough.
2.) Many code generation tools are subject to errors, deriving from the underlying implementation they are abstracting.

3.) People don't know the underlying structure, and depend upon the abstractions.
I think that this is over simplifying where the problems are. Our current abstractions are obviously limited by thier underlying impelementations. What we need are better base level tools, this means better object oriented programming languages, that aren't merely an abstraction of existing languages, because again they fall victim to resting on top of a faulty abstraction. What we also need is better documentation. I hate to say it, but often times example code looks like a train wreck.

Abstractions are important to computer science, if it were not for them, we wouldn't have any high level languages. Without those higher level languages, we wouldn't have complex applications. We need abstraction, it's the only way to handle complexity. Where does that leave us? What we need are tools designed from the ground up to be cross platform, object oriented, robust core data types, and useful error/exception handling.

It seems like he's talking about a lot of problems, but not presenting any solutions, other than his little advertisement at the bottom, which helps us abstract our development team, but little for the developers...how very constructive. Try again with some constructive criticism, not just criticism.

C-out

Leaky? There's got to be a better word. (2, Interesting)

unfortunateson (527551) | more than 11 years ago | (#4668690)

For something like IP packets, leaky is acceptable, but for many of those other abstractions, constipated might be a better adjective. Some of the tools and technologies out there (remember 4GL report-writers?) were big clogging masses that just won't pass.

The first thing I do when I start in on a new technology (VBA, CGI, ASP, whatever) is to start digging in the corners and see where the model starts breaking down.

What first turned me on to Perl (I'm trying hard not to flamebait here) was the statement that the easy things should be easy, and the hard things possible.

But even Perl's abstraction of scalars could use a little fiber to move through the system. Turn strict and warnings on, and suddenly your "strings when you need 'em" stop being quite so flexible, and you start worrying about when it's really got something in it or not.

On the HTML coding model breaking down, my current least-fave is checkboxes: if unchecked, they don't return a value to the server in the query, making it hard to determine whether the user is coming at the script the first time and there's no value, or just didn't select a value.

Then there's always "This space intentionally left blank.*" Which I always footnote with "*...or it would have been if not for this notice." Sure sign of needing more regularity in your diet.

Not all abstractions are leaky (2, Insightful)

thom2000 (529108) | more than 11 years ago | (#4668710)

Sure, the author points out a few examples of leaky abstractions. But his conclusion seems to be that you always will have to know what is behind the abstraction.

I don't think that's true. It depends on how the abstraction is defined, what it claims to be.

You can use TCP without knowing how the internals work, and assume that all data will be reliably delivered, _unless_ the connection is broken. That is a better abstraction.

And the virtual memory abstraction doesn't say that all memory accesses is guaranteed to take the same amount of time, so I don't consider it to be leaky.

So I don't entirely agree with the author's conclusions.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...