Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Book Review: Software Build Systems

samzenpus posted more than 3 years ago | from the read-all-about-it dept.

Books 29

MassDosage writes "Software Build Systems by Peter Smith is a well written, albeit rather lengthy book that covers the intricacies of systems used to build software. It tries to do this in a technology-neutral manner where possible but covers existing tools like Ant, Make and Scons as a means of illustrating the various concepts covered instead of just focusing on theory. The real world examples illustrate building Java, C/C++, C# and Python software and cover scaling up from small builds with tens of source files up to massive builds with tens of thousands. All of the technologies used are introduced in some depth which newcomers should find useful but experienced build developers may want to skim over." Read below for the rest of MassDosage's review.Software Build Systems weighs in at a hefty 580 odd pages and covers a lot of ground. It is targeted at developers, managers and build engineers and while there is definitely something for all of these groups, each of them will most likely find themselves skipping sections which are either not of interest to them or are too basic. It is also important to note that this book is not a hands-on tutorial of how to set up a build system from scratch but instead uses a variety of detailed examples to illustrate its concepts with pointers to external documentation for those wanting to actually implement a complete build system of their own.

The book is clearly divided into parts which move from "The Basics" to "The Build Tools" and then on to "Advanced Topics" and finally "Scaling Up". "The Basics" really is basic and most build engineers and developers worth their salt can probably skim read most of this. The fundamentals of C/C++, Java and C# are also covered illustrating a major theme in this book in that it tries to be programming language neutral and just use the different languages as examples to explain various concepts. While this is an inclusive, worthwhile aim it does mean that someone only interested in say, building Java software, will find large parts of the book irrelevant. This applies particularly to the chapters on the various build tools where Make, Ant, SCons, CMake and Eclipse are all covered in painstaking detail. The pros and cons of each of these tools are discussed in a non-biased manner using real world examples where possible. Anyone who has actually written a build system using one of these tools probably won't find much they don't already know but for a manager or someone new to any of these technologies the author provides a a good introduction with mentions of similar tools and pointers for finding out more about each of them.

"Advanced Topics" is where things start to get interesting as concepts like dependency graphs, change detection and version management that the build tools rely on to function are discussed in depth. A lot of the advice in these chapters feels like it comes from practical experience and the best parts are where tool-neutral tips are provided. There is a bit too much detail here as well as the odd digression which feels unnecessary in a book of this length. Do we really need to know the details of lex and yacc in order to create a build system? The author clearly has an understanding of open source development and competently discusses the wild world of building software that may run on a plethora of machines and platforms that the developers have little or no control over as well as software that is built in a more clinical manner for a limited set of environments.

Build systems for massive software projects are covered in "Scaling Up" and the author acknowledges that this probably isn't relevant for everyone. I however think that the first chapter in this section ("Reducing complexity for end users") is the best in the whole book and applies to all build systems, regardless of size or technology. A better title for this chapter would have been "Best practices for build systems" as it doles out plenty of good tips such as how to automatically detect dependencies, what not to keep in a source control system, when to abort a build (early), why to ensure there is always a way to clean up all build-generated artifacts and so on. The gist here is to try to reduce complexity wherever possible and the advice is all very well reasoned and practical. The book wraps up by covering methods for reducing the size of a build and ways to speed up and optimize builds.

Overall this is a very well written, edited and structured book but it does suffer from attempting to cover too much and going into detail on topics which aren't going to be of interest to everyone. A prime example of this is the section on packaging technologies where I doubt that someone concerned with creating Debian packages will find the information on the Nullsoft Scriptable Install System very useful, or vice versa. The same applies to the varying levels of technical detail in the book — a manager may find the introductions to concepts like compiled versus scripted languages enlightening but to most developers this will be old hat. Conversely the intricacies of how Make calculates its dependency graph is probably interesting to a build engineer but most managers will be out of their depth.

This isn't the kind of book most people will read from cover to cover. Instead I recommend skimming through the sections that aren't immediately applicable and just focusing on the parts that discuss the particular build technologies the reader is interested in as well as the more technology neutral parts towards the end. It is obvious that years (if not decades) of real world experience have been distilled in Software Build Systems. It is just a shame that this process wasn't a bit more focused as this could have been a great book, instead of just a good book with some great sections and some sections that most readers will find themselves skimming over.

I was given a copy of this book free of charge by the publisher for review purposes. They placed no restrictions on what I could say and left me to be as critical as I wanted so the above review is my own honest opinion.

You can purchase Software Build Systems: Principles and Experience from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

cancel ×

29 comments

Sorry! There are no comments related to the filter you selected.

Nice book! (-1)

Anonymous Coward | more than 3 years ago | (#36613976)

But the real question is... have Slashdot users heard of the person that dumps on whatever he pleases like corn on peas?

complexity (1)

Skapare (16644) | more than 3 years ago | (#36614066)

I wonder how much of the complexity of these things is due to people wanting to be able to do an incremental build of just part of the whole thing.

Re:complexity (0)

Anonymous Coward | more than 3 years ago | (#36614168)

It is no more difficult to build just part of a project in an incremental manner than to build the whole thing incrementally. All you need is (for each thing) a list of the things it depends upon. The issue is how to get and maintain such a list.

Re:complexity (2)

Skapare (16644) | more than 3 years ago | (#36614230)

And there is one of the complex parts, the automated dependency detection. If they would just build the whole thing, you don't need that. Otherwise you need to know what to build, and what you don't need to build. For very large projects, build everything can consume a lot of time. But most projects are smaller and should be able to build in under a minute.

Re:complexity (0)

Anonymous Coward | more than 3 years ago | (#36614392)

Even a 30s compile-run-test cycle is not so good if you are following the excellent paradigm of testing your program after every small change. I'd hate to be fixing compile errors using a build system you wrote.

Re:complexity (1)

Amigan (25469) | more than 3 years ago | (#36614676)

Obviously you've never been involved with Enterprise level software construction. Most folks refer to it as 'build', but I'm a fan of Steve McConnell's term - software construction. Large applications that support multiple platforms can take a long time to build - especially if you're doing binaries and not Java bytecode. Scratch builds (ie, directly from source) are the way to guarantee that the code in the repository is the code that produces the product, but it's not economically feasible to have product dedicated build machines - from a space/utilization perspective. That means that build machines are shared resources - and that makes scratch builds even slower. In industry, large projects/products are the norm - not the exception.

Good Reference Book (1)

Normal Dan (1053064) | more than 3 years ago | (#36614088)

Especially if you're into trying a little of everything.

not a packt :) (2)

JonySuede (1908576) | more than 3 years ago | (#36614114)

good review but what about continuous integration? Is this books already outdated or was this important part of software build omitted from the book.

Re:not a packt :) (0)

Anonymous Coward | more than 3 years ago | (#36614156)

Continuous integration isn't important to all deployment models. And suppose you claim it is, I would say it's of different relevance to different types of applications for different reasons, and beyond the scope of the book.

For users who don't have a concept of continuous integration OR build systems, it might be a bigger bite than a beginner can chew, to try and digest both.

Re:not a packt :) (1)

JonySuede (1908576) | more than 3 years ago | (#36614394)

it might be a bigger bite than a beginner can chew, to try and digest both.it might be a bigger bite than a beginner can chew, to try and digest both.

I guess you are right

Re:not a packt :) (1)

robi2106 (464558) | more than 3 years ago | (#36616316)

CI was going to be my question as well, since I spend my time at work managing a multi-site Jenkins CI system for ~100 jobs.

Re:not a packt :) (0)

Anonymous Coward | more than 3 years ago | (#36644944)

CI itself is a rather different topic. What's even worse is that people often confused with the practice of "Continuous Integration" from "Continuous Integration System". Having a proper automated build system is the very fundamental block essential for many development teams.

Students - get this! (2)

LikwidCirkel (1542097) | more than 3 years ago | (#36614174)

I think this is quite useful. This topic is super-important and highly neglected in most Universities and colleges. Newbies where I work as an embedded developer learn very fast that compiling things is actually a lot harder than just writing code.

Re:Students - get this! (3, Informative)

loom_weaver (527816) | more than 3 years ago | (#36614326)

It's highly neglected in most companies as well.

I bet more time is lost due to complex build systems than people want to admit. Gotta love the 72 hour builds on some systems I get to observe.

The complexity of the build system is sometimes on par with the complexity of the application itself but the tool support and resources just is not there.

No Maven? (2)

apetrelli (1308945) | more than 3 years ago | (#36614250)

Why not covering Maven? It's got a totally different way of building applications, together with some concepts like artifacts, dependency, artifact repository, that are not present in other build systems.
I think that it is worth at least a mention.

Re:No Maven? (0)

Lunix Nutcase (1092239) | more than 3 years ago | (#36614306)

Because books do not have unlimited pages to include a discussion of every topic under the sun?

Re:No Maven? (0)

Anonymous Coward | more than 3 years ago | (#36614434)

maven as a concept is really cool. maven as designed and implemented is not worth mentioning

Re:No Maven? (1)

Slashdot Parent (995749) | more than 3 years ago | (#36621830)

Why not covering Maven?

Probably because it would take an entire book to even think about covering Maven.

Make work (2)

EmperorOfCanada (1332175) | more than 3 years ago | (#36614280)

I have seen build environments that made life so much better. But in most cases I have seen complex build environments that could be described as nothing more than make work projects. When you have a dual processored 8-12 core desktops incremental build times are generally pretty damn short unless you are compiling something massive. So no savings there. Where build environments can help is when you are working with a cluster of monkeys who would simply screw the build up every time they added their so called contributions. After discovering Git I really came to realize that many tools are designed to prevent stupidity at the cost of efficiency. Git would be a great example of a tool that promotes efficiency but won't keep the morons at bay. For many smaller projects build environments are generally moron prevention tools.
But at a certain level of complexity they do really act as a traffic cop of sorts.

Re:Make work (1)

Amigan (25469) | more than 3 years ago | (#36614734)

As long as your target is current hw/sw, then being able to use 8-12 core machines as build engines is nice. Unfortunately, commercial customers don't upgrade as often as one would like, so supporting older versions of the OS is generally required. Depending on the OS, it may/may not run on recent hardware, which limits what kind of build machines you can actively support. Most hw/sw vendors provide backwards compatibility (something compiled on Solaris 8 runs on 9, 10, 11), but you should never count on forward compatibility (something compiled on Solaris 9 is all but guaranteed NOT to run on Solaris 8).

Cmake? Maven? (2)

dwheeler (321049) | more than 3 years ago | (#36614324)

The summary doesn't mention cmake or maven. That's bizarre. I hope the actual book covers those; those are really common and useful.

Re:Cmake? Maven? (3, Interesting)

DeKO (671377) | more than 3 years ago | (#36615376)

CMake is there in the summary. Maven is not that popular probably due to its design to do "everything".

What seems to be really missing is autotools. Even if you don't want to admit it is better than most alternatives, it's the only one that really solves a ton of problems that no other tool is able to handle. Simply reading through the autoconf, automake and libtool manuals will teach you a lot about the many issues most other tools just ignore, or solve poorly.

CMake? Maven? Forget that noise, use Tundra. (2)

Suiggy (1544213) | more than 3 years ago | (#36617338)

I've recently started using Tundra, a very light-weight and scalable (CPU core wise) build system, with very fast and reliable dependency analysis written in C, using pthreads/winthreads and taking advantage of SIMD instruction sets. It supports multiple build configurations and variants side-by-side, multiple disjoint build targets/deliverables, and has very DRY LUA-based DSL for project configuration. It doesn't hide CFLAG details, it gets out of the way and lets you precisely control what you're passing to the compiler for each tool chain you want to support. It's released under the GPLv3, was written by a DICE employee and is being used to build Battlefield 3.

I was able to get rid of ~25 CMakeLists.txt that made up thousands of lines of text and replaced it with a single ~350 line tundra build file, in building a 200,000+ LoC multi-platform project and combined with a very stripped down autoconf script for environment detection and config.h generation, was able to support all of my requirements. I had to hack and hack to get CMake to try to support what I needed.

https://github.com/deplinenoise/tundr,a [github.com]
http://j.mp/emwZ9Z [j.mp] for the PDF presentation slides.

Disclaimer: I have no relations with the author of Tundra or EA/DICE.

Re:CMake? Maven? Forget that noise, use Tundra. (1)

Suiggy (1544213) | more than 3 years ago | (#36617346)

Sorry, a stray comma got into that URL. It should be: https://github.com/deplinenoise/tundra [github.com]

Cross-platform? (1)

SanityInAnarchy (655584) | more than 3 years ago | (#36616192)

I doubt that someone concerned with creating Debian packages will find the information on the Nullsoft Scriptable Install System very useful, or vice versa.

While I love living in the world of web apps, where I can simply point people at the latest Chrome or Firefox for their chosen platform, I would certainly find both of these things helpful if I ever need to develop a non-trivial cross-platform native application. In fact, if you're going to cover Debian packages at all, any developer interested in those is probably also (unfortunately) interested in some sort of installer for Windows.

But this is, of course, why I prefer web apps. Otherwise I'd also have to deal with building a Mac .app or .mpkg, whatever iOS and Android support, and an RPM at least, if not also packages for Arch, Gentoo, and others, plus a simple tarball with some sort of install script or instructions for anyone I left out.

Shell scripts (1)

FranTaylor (164577) | more than 3 years ago | (#36616796)

Every product I've ever worked on is built with a shell script

There is really no other way to make everything happen

Small products might get away with make or ant or whatever but real products are made from a variety of projects and shell scripts are the only good way to glue them all together.

no maven or cmake (0)

Anonymous Coward | more than 3 years ago | (#36619936)

Oh please people. Is that really all you care about? Your fave buzzwords? Or a contest to bitch about the most obscure make replacement you can find?

First, that completely misses the point of explaining techniques. This is ironic because IME most people complaining about "classic" make and derivatives --sysV make, bsd's pmake, gnu's NIH gmake, etc.-- completely fail to understand how basic rules work, down to creating massive blobs of rules that really oughtn't work at all and can be replaced with just a few of the most basic of generic rules. That's a failure to attain easily achievable simplicity and thus a clear failure of understanding your own tools. My run-ins with cmake exhibit more of the same, only now through generated rules full of absolute paths and failure to use variables and generic rules seemingly just to artificially raise the importance of the generating tool. Instead of putting effort into creating more convoluted meta tools, spend a couple minutes with the manual and learn how to do it with the existing facilities, please.

Second, it's typical developer short-sightedness, conveniently forgetting that down the road the software needs to be used and thus usable by others, too. To wit: I've seen OSS projects "upgraded" to the very latest and hippest in make-like replacement things just so now the build has colours in it. But that does require that everyone building the system --not just the developer or a few package building specialists either-- now also has to have that thing just to build the software. That's a completely gratuitous added dependency, especially as the project is simple enough that a basic make does just fine and has for years. Replacing it does no good even if it does no harm. So why bother all of your users that build your software themselves with it? It's poor release engineering, is what it is.

Sadly, there's no simple cure for NIH disease.

Stay Away From scons (1)

smcdow (114828) | more than 3 years ago | (#36620914)

We're moving back to make. Full disclosure: it was my idea to move to scons in the first place. Dammit.

Software Build Systems (0)

Anonymous Coward | more than 3 years ago | (#36658436)

Software Build Systems, this is really a nice book, I got a lot from this book... now I build a crystal jewelry [aimengcrystal.com] online store adopt the ways from this book!

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?