Automated Linux Error Checking 25
Caydel writes "In a recent message to the Linux Kernel Mailing List (LKML), Ben Chelf, CTO of Coverity, Inc. announced an internal framework to continually scan open source projects for source defects and provide the results of their analysis back to the developers of those projects. The linux kernel is one of 32 open source projects monitored by Coverity. Coverity is looking for a few group-nominated maintainers to access the reports, in order to patch the bugs found before they are announced to the general public. For those not familiar with Coverity, they are a small company out of Stanford who monitor source code correctness through automatic static source code analysis."
LinuxSoft (Score:3, Funny)
Linux has caused an error and will be terminated
CTO passes the initial LKML scrutiny... (Score:4, Insightful)
Re:CTO passes the initial LKML scrutiny... (Score:2)
Yup, "and have been since it's inception" instantly marks him as a good fit for the Linux community!
Honestly, though, his reply was more polite and reasonable than the thread deserved. Do people really have nothing better to contribute to the LKML than silly hairsplitting about the difference between "Linux" and "the Linux kernel"?
Re:CTO passes the initial LKML scrutiny... (Score:3, Informative)
static_analysis++ (Score:5, Interesting)
Lots more on all that in my book [pmdapplied.com] - there's a downloadable free chapter there on using static analysis to improve JUnit tests if you want to get a feel for things.
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:5, Informative)
Good question:
That's JDK 1.4 on Linux, perhaps newer javac versions do a better job...
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:2)
Re:static_analysis++ (Score:3, Insightful)
Re:static_analysis++ (Score:2)
Quite true! But until Sun improves the Java compiler, it's nice to have PMD around to catch such things. And anyhow, the readability improvements might make that change worthwhile, regardless of bytecode savings.
As I understand it... (Score:4, Interesting)
Historically, Coverity's software has been used for some considerable time. It was first used under the name of the "Stanford Checker" and made an absolutely staggering difference. I believe it was first used in the 2.1.x era, though Linux historians can feel free to correct me on that.
Because it is not a run-time check, but a compile-time check, it is unclear to me what (if any) tests they have for violation of invariants, probable infinite loops, validating the parameters of functions passed as pointers, and other strictly run-time gremlins.
Because its documentation does say it's only really good for large programs, it is also unclear how effective it will be for debugging strictly external drivers or small pieces of support code. Many of the libraries and utilities out there for Linux are way too small to be reliably tested with this program. (IIRC, they recommend a minimum of something like half a million lines of code.)
Although the Linux kernel has been tested, bugs in compilation will render testing worthless. I do not know how extensively either GCC or the binutils package have been checked. I'm not even sure there is anything in binutils big enough to be checked. Presumably Gnu libc should also be tested - otherwise it's unclear if the checker or the compiler can be trusted... and how do you make sure that the absence of errors in libc aren't due to a problem in the checker caused by a bug in libc?
(Checking the checker is relatively easy. Checking longer loops is a harder problem, as there are more interdependencies.)
Re:As I understand it... (Score:4, Interesting)
They actually do a lot with violation of invariants: it's looking for cases where it can't prove that the invariant is maintained. Of course, there's always the chance of false positives, where the code is sure to work, but only for some tricky reason that the checker can't figure out, but these are often cases where the code should be made more obvious anyway, because somebody's likely to accidentally change it in ways that don't work if it's subtle. There's also a lot of extra-detailed type checking along particular code paths, so it can identify, for example, what it's safe to turn a particular void * into in a particular function, and then verify (e.g.) that ext3_release_file is only called on struct files where private_data is a struct dir_private_info. (Which is true, because nothing could create a struct file with something else there and with the ext3 fops, or create another set of fops with ext3_release_file)
Checking external drivers shouldn't be a problem. The reason they need a lot of code is because they're determining invariants from actual usage, and it needs the actual usage to cover the possibilities well (e.g., it tries to find a lock that almost always protects a particular bit of data, and then gives errors on violations; if you only have a couple of places that access something, and they don't match, it can't tell which is the real lock and which spots are wrong). External drivers are small, but their behavior can be compared against the rest of the kernel, because they do a lot of calling functions that the base kernel defines and uses. The hard case is really something like libpng, where it has a wide API, and there's no way for the program to determine what rules users of the library are told by the documentation to follow. (In this case, it may make sense to check the library combined with a number of common programs and use that to generate the checker's expectations of library usage.)
It's also not particularly useful to check things like gcc, because the checker doesn't have any way to determine if the compiler is compiling correctly or not; it can only tell if it might do crazy things in odd situations. Compilers are generally written sufficiently conservatively that problems tend to be safe-but-does-the-wrong-thing code, and the bugs you'd find with the checker would generally be "internal compiler error" bugs, not ones that lead to incorrect output (or any output at all...)
Re:As I understand it... (Score:3, Interesting)
Dunno why, but nearly all the errors I've found are of the "internal compiler error" kind. (Why can't I have a NORMAL error for once?) For Fortran code, for example, I'm generally using G95 because gfortran barfs on a lot of the comput
Re:As I understand it... (Score:2)
Re:As I understand it... (Score:2)
Reporting bugs before they're released (Score:4, Interesting)
Interestingly, the discussion on the mailing list so far has been primarily complaining that the announcement failed to take into account the fact that Coverity has been doing this already for a while. In fact, the only thing that's new is that they've put together internal infrastructure that lets them also handle other projects conveniently, and have therefore moved their kernel result info. It looks a bit like they sent a newly-scanned-project-directed form letter to the project they were already scanning, which is clearly a bit of a faux pas.
Eating Lint Is Good (Score:1)
I've generally been a believer in static analysis tools. I remember using lint quite a bit of code back in the day and remember it not only finding basic syntax errors, but it educated me more about the code itself and even found some bizarre bugs that normal peer reviews would not have found. The code was better for it, even if it took days to really get through all the analysis output and do something about it.
30 of the most critical and widely used projects (Score:1)
DHS out of a job? (Score:2)