Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Linux Troubleshooting

samzenpus posted more than 8 years ago

Linux 60

norburym writes "The Bruce Perens Open Source Series of books published by Prentice Hall PTR is a strong collection of nearly 20 volumes focusing on Linux and open source technology. Edited by Linux guru and former Debian GNU/Linux Project Leader, Bruce Perens, the books are aimed toward developers, sysadmins and power users. Several months following the release of a new print volume, a free electronic version is made available on Prentice Hall PTR's web site. The series includes some excellent editions including Official Samba-3 HOWTO and Reference Guide (2nd ed.), Linux Quick Fix Notebook and PHP 5 Power Programming. The newest book by Mark Wilding and Dan Behman, Self-Service Linux: Determining Problems and Finding Solutions, is another well-written and worthy companion to this series." Read the rest of Mary's review.

This is not a basic Linux HOW TO book: authors Wilding and Behman take the reader to a level past the introductory Linux OS installation instructions and KDE/GNOME window dressing changes. In all real life scenarios and at some critical point, a Linux user or admin will need to troubleshoot some aspect of the system they use at home or the systems they manage on the job. This book is for that power user, systems administrator or developer who will, out of base necessity, require a proper bag of tools and practical guidance in establishing an effective set of skills for troubleshooting one or more Linux systems.

A quick scan of the table of contents gives a very abbreviated summary of the book and actually belies the depth of the contents. The authors break the chapters into very self-contained topics including best practices and initial investigations, strace and system call tracing explained, the /proc filesystem, compiling, GDB (GNU Debugger), Linux system crashes and hangs, and kernel debugging, among others. These chapters are filled with detailed examples that perfectly illustrate real world scenarios that any Linux user will be familiar with.

Chapter 1 is an overview of the complex process of problem determination and resolution and begins with steps to configure your Linux system(s) for optimal troubleshooting. The authors outline a selection of tools they recommend the reader/user install on their Linux system(s) in anticipation of future problems: strace, ltrace, lsof, top, traceroute/tcptraceroute, ping, hexdump, tcpdump/ethereal, GDB and readelf. These and many others are categorized by type (process information and debugging, network, system information, files and object files, kernel and miscellaneous) in Appendix A, "The Toolbox." Wilding and Behman stress the importance of balancing the need to solve issues immediately vs. building troubleshooting skills. They outline four phases of problem investigation (using your own knowledge and skills to investigate, using the Internet, conducting a deeper investigation, and getting help) and discuss where the various tools fit into different scenarios, how to collect information about system changes, what resources are available on the Internet (Google, USENET, Bugzilla, etc.), how to handle more difficult problems and where and how to get outside help, if necessary.

Chapter 2 explains system call tracing, introduces the strace tool (traces system calls between a process and the kernel) and how to use it to diagnose errors related to the operating system. This is a very well organized chapter with plenty of depth. Wilding and Behman offer an extensive discussion of this first tool, progressing from simple examples that illustrate how to read the strace output, how/when to use strace options, timing system call activity, tracing an existing running process, to many practical debugging examples.

Chapter 3, "The /proc Filesystem," looks at user process information (/proc/self, /proc/<pid>, /proc/<pid>/environ, /proc/<pid>/mem), kernel information and manipulation (/proc/cpufreq, /proc/cpuinfo, /proc/devices, /proc/meminfo, /proc/partitions), and system information and manipulation (/proc/sys/fs, /proc/sys/kernel, /proc/sys/vm). The authors run through files and directories relative to /proc and describe how to view information about the kernel and currently running processes. This chapter gives a good example of how to use the "kernel magic sysrq key" feature (using the ALT-SysRq hotkeys to get kernel information) when a system hangs. Output from the commands showPC, showMem and showTasks are given as examples.

The next chapter details the GCC (GNU Compiler Collection) and compiling. The authors don't attempt to walk the reader through basic kernel source compiling but rather they concentrate on how to decipher errors that arise from compiling source. They give a basic outline of some basic compile failures (environment/setup errors, compiler version differences, user error, code error, etc.) then show a common error involving both incorrect code and different allowances made between compiler versions. Wilding and Behman show the reader how to decipher the kernel error and how to use both existing documentation and bugs posted on the Internet to correct the errors and rerun the compilation successfully. This is a very practical demonstration of how compile errors can be worked out and solved quickly.

Chapter 5 begins with a definition of "stacks" and a description of stack structure, local variables, and stack frames. Also shown is how to display the raw stack in a debugger like DDD (Data Display Debugger) or GDB (The Gnu Debugger) in order to perform a detailed stack analysis. The authors use the backtrace command to look at stack traceback output from GDB, "walking the stack" (manually walk the raw stack frame by frame using the dladdr function), common causes of stack corruption, and SIGILL signals.

Debugging applications is the subject of the next chapter with the majority of the chapter dedicated to The GNU Debugger. This is a logical place for a discussion on debuggers as the authors point out that they are particularly useful when problems can't be solved through log files, error messages, etc., when a problem is of an immediate nature (i.e. doesn't extend over a long period of time) and when source is available. GDB command line editing is covered along with how to control a live process by running the process directly through the debugger, how to attach to a running process and how to use a core file (or a process image) to perform debugging. The authors also examine viewing the memory map and variables, looking at the contents of register dumps, working with C++ (inline functions and exceptions), and problems with threaded applications. A brief description of the Data Display Debugger (DDD) GUI front-end to GDB is included at the conclusion of the chapter.

Chapter 7 deals with "System Crashes and Hangs" and how to assemble the appropriate information necessary to troubleshoot a system problem using various tools and techniques: using the syslog, setting up and using a serial console, using the SysRq kernel magic hotkey, examining the oops report generated by a manual kernel trap, considering hardware failure issues, and setting up cscope to index kernel sources. This chapter prepares the reader for documenting proper and extensive details about errors and problems not only for rapid diagnosis but also in the event he or she needs to call in an expert.

Kernel Debugging with KDB is a brief chapter that instructs the reader on how to enable and activate KDB, basic commands associated with its use, and some examples on how to use it. Several good illustrations of where KDB proves useful over other tools are included.

The final chapter explores the ELF file format (executable and linking format) for shared libraries and executables. The authors provide a comprehensive look at the ELF standard on Linux. They start with basic definitions and concepts (symbols names and C versus C++, linking with static libraries and run time linking, and run time linker) and prep the reader with some source code that is used in later chapter examples. They examine the ELF file structure (the header using hexdump, segments/sections with readelf, the program header table, and the section header table). This is probably the strongest chapter in the book. There is enough information and instruction in this chapter to arm a Linux system troubleshooter to follow the practical examples with little effort.

The book concludes with two valuable appendices that detail the authors' selected tools for Linux problem determination and include a data collector script intended to capture basic critical system information in the event of a problem. As discussed above, the "Toolbox" appendix is a summary of the authors' selection of best Linux tools for diagnosing problems. Each tool has a brief description, where to get it, level of usefulness, when to use the tool and additional notes. Appendix B, "Data Collection Script," offers the reader a sample bash script tool that gathers a broad range of system information. The authors provide several optional switches to increase the amount of data collected with the caveat that time to collect that information also increases.

Wilding and Behman assume some familiarity with the Linux system: their advice and instruction are intended for those users who are not afraid of the CLI and who understand basic Linux operating and file system structure. That said, Self-Service Linux: Mastering the Art of Problem Determination is a valuable resource for advanced users and system administrators. In short, this book is for anyone who uses Linux on a daily basis on one or multiple systems. The examples are fully detailed: the reader gets commands, options, output, sample code, and a variety of possible outcome scenarios. Wilding and Behman set out a realistic and practical approach to problem solving; they satisfy the troubleshooter in all of us. Self-Service Linux is a welcome addition to the Bruce Perens Open Source series of Prentice Hall PTR professional reference books."

You can purchase Self-Service Linux: Determining Problems and Finding Solutions from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered


Awesome! (5, Informative)

matr0x_x (919985) | more than 8 years ago | (#14340371)

A free electronic version is available for download - thats sweet :)

Re:Awesome! (3, Interesting)

smorpheus (868363) | more than 8 years ago | (#14340786)

How Apropos! But seriously, it makes sense, I think to give away large samples of books and other works because if someone really likes the book, they're going to want a print copy for quicker references.

Re:Awesome! (2, Interesting)

morgan_greywolf (835522) | more than 8 years ago | (#14340914)

I couldn't find it by following the link in the article. All I found was a free online chapter. Anyone else find it?

Re:Awesome! (5, Informative)

Anonymous Coward | more than 8 years ago | (#14342073)

Yes. I clicked on the Get your FREE downloads and book information here. [phptr.com] link. Then most of the books listed there had a download in PDF option. Some of them didn't or only had a sample chapter, but I downloaded the Linux Quick Fix Notebook and that did seem to be the whole book (696 pages is a bit excessive for just a sample.)

Hey, wait a second! (-1, Flamebait)

Anonymous Coward | more than 8 years ago | (#14340435)

I thought Linux didn't need troubleshooting! ;-)

Re:Hey, wait a second! (0)

Anonymous Coward | more than 8 years ago | (#14340494)

Try getting Wireless WEP running on SUSE 10 and then tell us that Linux doesn't need toubleshooting!

Re:Hey, wait a second! (2, Funny)

Anonymous Coward | more than 8 years ago | (#14340518)

Wireless Wireless Encryption Protocol brought to you by the Department of Redundancy Department...

Re:Hey, wait a second! (0)

Anonymous Coward | more than 8 years ago | (#14343454)

Well, that's not really Linux problem. If you, like me, have wlan card from fucked-up vendor (Broadcom in my case) who won't release specs to kernel developers, those guys can't do much.

Re:Hey, wait a second! (3, Funny)

mordors9 (665662) | more than 8 years ago | (#14340682)

No, any OS can have trouble that needs to be corrected. It's just with Linux, you don't just restart a couple times a day to try to solve the problem ;-)

Re:Hey, wait a second! (0, Flamebait)

Donald Ferrone (863523) | more than 8 years ago | (#14340706)

And unless you're a complete and total fucking moron, you don't restart that often with most any other operating system to solve trouble either. I guess your experience would fall under that category though.

Re:Hey, wait a second! (0)

Anonymous Coward | more than 8 years ago | (#14342239)

Mods. Come on. Common sense. That was intended to be funny.

Kreskin confirms (-1, Offtopic)

Anonymous Coward | more than 8 years ago | (#14340444)

Linux is dying like hot grits down Caleb Jeffas pants. Long live OpenBSD

nice (4, Interesting)

gcnaddict (841664) | more than 8 years ago | (#14340452)

This book seems helpful for those who already got their feet wet in the linux seas, but what about for the ultimate linux n00b? I wouldnt mind reading a book which does a good job presenting linux to the absolute beginner.

Re:nice (1)

Tharkban (877186) | more than 8 years ago | (#14340627)

Personally, I would hate reading a complete n00b book on linux. I enjoy knowing the inside out of the command prompt and doing things in glorious piped heaven. I already know where most of my config files are on various distributions. And I know how to man and google to find out the rest. Why would I care about a book which starts from the very beginning. Except, of course, to give as a reference so I don't have to explain it myself to people.

Re:nice (1)

mordors9 (665662) | more than 8 years ago | (#14340705)

I think most of the Distro specific books you can get at a bookstore have that design in mind. Running Linux, was an excellent book to get you started as well, IMO.

Re:nice (4, Informative)

wrecked (681366) | more than 8 years ago | (#14340830)

Books for the ultimate linux n00b? Here ya go, fill your boots:

Also, the major distros have their own manuals, handbooks, wikis, FAQs etc. that cover the basics.

There are plenty of books for that beginners... (1)

Anti-Trend (857000) | more than 8 years ago | (#14341074)

...but these are not those books. If you want an absolute beginner's looks at Linux, check out Marcel Gagné's excellent Moving to Linux, Second Edition : Kiss the Blue Screen of Death Goodbye! [amazon.com] (direct link, no paid click-through). Also worth noting is Paul Sheer's LINUX: Rute User's Tutorial and Exposition [2038bug.com], that while a bit dated covers the basic concepts of Linux quite thoroughly and also makes quite a good reference guide. I would start with the former, but the latter is free online and the paperback inexpensive to purchase.

Simple solution (0)

Anonymous Coward | more than 8 years ago | (#14341952)

Install Windows. It'll make all of those Linux headaches go away.

Re:nice (1)

Budenny (888916) | more than 8 years ago | (#14343613)

How Linux Works, by Ward.
Linux Cookbook, by Schroder

The one explains and gives exercises, the other is a collection of recipes for how to do things.

Proof that free stuff 4 download works (2, Insightful)

djsmiley (752149) | more than 8 years ago | (#14340473)

I know im very offtopic here, but the fact this is out for free download, yet he still manages to publish and see these books, prove that mp3 downloading doesn't really cut into music sales?

Hell i know that more mp3s are downloaded than books, but far less people will buy this book anyway.

Re:Proof that free stuff 4 download works (2, Insightful)

SleepyHappyDoc (813919) | more than 8 years ago | (#14341001)

No. There's a lot more iPods out there than eBook readers (well, I suppose you could put Linux on your iPod and use it to read eBooks, but the vast majority of iPod owners wouldn't even consider doing that). Since reading on a screen often isn't as comfortable as reading a dead-tree book, even with the availability of a free downloadable version, many people will choose the physical version. The same could be said for CDs, but (as of now, and in my opinion) mp3's are still more convenient, when compared to CDs, than eBooks are when compared to dead-tree. If there was a perfect eBook format and reader that was convenient and ubiquitous, then your comparison might have more weight.

Re:Proof that free stuff 4 download works (1)

jpostel (114922) | more than 8 years ago | (#14344735)

The book in question has a PDF download option rather than ebook format. PDF is fairly ubiquitous as far as doc formats go.

That said, I prefer dead-tree reading for books. I have read two books electronically, one on my Treo and one published on the web. I had no problem reading either one, but I can see how people would find it difficult or inconvenient to do it.

KDB (1)

kuchin (902689) | more than 8 years ago | (#14340585)

Who uses KDB??

Re:KDB (2, Informative)

undeadly (941339) | more than 8 years ago | (#14340740)

Who uses KDB??

A kernel is just software and may crash (panic). To get some useful debugging info about the crash (panic message, traceback, and ps output, for instance) you use a kernel debugger. Of course, a kernel debugger is used to more things than just print a call stack in case of a crash, of you are a kernel developer.

On OpenBSD you'll automatically enter the kernel debugger in case of a crash. With output from the debugger it's possible to make a usefulbug rapport. I don't know if the usual Linux kernels have the kernel debugger enabled.

Re:KDB (2, Informative)

GigsVT (208848) | more than 8 years ago | (#14340959)

You get an panic report if it doesn't hang hard (which is often hardware anyway).

If you are in X though, it's hard to see the panic report.

Re:KDB (1)

PugMajere (32183) | more than 8 years ago | (#14341526)

There's no debugger in the mainline kernel, but most things that get flagged as "oops, this shouldn't be happening", generate a stack backtrace, dump registers, decode symbols, show the loaded list of modules, etc.

Probably about the same as what bug reports from reasonably knowledgeable users of OpenBSD collect out of the in-kernel debugger and submit (thouhg I have to admit that I've never even tried to look for an OpenBSD kernel bug report).

Re:KDB (1)

ClamIAm (926466) | more than 8 years ago | (#14341929)

On OpenBSD you'll automatically enter the kernel debugger in case of a crash. With output from the debugger it's possible to make a useful bug rapport.

Looks like OpenBSD is far from dead, just butter up your bugs with some RAM and they'll tell you everything!

where is download (1)

kokoko1 (833247) | more than 8 years ago | (#14340611)

I can't find a download link or to read the series online. I'm getting blind ?

Re:where is download (1)

kokoko1 (833247) | more than 8 years ago | (#14340653)

okay i thinks i got it :), too quick to post or you could say too excited to read these series.

Re:where is download (0)

Anonymous Coward | more than 8 years ago | (#14340655)

its the link that says,"Get your FREE downloads and book information here."
...and i thought my eyes were bad:-)

A Layman's Troubleshooting Guide? (4, Informative)

vectorian798 (792613) | more than 8 years ago | (#14340631)

Is there a troubleshooting guide in general for linux? A lot of newbies are intimidated by the number of hoops they have to jump through for things like setting up sound etc. I know it can be quite frustrating because I recall back when I first installed Ubuntu in different installations of the same version of Ubuntu, different tricks got my sound to work on my laptop. For that matter, is something like this feasible because of the various distros and the difficulty of hardware support? So far, the best one I've found is the Ubuntu Starter Guide [ubuntuguide.org], but it is distro specific...

PS: I've already checked the Linux Documentation Project [tldp.org]

Re:A Layman's Troubleshooting Guide? (2, Informative)

Advocadus Diaboli (323784) | more than 8 years ago | (#14340735)

Working in Linux support for customers I can just give you a hint. The first step toward a successful problem solution is that the one who wants to solve the problem understands the problem. You can achieve this goal by learning How To Ask Questions The Smart Way [catb.org].

Another important thing that other people want to know if they have to solve your problem is all available information about your environment. Tools like hwinfo or sysreport can be of great help to the supporter.

First step: (1)

CarpetShark (865376) | more than 8 years ago | (#14343821)

So far, the best one I've found is the Ubuntu Starter Guide, but it is distro specific...
Troubleshooting step 1: Use a debian-based distro, preferably (k)ubuntu.

Yeah... (-1, Offtopic)

Anonymous Coward | more than 8 years ago | (#14340711)

but where are the troubleshooting books for Windows?

Forgotten resource of information not mentioned... (2, Insightful)

undeadly (941339) | more than 8 years ago | (#14340834)

is the man pages and other system documentation. Correct, updated and relevant man pages/documentation is a gold mine for trouble shooting and configuration.

Re:Forgotten resource of information not mentioned (1)

CarpetShark (865376) | more than 8 years ago | (#14343842)

Correct, updated and relevant man pages/documentation is a gold mine for trouble shooting and configuration.
Unfortunately, the linux manpages are neither correct, nor updated. Just go compare a freebsd manpage to a linux one. The difference in quality is staggering. Despite being really busy, I've thought about re-writing them myself a few times, even though I barely know the details necessary. That says something about the quality in and of itself :(

Re:Forgotten resource of information not mentioned (1)

andyfaeglasgow (782943) | more than 8 years ago | (#14361660)

Obviously not so busy that you don't have time to post on /.

Which manpages are incorrect? Have you informed the authors? Do you know how to make a patch?

Re:Forgotten resource of information not mentioned (1)

CarpetShark (865376) | more than 8 years ago | (#14361690)

Don't be a dumbass please. I contribute plenty. And I'm not going to justify having a few minutes to post while keeping up with the community.

Wheres the electronic version? (1)

MrBandersnatch (544818) | more than 8 years ago | (#14340924)

I thought Id give it a shot....only to find this listed as a title thats "coming soon". Am I missing a link here?

Troubleshooting Tip (1)

Dpaladin (890625) | more than 8 years ago | (#14340969)

fsck Congratulations, you now know how to solve most of the problems you will ever have with Linux.

you missed a N there ... (1)

schmu_20mol (806069) | more than 8 years ago | (#14342119)

fsck Congratulations, you now know how to solve most of the problems you will Never have with Linux.

I don't recommend this one too highly. (2, Interesting)

theendlessnow (516149) | more than 8 years ago | (#14341186)

This book sounds like a troubleshooting book. 99.99999999999% of all problems in Linux can be diagnosed, guessed at and determined without the things mentioned in this book.

If you are a very experienced user... this could be interesting. If you don't think yourself to be a very, very, VERY experienced Linux user, this isn't your book. Even so, again, most things are determined through much, much, much easier techniques.

Not a great troubleshooting book IMHO. Good book for someone who thinks they know it all.

See my review at: http://theendlessnow.com/ten/Main/BookReviews#toc9 [theendlessnow.com]

Save SEVENTEEN ($17) bucks! (-1, Troll)

Anonymous Coward | more than 8 years ago | (#14341537)

Save yourself $17.60 by buying the book here: Linux Troubleshooting [amazon.com]. And if you use the "secret" A9.com discount [amazon.com], you can save an extra 1.57%!

Re:Save SEVENTEEN ($17) bucks! (1, Informative)

Anonymous Coward | more than 8 years ago | (#14341965)

Why is this marked as troll? The parent is absolutely correct. Amazon's price is $17 bucks cheaper than B&N. Perhaps Slashdot should consider posting alternative links to Amazon in their book reviews now? I thought the "boycott" was over a while ago.

Best Linsux Troubleshooting tip is... (1)

kirk26 (811030) | more than 8 years ago | (#14342194)

oops, it doesn't work. Format. Reinstall Windows! Problem solved!

A good book and a good review (0)

Anonymous Coward | more than 8 years ago | (#14342303)

I have also bought this book for myself. And it is everything what the reviewer says it is. In fact, I decided to buy this book after reading a review of the book at http://linuxhelp.blogspot.com/2005/12/book-review- self-service-linux.html [blogspot.com]. And boy! I was able to set right quite a few problems related to softwares after reading this book. I strongly recommend buying this book.

linuxtroubleshooting.com (1, Informative)

Anonymous Coward | more than 8 years ago | (#14343073)

see also http://www.linuxtroubleshooting.com/ [linuxtroubleshooting.com]

Another site about linux troubleshooting techniques and
strategies. Unaffiliated with the book.

For Linux to reach critical mass (0)

Anonymous Coward | more than 8 years ago | (#14343123)

...we should publish a similar book for the n00bies, the cheerleaders, the lingerie models, and the occasional receptionist.

I thought I'd throw in the last three, you know, to bring some sort of balance of nature, and stuff.

Is that it?? (1)

SupremeTaco (844794) | more than 8 years ago | (#14343169)

In MY day, we just used a good old logic probe, and an analog meter. Now THAT'S troubleshooting.

Luser: My spreadsheet won't open.
Tech: Grab the big blue capacitor, and call me back. . . .

Re:Is that it?? (3, Funny)

Macka (9388) | more than 8 years ago | (#14343322)

A logic probe .. pah! Real hackers use a magnet. Now THAT'S troubleshooting. ;-)

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account