×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Beginning Portable Shell Scripting

samzenpus posted more than 5 years ago | from the read-all-about-it dept.

Unix 186

Joe MacDonald writes "The earliest UNIX shell I encountered was the Bourne shell on a SPARCStation 2 at my university. As with many students of my generation, prior to that nearly all of my exposure to command line interfaces was some variant of DOS. I was quite proficient with the primitive scripting language that was available on such platforms but I immediately felt far out of my depth in this new environment. The commands seemed arcane, possibly dangerous, and almost immediately I regretted stepping into this unfamiliar wilderness without some sort of guide." Read below for the rest of Joe's thoughts.

It was probably a few weeks after that first, rough introduction that I returned for another round with this strange but somehow seductive tool, armed with a book I'd found and a determination to learn it's secrets. I had no idea then that seventeen years later I'd still be learning new tricks, discovering new features and taking so much pleasure from sharing what I've learned with others. In fact, in those early forays into the realm of shells and scripting, I didn't even really have a strong concept of the separation between the shell and the operating system, so at the time I couldn't have conceived of how much fun I would have in later years discussing and debating the relative strengths and weakness of shells with friends and colleagues, but it is probably my favorite touchstone of computer geek conversation. Discussion of shell features, scripting tricks and semantics almost always result in my learning something new and interesting and having a new tool to add to my collection.

Peter's book, Beginning Portable Shell Scripting, therefore may sound like something intended as a gentle introduction, aimed at the initiate — the sort of text I'd been seeking to carry with me when I first attempted to write what I thought of as "batch files" on that now-ancient UNIX machine — but there's more truth in the subtitle, From Novice to Professional, than one might expect. He writes in an accessible, at times conversational, style and presents detailed technical information alongside a mixture of anecdotes and historical detail that does more than simply serve as a technical reference, it helps the reader understand a great deal about why things are the way they are. It was such an entertaining read that I frequently found myself skipping ahead, reading a section I knew was coming up, then resisting the urge to just keep going from that point. The first of these I encountered on page 18 in which he discusses the relative portability of printf in shell scripts. I knew what he knew, it's clearly non-portable and should be avoided, and thoroughly enjoyed the explanation of how he determined his (and by extension my) assumption was in error. Another on page 108 is the sort of good advice all UNIX users, not just those aiming to write good scripts, should take to heart. Many times, though, I've related precisely the same advice to colleagues to be met with confused stares, so it certainly bears repeating.

This book is a desktop reference in the truest sense of the term for me, it is an interesting, at times laugh-out-loud amusing, discussion of how to write shell scripts that will work on the widest possible range of Bourne-derived and POSIXly correct shells and why this is a desirable goal. In true UNIX tradition, the author doesn't provide simply a set of rules, but guidelines that will help you find your own way through the task of creating portable, maintainable shell scripts.

The real meat of the book begins in Chapter 3 (more on Chapter 2 in a moment) with a discussion of control structures and redirection, the latter being perhaps the defining characteristic of UNIX command line interfaces. I struggled somewhat with trying to decide if redirection would be better discussed after the material on how the shell parses tokens, presented in the first part of Chapter 4, but it does seem that the correct logical grouping is the one presented. It would be easy to get lost, for example, in the semantics of why the same streams of redirection tokens behave differently on different shells, but the key concept in the early chapters is that of many tools, each doing a specific task, working in concert. That objective is achieved quite effectively.

Chapters 5 and 6 go into detail (possibly too much for some, just right in my opinion) on how UNIX executes shells and how shells can spawn other shells, the costs and the benefits and the available alternatives for one to make an informed decision. Frequently there isn't one right answer whether some activity is better done in a script, in a shell function or in a subshell, but the material here will certainly aid in making those determinations. My personal bias being almost always toward writing a shell function — perhaps an indication I've had too much exposure to C programming, perhaps more due to a frugal upbringing and my own sense that spawning a whole new shell to do something is overkill — had me wishing for a larger section on the value of such constructs, but there should be enough there for me to win some converts to my cause.

By far the sections I learned the most from, however, would be Chapter 7: Shell Language Portability and Chapter 8: Utility Portability since I actively avoid exposure to other shells. I have my two preferred options and a third that I will use when presented with no alternative. While this does mean I know "my own" shells very well, it also means that I often bump into the furniture, so to speak, when I find myself using a new shell. These chapters haven't been immediately useful to me, but I know they're the ones that I'll be turning to in the future, I've needed something like them in the not-too-distant past, after all.

The final three chapters assemble the information presented in the earlier sections and suggest a sort of "best practices" approach to writing scripts. Concepts like "degrade gracefully" seem like pretty fundamental ideas when you hear them but I frequently find myself writing functions or scripts that don't do that at all when intended for a limited, usually singular, audience. It may seem like an okay idea when you're doing something for your own use, but when you write a complex function that works then discover a bug in it two or three years late and you have to return to fix it, it can be just as helpful for it to simply fail in an informative way as it would be to have detailed comments explaining the intent and the mechanics.

Truly, there's something here for everyone. In my office I'm considered something of an expert when it comes to complex regular expressions and the subtleties of using them in different editors and tools, but Chapter 2 and Appendix C both had enough new material in them that I found myself frequently making notes in the margins.

I have many, many books in my bookshelf in my office but nearly none on my desk. Beginning Portable Shell Scripting is going to be one of the very few that will be spending a great deal of time lying flat on my desk, in easy arm-reach.

You can purchase Beginning Portable Shell Scripting from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

186 comments

portable shell scripting is an oxymoron (1, Flamebait)

rlseaman (1420667) | more than 5 years ago | (#26815879)

Even if a shell script is portable, the underlying OS will always fail to be for any task significantly complex to do useful work.

Re:portable shell scripting is an oxymoron (1)

MrEricSir (398214) | more than 5 years ago | (#26815977)

Who needs to invoke the OS when you can calculate Pi with only a mere batch script?
http://thedailywtf.com/Comments/Stupid-Coding-Tricks-A-Batch-of-Pi.aspx [thedailywtf.com]

Re:portable shell scripting is an oxymoron (1)

MrEricSir (398214) | more than 5 years ago | (#26816029)

That link should really be
http://thedailywtf.com/Articles/Stupid-Coding-Tricks-A-Batch-of-Pi.aspx [thedailywtf.com]

(I should write a shell script to correct my comments.)

Re:portable shell scripting is an oxymoron (2)

jellomizer (103300) | more than 5 years ago | (#26815985)

Care to elaborate?

I have seen some rather complex scripts that are portable that do some useful things.

Re:portable shell scripting is an oxymoron (3, Interesting)

rlseaman (1420667) | more than 5 years ago | (#26816597)

As usual, it comes down to use cases. Describe the useful things that are done. One might choose to write a shell script to perform some pure mathematical utility function, but this certainly isn't the usual role of such scripts. Rather, one uses a shell script when accessing files (logs, etc.) on the disk, or when opening sockets, or when spawning host level commands. Other device level access is often required, for instance, a local or UTC clock might be consulted, requiring knowledge of timezones. All of these are very dependent on whether the script is running under a BSD or Sys-V flavored OS (assuming Unix, of course), or even on micro-versioning of specific OSes.

The question of complexity is also not some challenge to find a lengthy script, but rather a statement of the inherent design-level complexity of the application.

Re:portable shell scripting is an oxymoron (3, Informative)

Fred_A (10934) | more than 5 years ago | (#26817235)

As usual, it comes down to use cases. Describe the useful things that are done.

Take a naked box and boot it (/etc/init.d/*) ?
I know it's a bit trivial but it still qualifies as useful in my book.

Actually /etc is just pretty much a collection a collection of fairly useful shell scripts. I've always found it interesting that Unix was mostly held together by /bin/sh (aka /bin/bash on a lot of systems nowadays) and spit. And that it worked.

To take one of the posts above where the poster had been exposed to DOS. The DOS system (although it wasn't really a system, merely a program loader) was configured by the autoexec script. All the Unix do the same with a number of chained scripts (and their order can even dynamically change nowadays) all running sh (or an extended version of it).

I still wonder at it sometimes. It's simple and accessible on one side. And it can degenerate into an awful mess on the other :) (less so nowadays thankfully)

More ?

Anyway, wanted something useful the shell could do ? How about run the whole operating system (find a service that isn't actually handled by a !#/bin/sh script...).

Re:portable shell scripting is an oxymoron (1)

rlseaman (1420667) | more than 5 years ago | (#26817419)

Booting a computer is almost the definition of non-portable. Not sure what you mean by a naked box, but there are all the device drivers to load, clocks to set, file systems to mount, services to start. These are precisely the things that vary from one flavor of Unix (or Linux) to another. The script itself may be very portable, but the underlying description of the resources - or even the description of this description - will not be portable

I'm a bit at a loss for why my message has been labeled flamebait :-) These assertions seem unremarkable. Personally, I'm all for writing portable code, whether in an interpreted or compiled context. But the real challenge to portability lies in the libraries you call (the OS itself for shell scripts), not in the language you write the code in.

Re:portable shell scripting is an oxymoron (3, Informative)

seebs (15766) | more than 5 years ago | (#26818567)

Portability isn't boolean.

I wrote a wrapper around cdrecord to clean up the UI, automatically handle things like creating an isofs from directories, and so on.

It's not 100% portable; every new system, I change the path to cdrecord, the device spec for the CD drive, and the command used to eject a CD.

Everything ELSE stays the same, and I don't need to remember how to use mkisofs, or anything like it. Directories, bzipped images, whatever; it gets burned correctly. I win.

If the script were not written in otherwise-portable shell, it might not work on the broad variety of boxes I've wanted to use it on.

I've done scripts to handle tasks like "open this file" (not as flexible or smart as the OS X one, but quite good about various compressed tarballs and archives). Surprisingly portable.

I have a script for the idiom of "for every file named or provided as standard input, run it through this filter in place". Repeating commands at intervals, for a given number of times, until they fail... Tons of little utilities like this that save me time.

If you want complete applications with no dependencies, that's harder to find. That said... Have you ever used autoconf to configure something? That's a fair bit of portable shell right there...

Re:portable shell scripting is an oxymoron (1)

rlseaman (1420667) | more than 5 years ago | (#26818809)

I had to laugh. I wrote a similar wrapper script for cdrecord. This may say more about mkisofs and the management of recordable media than it does about scripting :-)

This script (well, I broke it into the obvious separate steps of building the ISO image and dumping it to duplicate media) was orchestrated from within a Unix daemon called as a BSD lpd filter. Whatever else it does, lpd serves as a reasonable queuing manager. A humorous but pragmatic choice.

That was an evolution of a system for archiving digital imaging data onto exabyte tapes. The original tape-based system was ported after a few years from SunOS to Solaris, that is, from BSD lpd to Sys-V lpsched. I balked at repeating the port for optical media, very much due to portability issues unrelated to scripting. Scripting is the least of your worries with portability.

The digital archive has long since evolved to a system with a replicated online (hard drive based, that is) architecture. Much more portable (non-boolean, as you say), but certainly not 100%

Re:portable shell scripting is an oxymoron (5, Interesting)

dgatwood (11270) | more than 5 years ago | (#26817081)

Well, there is some truth to the GPP's comment. Linux and Mac OS X don't even agree on how to tell echo not to print a newline or how to enable extended regular expression mode in sed. May heaven help you if you want to do something as esoteric as creating or mounting a filesystem, creating or mounting a disk image/ramdisk, talk to a USB device in any way, get a list of processes in any useful way, etc. There's a very big lack of standardization in a lot of things you might like to do with scripts, in other words. The Single UNIX Spec and POSIX are not quite sufficient, but more annoyingly, most OSes (Linux, *BSD) out there don't even come close to conforming to it, so you end up with this dichotomy between BSD behavior and AT&T behavior.

That said, a lot of things are standardized, and many others can be worked around with clever use of variables (or possibly eval in a few extreme cases). I've written chapters on the subject myself. The big things you need to remember are that $(( $FOO + 3 )) is not portable, nor for ((...)), nor >&, nor anything involving extended regexp except using Perl, that even "the one true awk" is not quite SUS-compliant, GNU awk doubly so, bash triply so, that you should use printf instead of echo for output if you don't want newlines, that signal numbers are not portable (for trap), that proper quoting of arguments is crucial, and that you need to work with the bare minimum base behavior of utilities (using few or no flags) if you expect any hope of portability without needing to make platform-specific changes.

For some quick examples of some interesting portability issues, read some of my comments in the games at shellscriptgames.com or search for the word "compatibility" in Apple's "Shell Scripting Primer". It's a real eye opener to see how many portability problems exist even for fairly simple shell scripts.

Re:portable shell scripting is an oxymoron (1)

steelfood (895457) | more than 5 years ago | (#26818105)

From my experience, shell scripts that use a significant amount of non-shell builtin commands are not portable. The typical shell script is highly dependent on the awk, grep, sed, etc. version on the system. And that varies not only between platforms, but between OS versions.

Simple things, like the improved mv that's floating around, tend to be easier to port. But the chance of a successful port is inversely proportional to the complexity of the script. As well, usefulness, while a loaded term, tends to be proportional to simplicity.

Your experience might be with long scripts (not to be mistaken with complex scripts) that predominantly make use of shell builtins to do all of the work, or with a relatively homogenous network environment. But your situation if true, from my experience, is rather unique.

Re:portable shell scripting is an oxymoron (1)

code4fun (739014) | more than 5 years ago | (#26816433)

Check out GNU autoconf. That's a good example of how a script works on *nix box. And, yes, they are useful!

Re:portable shell scripting is an oxymoron (2, Informative)

ultrabot (200914) | more than 5 years ago | (#26817751)

Check out GNU autoconf. That's a good example of how a script works on *nix box. And, yes, they are useful!

Autoconf is also a horrible peace of crap. One of the better reason to hate the concept of shell scripts, actually.

Check out SCons for comparison:

Program('hello.c')

or

SharedLibrary('foo', ['f1.c', 'f2.c', 'f3.c'])

And that's pretty much it. I'm not sure all the horrors required by autoconf would fit into a slashdot posting.

Re:portable shell scripting is an oxymoron (0)

Anonymous Coward | more than 5 years ago | (#26817845)

Hmm... did someone re-invent imake?

Re:portable shell scripting is an oxymoron (1)

ultrabot (200914) | more than 5 years ago | (#26817985)

Hmm... did someone re-invent imake?

No, SCons does not generate makefiles.

SCons here was just an example to illustrate how much Automake sucks. Obviously we have other systems like CMake as well.

Re:portable shell scripting is an oxymoron (1)

dgatwood (11270) | more than 5 years ago | (#26818361)

I'm not sure all the horrors required by autoconf would fit into a slashdot posting.

I think it could fill an entire book.

  • Start with all the custom extensions for things like GNOME and KDE that each have their own custom syntax instead of coming up with a single syntax that works reliably, and many of the modules aren't included universally with various OSes and/or distros, so trying to rebuild the "configure" script with autoconf results in utter failure unless you have all the various development packages installed, including pieces that are optional.
  • Next, add the fact that about half the Autoconf modules I've seen don't respect shell variables like $CFLAGS and $LDFLAGS, so you end up with a dozen different --with-foo-lib directives that are specific to each individual project to force it to look in the right place if it isn't where the original Autoconf script author expected it (and even then, sometimes I end up having to hack the Autoconf script and manually splice in extra arguments in the compile directive to make it work).
  • If you ever need to compile with a different compiler, you're in an even bigger world of hurt, since most AC modules don't seem to care about the value of $CC.
  • Oh, yes, and if you need to pass linker flags, you'll often have a great deal of fun if those flags aren't supported natively by gcc.

And so on. Anybody who has ever used Autoconf at any significant level could probably go on for weeks....

Autoconf is the antithesis of portability except when porting to platforms to which the software has already been ported, and then really, what's the point? So you save a couple of extra tiny makefiles, but you end up with something that's an utter bear to port to any platform instead of having a makefile for BSD that you can tweak trivially and port to a similar platform. When I'm porting something to a new platform, I'd much rather get a Makefile with neatly laid out variables at the top for things like FOOLIBDIR, BARLIBDIR, etc. and modify it by hand than get an Autoconf script (no matter how well written) any day of the week.

Bah, dangerous? (1)

sepelester (794828) | more than 5 years ago | (#26816021)

I was behind the comfortable wall of a multi user system even with disk quotas. We had access (read only) to our backups and pretty much coudn't screw up. Bourne was standard but I used zsh and later switched to bash. Man pages was all we had to study and we competed in learning as much as possible, writing cool scripts, exploiting the system in various ways. The system was rooted and sabotaged a couple of times but nothing in it was critical in any way. A playground really. I remember the tingling feeling of the 'su' command, the only really dangerous one. Nice that they write books about this but nothing beats the good old feeling of having worked HARD to get the knowledge. The cry of success is so much louder.

If you have a choice... (4, Insightful)

ultrabot (200914) | more than 5 years ago | (#26816049)

Don't do it.

Shell scripts have horrible error handling, and quickly become a maintenance nightmare. These days, e.g. Python is installed everywhere you need to go.

Just do this:

def c(s): os.system(c)

and you have mostly covered the area where shell scripts excel. You can still write minimal "shell scripts" inside c().

Unluckily, you still *need* to grok shell scripting to some extent, or at least be able to read them. Just don't write them if you can help it.

Re:If you have a choice... (4, Insightful)

ncw (59013) | more than 5 years ago | (#26816131)

I agree!

My personal limit is 10 lines for a shell script. If is longer than that I convert it to Python.

Python scripts have the advantage that they work on Windows too, and they have lots of os independent abstractions for file names, processes etc.

Why learn an arcane language like sh when you can learn a nice well structured language like Python and write better scripts?

A few years ago I would have used Perl rather than Python, but I'm converted now ;-)

Re:If you have a choice... (2, Insightful)

cmdr_tofu (826352) | more than 5 years ago | (#26817087)

The tradeoff is that your scripts have a huge memory footprint now. I loathe shell script as much as the next person, but find it necessary when I am working in a minimal busybox-type environment. Perl/Ruby it is whenever I have the chance.

Re:If you have a choice... (2, Insightful)

Tikkun (992269) | more than 5 years ago | (#26817749)

Why learn an arcane language like sh when you can learn a nice well structured language like Python and write better scripts?

Where I work pretty much everything has bash already (I install cygwin on all the Windows boxes. Of course, Python is usually there too :) ).
If you already have a bash script (or find one via the Google), changing it is usually simpler than porting it to Python.
If you work with people that already know bash scripting but don't know Python using the lowest common denominator can be easier than training.
There is less memory overhead for a simple shell script than there is for a simple Python script.

This being said, I try to keep my bash scripts under 40 lines and port them to python if they get more complex and I know the machine I'm going to be using them on has Python.

Re:If you have a choice... (1, Interesting)

Anonymous Coward | more than 5 years ago | (#26818511)

Why learn an arcane language like sh

Because sometimes it is exactly the right tool for the job. There are many tasks for which python or perl are overkill. Specifically, things which could be accomplished interactively on the command line (like repetitive tasks) but are better automated. Writing shell scripts also makes you more proficient on the command line, which is a valuable skill for any programmer or sysadmin. Your knowledge of command line tools and where/when to use them will surely increase.


#!/bin/sh

cat musicbrainz.xml | sed 's/>/>\n/g' | grep 'title>$' | sed 's/musicbrainz.txt

This is a quick and dirty script I use to convert a musicbrainz [musicbrainz.org] XML file (found on the details page of each album) to straight text. It's part of a collection of scripts which make up my custom tagging and music management system. It works great. But imagine what the equivalent script would look like in python or perl. A lot longer, for starters.

Re:If you have a choice... (0)

Anonymous Coward | more than 5 years ago | (#26818995)

Sorry, that should be:

cat musicbrainz.xml | sed 's/>/>\n/g' | \
grep 'title>$' | sed 's/</\n</g' | \
grep -v '^<' | grep -v '^$' | sed '1d' >musicbrainz.txt

Re:If you have a choice... (0)

Anonymous Coward | more than 5 years ago | (#26816321)

You mean: def c(s): os.system(s)

Re:If you have a choice... (3, Insightful)

$RANDOMLUSER (804576) | more than 5 years ago | (#26816393)

Shell scripts have horrible error handling, and quickly become a maintenance nightmare. These days, e.g. Python is installed everywhere you need to go.

That's exactly what the Perl people said years ago, and we all know how well that worked out (for low maintenance sysadmin-type tasks). I know the sinking feeling I get every time I find a crontab entry pointing to a Perl script.

Re:If you have a choice... (-1, Flamebait)

Anonymous Coward | more than 5 years ago | (#26817179)

That's exactly what the Perl people said years ago, and we all know how well that worked out (for low maintenance sysadmin-type tasks). I know the sinking feeling I get every time I find a crontab entry pointing to a Perl script.

I have a similar bowel-loosening feeling I get when I find a crontab entry pointing to a Python script. It appears the inventor of Python wiped out on his BMX bike, and now thinks everyone should ride Tricycles.

The difference between me "the Perl guy" and you "the Python guy" is that I won't force my way into your home to tell you how much I hate your carpet.

Re:If you have a choice... (1)

swillden (191260) | more than 5 years ago | (#26817395)

Shell scripts have horrible error handling, and quickly become a maintenance nightmare. These days, e.g. Python is installed everywhere you need to go.

That's exactly what the Perl people said years ago, and we all know how well that worked out (for low maintenance sysadmin-type tasks). I know the sinking feeling I get every time I find a crontab entry pointing to a Perl script.

There is a big difference between Perl and Python. While you can write readable Perl, or write horribly opaque Python, there's a reason that Perl has a reputation as a write-only language and Python has a reputation for being quite readable.

Then there's the guy I worked with years ago who wrote (writes, I'm sure) all of his scripts in elisp...

Re:If you have a choice... (3, Insightful)

Lord Ender (156273) | more than 5 years ago | (#26818703)

That's not what they said years ago. People arguing to use Perl instead of Bash did so because Perl was just far more functional. Perl and Bash both have pretty terrible maintainability, but Perl is a million times more functional.

Python and Ruby have the functionality of Perl without the maintenance issues inherent in a language which is really a hodge-podge of ancient unix idioms.

Re:If you have a choice... (2, Interesting)

Etherized (1038092) | more than 5 years ago | (#26816399)

I can't agree more. I switched over to python for all non-trivial scripting a couple of years ago, and I find it much more pleasant. I even sometimes use iPython instead of bash when I know I'll need to do something complex interactively.

By the way - if you like using python to control systems, you might also enjoy the func project [fedorahosted.org] .

Re:If you have a choice... (1)

JamesP (688957) | more than 5 years ago | (#26816509)

To me it's mixed feelings there

Even though python is very easy, you can't beat the ease of use of find / sed / others

And in python you would need to go to popen/pclose usually, to get the output and return values

Re:If you have a choice... (4, Informative)

digitalhermit (113459) | more than 5 years ago | (#26816623)

Python is nice, but hardly installed everywhere. It's available on Linux certainly, but not always on AIX or Solaris. Yes, it is just an installation away, but many of the systems I maintain require change management procedures to even chmod a file.

Shell scripts do have decent error handling for what they need to do. With traps and proper usage of error codes, they are not much different from lower level languages.

I'd agree that I now *prefer* to write longer scripts in Python. However, few of the people I work with know Python, or even Perl. They can get around with korn and bourne as these are the default scripting languages on more traditional Unix systems.

Which comes down to the gist of the issue. Do you write code in a language you prefer or one that can be maintained by the admins? I'd argue that it doesn't matter what language you use. If you write poor code in shell you will likely write poor code in Python too.

Re:If you have a choice... (1)

bol (152634) | more than 5 years ago | (#26816819)

While I completely agree that Python is a fantastic system admin tool and a very suitable replacement for almost all shell scripting I disagree that shell scripts have to have horrible error handling.

"if [ $? != 0 ]; then" goes a long way and you get bonus points for wrapping it in a function.

There are some tasks that belong in a shell script and some that belong in a programing language like Python. It's all about the right tool for the job.

Re:If you have a choice... (1)

ultrabot (200914) | more than 5 years ago | (#26817665)

"if [ $? != 0 ]; then" goes a long way and you get bonus points for wrapping it in a function.

What I like about the python way is that you don't have to write any of that - if something fails, the whole function fails (unless you specifically catch the exception). No need for explicit error handling to clutter your scripts.

Re:If you have a choice... (0)

Anonymous Coward | more than 5 years ago | (#26818023)

Shell scripting is fun

If you don't know how to write error handling yourself, then your not much of a coder. No matter what language... Its called, write the code. Test the code... Check the values of your returns and variables. And then modify your code.

Seriously, I have never had issues with error handling... And I've writting log parsing routines in nothing but Shell scripts.

Re:If you have a choice... (2, Interesting)

Haeleth (414428) | more than 5 years ago | (#26818217)

These days, e.g. Python is installed everywhere you need to go.

No it isn't. Not even remotely close.

And in most cases the response to a request to install an entire programming language would be flat rejection, turning to raucous laughter when they realise you only want it because you don't like any of the several scripting environments that are already available.

Re:If you have a choice... (1)

ultrabot (200914) | more than 5 years ago | (#26818445)

These days, e.g. Python is installed everywhere you need to go.

No it isn't. Not even remotely close.

And in most cases the response to a request to install an entire programming language would be flat rejection, turning to raucous laughter when they realise you only want it because you don't like any of the several scripting environments that are already available.

Sounds like a rather dysfunctional working environment.

I had something like this in mind when I said "everywhere you need to go" (as opposed to just saying "everywhere").

Re:If you have a choice... (2, Informative)

fjollberg (1061600) | more than 5 years ago | (#26818355)

> These days, e.g. Python is installed everywhere you need to go.

Sorry, but no, it isn't.

Re:If you have a choice... (3, Informative)

0xABADC0DA (867955) | more than 5 years ago | (#26818961)

Shell scripts have horrible error handling, and quickly become a maintenance nightmare. These days, e.g. Python is installed everywhere you need to go.

Python doesn't help much over shell scripts without extra libraries, which may or may not be present on any given system.

Python has changed incompatibly several times already.

Python has a large startup overhead:
    20 seconds: 1000x python -c 'print("test")'
    2 seconds: 1000x sh -c 'echo test'

Python is clumsy to use for gluing several programs together.

Python is not the same syntax as the shell. If you don't learn the shell then your day-to-day command lines are gimped.

So Ruby or Python or anything else is better for writing actual programs that do anything complicated, but there are plenty of appropriate uses for shell scripting. Ruby is actually much better... since it has a sensible syntax you could make a rubysh that wouldn't suck.

Python still can't replace quick scripting (3, Insightful)

Chris_Jefferson (581445) | more than 5 years ago | (#26816067)

While I find anything I want to be stable and distributable I now write in Python, I still can't resist pulling out shell scripting, and a splattering of grep, awk, find, mv and xargs to do 95% of the simple pushing around and chopping up of files I find myself doing every day.

I find shell scripting have a nasty habit of not working quiet right when moved between Linux, the BSDs and Mac to be safe, and it's always a pain to write scripts that work correctly with spaces in file names.

Why isn't there (or is there?) a simple python cheat guide, or library, that do the same things as grep, awk, find, mv and xargs?

Re:Python still can't replace quick scripting (3, Informative)

ultrabot (200914) | more than 5 years ago | (#26816269)

Why isn't there (or is there?) a simple python cheat guide, or library, that do the same things as grep, awk, find, mv and xargs?

re.findall, s.split(), os.walk, shutil.move,
" ".join

One area Python still falls short (1)

clawsoon (748629) | more than 5 years ago | (#26818755)

I've run into one big problem replacing find/xargs in Python: There's no good equivalent of the find '-print0' and xargs '-0' options. Efficient iteration over stdin (or file) lines in Python can *only* use the regular line separator. If you want to write a backup script that uses find instead of os.walk for speed and pump it into Python for convenience, everything breaks as soon as you hit a filename with a newline in it.

(And no, trying to enforce a "no newlines in filenames" policy doesn't work. Been there, tried that.)

Re:One area Python still falls short (1)

ultrabot (200914) | more than 5 years ago | (#26819019)

I've run into one big problem replacing find/xargs in Python: There's no good equivalent of the find '-print0' and xargs '-0' options.

This seems to work:

fs = os.popen('find -print0').read().split('\0')

Re:Python still can't replace quick scripting (1)

ultrabot (200914) | more than 5 years ago | (#26816373)

Why isn't there (or is there?) a simple python cheat guide, or library, that do the same things as grep, awk, find, mv and xargs?

Forgot to mention - no need to do it all in raw python. Just call out to shell when you feel like it, if you are sure it can stay unix-only. You can do wonders with some shell invocation + os.popen(...).read()

Re:Python still can't replace quick scripting (0)

Anonymous Coward | more than 5 years ago | (#26816861)

it's always a pain to write scripts that work correctly with spaces in file names

Well, the only time you would run into those is if you're dealing with a Windows filesystem. That is, unless you use spaces in filenames on a unix-type system. You wouldn't do such a nasty, dirty thing, would you? ;)

Re:Python still can't replace quick scripting (0)

Anonymous Coward | more than 5 years ago | (#26817265)

I was just getting ready to say, Python is essentially a shell scripting language on steroids. I don't write bash scripts anymore, and its nicely portable between various *NIXes. It's a nice replacement for batch and vbs on windows too.

Hardly Need a Whole Book (0)

Anonymous Coward | more than 5 years ago | (#26816081)

Want your shell to be portable? Write it in Korn Shell. You will find this shell on 15 year old *NIX boxes and the script will still work with bash on Linux.

Re:Hardly Need a Whole Book (0, Offtopic)

MrEricSir (398214) | more than 5 years ago | (#26816111)

Korn had some cool videos, but I was never a fan of their music.

Re:Hardly Need a Whole Book (1)

multisync (218450) | more than 5 years ago | (#26817267)

Korn had some cool videos, but I was never a fan of their music.

But you gotta love that mic stand [hrgiger.com] H.R. Giger created for Jonathan Davis.

Re:Hardly Need a Whole Book (4, Informative)

CannonballHead (842625) | more than 5 years ago | (#26816323)

It's true. I work with a guy (rather old himself) that writes on the Korn shell because it's the only shell that is included on pretty much all Unix based OSs, including Linux. (and Solaris, HP-UX, and AIX, which we also use).

Re:Hardly Need a Whole Book (-1, Troll)

Anonymous Coward | more than 5 years ago | (#26816565)

I'd eat the korn out of natalie portman's asshole!

Re:Hardly Need a Whole Book (2, Insightful)

seebs (15766) | more than 5 years ago | (#26816827)

Sounds great.

Now, off the top of your head, what happens to variables set in the last component of a pipeline in ksh? Do you know whether it's the same on systems where "/bin/ksh" is actually pdksh? ... Oh, and just for reference, about half the Linux systems our IT department installs don't have ksh. No, I don't know why. (I only know because I can't log into them because my default login shell is /bin/ksh...)

Re:Hardly Need a Whole Book (1)

wkcole (644783) | more than 5 years ago | (#26817857)

Want your shell to be portable? Write it in Korn Shell. You will find this shell on 15 year old *NIX boxes and the script will still work with bash on Linux.

Except when it doesn't.

There are differences between bash and ksh, and between different versions of ksh. In some cases the difference means syntax errors, but sometimes it can be more subtle and a script written for ksh runs just fine but doesn't do in bash what it did in ksh, or doesn't do in pdksh what it does in ksh88.

It is also not the case that ksh exists everywhere. It wasn't on MacOS until 10.4. It isn't on FreeBSD by default. It isn't on many Linux boxes, particularly the small network devices that use stripped-down systems. If you don't work in a broad variety of systems you may not need to understand the issues of shell portability, but if you don't understand them you aren't really prepared to work on different sorts of systems.

Shell scripts are a glue language (5, Insightful)

pieterh (196118) | more than 5 years ago | (#26816101)

One does not write a web server in Bash, one wraps a webserver in it, pipes its output to a log analyzer, restarts it automatically if it crashes, and so on.

The most important part of any UNIX-derived shell langauge is not its syntax or power but the fact it lets you construct large ad-hoc applications out of a toolbox of tens of thousands of pieces.

This is where all other operating systems (that I've ever used, and that's 30-40) have failed.

Any serious developer should know several glue languages, Unix shells being the most flexible and accessible.

Re:Shell scripts are a glue language...to some (0)

Anonymous Coward | more than 5 years ago | (#26816743)

One does not write a web server in Bash, one wraps a webserver in it, pipes its output to a log analyzer, restarts it automatically if it crashes, and so on.

Well said, if you limit your scope to publishing web pages. Many servers have other insignificant purposes, but hope to grow up to be web servers someday.

Re:Shell scripts are a glue language (-1, Troll)

Anonymous Coward | more than 5 years ago | (#26816765)

Unix shells? Most accessible? Is that sh, bash, csh, tcsh, ksh, or one of the other arcane shell scripting languages that's "most accessible"?

dom

Re:Shell scripts are a glue language (4, Funny)

seebs (15766) | more than 5 years ago | (#26817057)

Wow. That's a really brilliant question.

Wouldn't it have been cool if someone had written a book on the common ground of the major shells covering how to use them as a single highly-accessible and universally-available language? :)

Re:Shell scripts are a glue language (4, Interesting)

dgatwood (11270) | more than 5 years ago | (#26817263)

One does not write a web server in Bash

I accept your challenge. :-D

But seriously, yeah, you're absolutely right. Ooh, but a basic web server written as a Bourne shell script called by inetd would be so freaking cool....

Oh, no. Somebody actually did that [debian-adm...ration.org] .... Yikes! Now I'm scared.

Re:Shell scripts are a glue language (1)

ClosedSource (238333) | more than 5 years ago | (#26817707)

"The most important part of any UNIX-derived shell langauge is not its syntax or power but the fact it lets you construct large ad-hoc applications out of a toolbox of tens of thousands of pieces."

What's the advantage supposed to be? Constructing a "large ad-hoc application" doesn't sound like a great idea to me.

Re:Shell scripts are a glue language (4, Interesting)

seebs (15766) | more than 5 years ago | (#26818675)

One of the points might be that there's a fairly specialized task which takes a person six hours to do, but which is NEARLY all done automatically -- just a bit of hand twiddling.

Lemme give an example that isn't portable. One of the things I do fairly frequently is take about six large toolchains distributed to me as binaries and source tarballs, and turn them into patches against upstream versions, reorganize them, delete some unused files, create configuration files that refer to the binaries, generate md5sums, and so on.

This is a task which, if I sit down at 10AM and start typing, is usually done by about 4PM. Testing takes a bit longer, and usually uncovers SOME kind of typo or thing I forgot to do.

Enter the shell script.

I tell the script where the files are, and I walk away. An hour later I have the results. Testing is also automated (another script). But testing is also uneventful, because the script never forgets a step, makes a typo, or otherwise screws up.

By the second time I did this, the script had saved me time. By the third, it had saved me close to a full working day. By now it's closer to a week of my time that wasn't spent messing with this stuff.

Portability isn't entirely crucial here, you might think? Well, not ENTIRELY crucial, except that when they had me start doing this on a new box running a different variety of Linux, the total time I spent revising the script was 0 minutes.

Re:Shell scripts are a glue language (4, Funny)

seebs (15766) | more than 5 years ago | (#26817831)

I think "One does not write a web server in Bash" is like "One does not simply walk into Mordor." You're practically daring short people with hairy feet to attempt it.

But your point is basically good. :)

Re:Shell scripts are a glue language (1)

starfishsystems (834319) | more than 5 years ago | (#26818663)

But it's reasonable, indeed fairly elegant, to write a web server in Tcl. And Tcl was specifically conceived as a glue language. It doesn't hurt that Tcl commands look a lot like interactive shell commands, but expressed in a more regular syntax.

I'm mentioning this just to point out that these distinctions of granularity are a bit artificial. Yes, glue languages give you composability at the coarse end of the scale. But they're often quite acceptable as programming languages as well.

PSS can be a recurring problem (3, Insightful)

Anonymous Coward | more than 5 years ago | (#26816137)

I'm sure all of us at one time or another have had a shell script we've relied on for years fail miserably when bringing it to a new environment. The sad fact is, shell scripts were never meant to be programming languages in and of themselves, and I wonder if, knowing what we know now, it isn't overly ego-driven and masochistic to try to take this feature -- tied to a shell which is tied to an operating system -- and promote it beyond its competency.

So, let's say we take the PSS principles seriously, and abstract away any non-platform-agnostic features you can think of. A few years down the road, you've got PSS all over the shop and you want to upgrade to a different platform nominally supporting your shell of choice. Even if you shake off PSS features you thought could create incompatibilities, you discover the new system buffers differently. Or added a parameter somewhere. Point is, if you went with something like Perl which is designed for cross-compatibility you would have been fine, but now you're all wet.

Shell scripting is good for what it's meant for, but at the risk of oversimplifying with a tool analogy, I'm concerned that this falls into the trap of "If all you have is a hammer, all your problems look like nails".

Re:PSS can be a recurring problem (1)

seebs (15766) | more than 5 years ago | (#26816777)

You know, that's exactly why a book on it is useful -- because it turns out that you can, in fact, write scripts which port quite nicely.

Seriously, "buffers differently"? What's THAT supposed to mean?

That said, I won't deny for a moment that there are things shell isn't a good choice for. In fact, one of the first sections in the book is under the heading "What Shell Scripting Isn't". Because sometimes the best way to write a good program is to pick the right language to begin with. Often, even.

But sometimes, that language IS shell.

Re:PSS can be a recurring problem (1)

Alpha830RulZ (939527) | more than 5 years ago | (#26817903)

Seriously, "buffers differently"? What's THAT supposed to mean?

One example is how and when data gets written to disk in the absence of a flush().

Re:PSS can be a recurring problem (1)

seebs (15766) | more than 5 years ago | (#26818389)

And what "flush()" do you expect to see in a shell script?

You're thinking at a level that generally doesn't apply well to shell scripts. Scripts rarely deal with questions such as "has this been actually written to disk" -- because that's at the wrong level. If you need that information, you shouldn't be writing in shell anyway.

But normally you don't...

Free guide to shell scripting (0)

Anonymous Coward | more than 5 years ago | (#26816217)

If you want a free guide to shell scripting have a look for "Aliens Bash Tutorial" on google

Re:Free guide to shell scripting (1)

zippthorne (748122) | more than 5 years ago | (#26816439)

Not so much a guide, but "man bash"* or "info coreutils" are incredible resources for those common bits you routinely forget. If you stick to what's in coreutils, I think you've got a pretty good chance of portability.

*for systems that use bash as the shell, of course. Frankly, I think there might be too much in there, though and some of the builtins should have their own man pages.

Oh I hate those [ "X$var" == "X" ] (0)

Nicolas MONNET (4727) | more than 5 years ago | (#26816367)

Why bother with portable shell scripts, seriously? Everybody has bash installed, and/or zsh that is mostly compatible, and even then you have bash anyway.
I understand retro-nostalgia and all that, but necrophilia is overrated.

Re:Oh I hate those [ "X$var" == "X" ] (1)

seebs (15766) | more than 5 years ago | (#26816517)

Actually, that's useful in bash too sometimes. :)

Anyway, the reason I bother is this:

I wrote a bunch of scripts in 1992 or so. I'm still using them. I haven't touched any of them in years, except for one update to deal with Linux differing from BSD in where the actual errno definitions are.

I don't have to worry about what shells are installed, I don't have to guess whether bash is "/bin/bash" or "/usr/pkg/bin/bash", I don't have to wonder whether the sysadmin bothered to install the "GNU utilities". I just copy my scripts into ~/bin and go.

Re:Oh I hate those [ "X$var" == "X" ] (2, Insightful)

larry bagina (561269) | more than 5 years ago | (#26816611)

Not everybody has bash and/or zsh installed.

So WHO does not? (1)

Nicolas MONNET (4727) | more than 5 years ago | (#26817913)

I've heard that said for 15 years, that might have been true 15 years ago, but now ... ?

Re:So WHO does not? (0)

Anonymous Coward | more than 5 years ago | (#26817943)

Welcome to the wonderful world of slow moving corporate standards.
I recently left a company that requires a business case for bash shell requests

Re:So WHO does not? (0)

Anonymous Coward | more than 5 years ago | (#26818811)

The only 'nix-like that seems to have bash as standard is Linux.

News flash. There are other 'nix-likes.

Re:Oh I hate those [ "X$var" == "X" ] (0)

Anonymous Coward | more than 5 years ago | (#26818317)

Everybody has bash installed, and/or zsh that is mostly compatible, and even then you have bash anyway.

The utterances of experts. Now go and try a real traditional Unix -- not Linux, not even OS X, I mean something like Solaris, AIX, or HP-UX. Enjoy hunting for bash or zsh. (Hint: your choice will be between ksh and csh.)

Re:Oh I hate those [ "X$var" == "X" ] (4, Informative)

wkcole (644783) | more than 5 years ago | (#26818407)

Why bother with portable shell scripts, seriously? Everybody has bash installed, and/or zsh that is mostly compatible, and even then you have bash anyway. I understand retro-nostalgia and all that, but necrophilia is overrated

False.

The majority of systems I work on these days and the majority of systems I have worked on since the mid 90's have not had bash installed. That includes systems running FreeBSD, NetBSD, OpenBSD, AIX, Tru64, Solaris, MacOS, and even Linux. Current versions of some of those will usually have bash in a default installation, but some still do not. Companies running stable systems as important parts of their business do not generally upgrade their OS's just for the sake of novelty. Running older systems isn't usually about nostalgia or necrophilia, it's more often about not having any compelling reason to upgrade. There is also a system hygiene practice common on the BSD's of keeping the base system minimal and only adding on what is needed, a practice that helps in keeping systems secure and stable because they are easier to fully understand. This is also common in many virtualization environments, where a running OS instance is likely to exist for a very narrow purpose and intentionally have a stripped-down set of utilities fit to that narrow purpose.

Re:Oh I hate those [ "X$var" == "X" ] (1)

Nicolas MONNET (4727) | more than 5 years ago | (#26818945)

Recent versions of MacOSX have bash by default. By recent I mean 10.4 had bash, and probably 10.3 but I'm not sure.

All Linux distribs have had bash installed by default for ever. And by all I mean 99.999% of the installed base, I'm sure you can find a silly exception.

Recent versions of Tru64 ... do not exist.

As for the BSDs, Netcraft confirms it, .. err. I don't know, what's their default shell?

And as for Solaris, its default shell -- a Soviet-era knock off of the original Unics v1.0 -- is so fucktarded that nobody in their right mind keeps it as their default shell, and I've always seen bash or zsh instead.

There is also a system hygiene practice common on the BSD's of keeping the base system minimal and only adding on what is needed, a practice that helps in keeping systems secure and stable because they are easier to fully understand.

Yeah, you're right, installing such an experimental, little used and unmaintained software package as bash is irresponsible. /facepalm

Get ur histry right fella! (0)

Anonymous Coward | more than 5 years ago | (#26816463)

DOS emulates Gary Kildal's CP/M .. CP/M got it from the Digital Equipment VAX .. they got it from the original UNIX/ATT. .. talk about copyright theft!

IBM came up with the hierarchical file system.

--- I was there when it all happened!!

BTW -- Microsoft stole the mouse and windows from Xerox PARC and DOS from Digital Research!! Neither of these guys chose to sue even though they could have!!

Re:Get ur histry right fella! (1)

quickOnTheUptake (1450889) | more than 5 years ago | (#26817785)

Speaking of knowing your history:

Digital Research threatened legal action, claiming PC/MS-DOS to be too similar to CP/M. IBM settled by agreeing to sell their x86 version of CP/M, CP/M-86, alongside PC-DOS.

So Digital did threaten, but there was a settlement. DR-DOS on Wikipeida [wikipedia.org]

John Gibson: Cocaine Addict +1, Informative (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#26816473)

I'm sure you've heard John Gibson's [foxnews.com] radio rant. The rant is caused by his cocaine addiction. Wait. There's more. Recent troubling developments prompt me to revisit a subject I've discussed in the past: John Gibson and his plan to redefine humanity as alienated machines/beasts and then convince everyone that they were never human to begin with. In the first place, I defy the snippy, pea-brained ninnies who confiscate other people's rightful earnings and I defy the powers of darkness that they represent. I, hardheaded cynic that I am, have this advice to offer: The world has changed, Gibson; get used to it. He has been known to "prove" statistically that a plausible excuse is a satisfactory substitute for performance. As you might have suspected, his proof is flawed. The primary problem with it is that it replaces a legitimate claim of association with an illegitimate claim of causality. Consequently, Gibson's "proof" demonstrates only that he wants to exhibit a deep disdain for all people who are not heartless vagrants. You know what groups have historically wanted to do the same thing? Fascists and Nazis.

Gibson's thralls seem to be caught up in their need for enemies. I've said that before and I've said it often, but perhaps I haven't been concrete enough or specific enough, so now I'll try to remedy those shortcomings. I'll try to be a lot more specific and concrete when I explain that Gibson's favorite buzzword these days is "crisis". He likes to tell us that we have a crisis on our hands. He then argues that the only reasonable approach to combat this crisis is for him to make us less united, less moral, less sensitive, less engaged, and more perversely muddleheaded. In my opinion, the real crisis is the dearth of people who understand that I sincerely don't believe that war is peace, freedom is slavery, and ignorance is strength. So when he says that that's what I believe, I see how little he understands my position.

Unless we drive off and disperse the sex-crazed psychics who encourage men to leave their wives, kill their children, practice witchcraft, destroy capitalism, and become imperious nutters, our whole social structure will gradually disintegrate and crumble into ruins. The poisonous wine of simplism had been distilled long before Gibson entered the scene. Gibson is merely the agent decanting the poisonous fluid from its bottle into the jug that is world humanity. Like other nugatory phonies, he has a finely honed ability to require religious services around the world to begin with "Gibson is great; Gibson is good; we thank Gibson for our daily food". He will almost certainly tiptoe around that glaringly evident fact because if he didn't, you might come to realize that I can easily see him performing the following homicidal, misinformed acts. First, Gibson will acquire public acceptance of his censorious grievances. Then, he will confuse, befuddle, and neutralize public opposition. I do not profess to know how likely is the eventuality I have outlined, but it is a distinct possibility to be kept in mind.

Under these conditions, Gibson's plan is to scorn and abjure reason. Gibson's grunts are moving at a frightening pace toward the total implementation of that agenda, which includes preventing me from sleeping soundly at night. But this is something to be filed away for future letters. At present, I wish to focus on only one thing: the fact that that fact is simply inescapable to any thinking man or woman. "Thinking" is the key word in the previous sentence.

To put it another way, Gibson's prevarications are as predictable as sunrise. Whenever I confront and reject all manifestations of irreligionism, his invariant response is to depressurize the frail vessel of human hopes. Gibson keeps telling us that his blessing is the equivalent of a papal imprimatur. Are we also supposed to believe that all literature that opposes officialism was forged by unscrupulous sad sacks? I didn't think so.

Gibson is simply incapable of entertaining an unorthodox idea. To top that off, Gibson wonders why everyone hates him. Apparently, he never stopped to think that maybe it's because if everyone does his own, small part, together we can criticize his complicity in the widespread establishment of lexiphanicism. The point is that if everyone spent just five minutes a day thinking about ways to provide people the wherewithal to urge lawmakers to pass a nonbinding resolution affirming that Gibson may come to represent the most insidious corruption of ideals yet, we'd all be a lot better off. Is five minutes a day too much to ask for the promise of a better tomorrow? I sure hope not, but then again, Gibson is stepping over the line when he attempts to direct social activity toward philanthropic flimflam rather than toward the elimination of the basic deficiencies in the organization of our economic and cultural lifeâ"way over the line.

Worst of all, our children's children would never forgive us for letting Gibson create an atmosphere of mistrust in which speculations and rumors gain the appearance of viability and compete openly with more carefully considered theories. I guess I can't blame him for wanting to pander to our worst fears. After all, according to him, a book of his writings would be a good addition to the Bible. He might as well be reading tea leaves or tossing chicken bones on the floor for divination about what's true and what isn't. Maybe then Gibson would realize that you may have noticed that we must do away with the misconception that he is a martyr for freedom and a victim of antinomianism. But you don't know the half of it. For starters, we must remove our chains and move towards the light. (In case you didn't understand that analogy, the chains symbolize Gibson's headstrong words and the light represents the goal of getting all of us to condemnâ"without hesitation, without remorseâ"all those who control your bank account, your employment, your personal safety, and your mind.)

Gibson has no discernible talents. The only things he has certainly mastered are biological functions. Well, I suppose Gibson's also good at convincing people that the kids on the playground are happy to surrender to the school bully, but my point is that Gibson hates youâ"yes, you, because you, like me, want to put inexorable pressure on Gibson to be a bit more careful about what he says and does. This is particularly interesting when you consider that over the years, I've enjoyed a number of genuinely pleasurable (and pleasurably genuine) conversations with a variety of people who understand that until recently, his plaints have gone unnoticed and unanalyzed. In one such conversation, someone pointed out to me that I admit I have a tendency to become a bit insensitive whenever I rebuke Gibson for trying to contaminate or cut off our cities' water supply. While I am desirous of mending this tiny personality flaw, Gibson claims to have turned over a new leaf shortly after getting caught trying to cause people to betray one another and hate one another. This claim is an outright lie that is still being circulated by Gibson's brethren. The truth is that Gibson often argues that the best way to make a point is with foaming-at-the-mouth rhetoric and letters filled primarily with exclamation points. A similar argument was first made over 1200 years ago by a well-known hoodlum and was quickly disproved. In those days, however, no one would have doubted that Gibson's sentiments are so insecure that in minutes they can wipe out of a child's brain what that child had learned in six months at home, church, or school. Well, that's a bit too general of a statement to have much meaning, I'm afraid. So let me instead explain my point as follows: I welcome Gibson's comments. However, Gibson needs to realize that I stand by what I've written before, that if I may be so bold, if you've read any of the obdurate slop that he has concocted, you'll unquestionably recall his description of his plan to sow the seeds of discord. If you haven't read any of it, well, all you really need to know is that if Gibson were to make all of us pay for his boondoggles, social upheaval and violence would follow. It is therefore clear that our path is set. By this, I mean that in order to expand people's understanding of Gibson's ostentatious expostulations, we must examine his worldview from the perspective of its axiology (values) and epistemology (ways of knowing). I consider that requirement a small price to pay because Gibson is chomping at the bit for a chance to abrogate some of our most fundamental freedoms. The mere mention of that fact guarantees that this letter will never get published in any mass-circulation periodical that Gibson has any control over. But that's inconsequential because Gibson's comrades are merely liars with charisma. That said, let me continue.

Needless to say, Gibson keeps trying to introduce disease, ignorance, squalor, idleness, and want into affluent neighborhoods. And if we don't remain eternally vigilant, he will unmistakably succeed. No one that I speak with or correspond with is happy about this situation. Of course, I don't speak or correspond with larcenous caitiffs, Gibson's cronies, or anyone else who fails to realize that Gibson once had the audacity to tell me that the federal government should take more and more of our hard-earned money and more and more of our hard-won rights. My riposte was that when a friend wants to drive inebriated, you try to stop him. Well, Gibson is drunk with power, which is why we must reveal the constant tension between centripetal and centrifugal forces of dialogized heteroglossia resulting from his ramblings.

Other than that, one of Gibson's secret police once said, "The rigors that Gibson's victims have been called upon to undergo have been amply justified in the sphere of concrete achievement." Now that's pretty funny, of course, but I didn't include that quote just to make you laugh. I included it to convince you that Gibson's arguments would be a lot more effective if they were at least accurate or intelligent, not just a load of bull for the sake of being controversial. Although I can no more change the past than see the future, it's safe to say that society must soon decide either to clean up the country and get it back on course again or else to let Gibson harvest what others have sown. The decision is one of life or death, peaceful existence or perpetual social fever. I can hope only that those in charge realize that many of the people I've talked to have said that Gibson and his factotums should all be put up against a wall and given traitors' justice. Without commenting on that specifically I'd merely like to point out that Gibson has a talent for inventing fantasy worlds in which everyone who scrambles aboard the John Gibson bandwagon is guaranteed a smooth ride. Then again, just because Gibson is a prolific fantasist doesn't mean that "metanarratives" are the root of tyranny, lawlessness, overpopulation, racial hatred, world hunger, disease, and rank stupidity. Let me end by appealing to our collective sense of humanity: John Gibson's double standards serve only to safeguard his own power and privilege.

Yours In Socialism,
Kilgore Trout

Why Buy? (1)

Monkey_Genius (669908) | more than 5 years ago | (#26816687)

When it's free -as in beer: 'Advanced bash Scripting Guide - Mendel Cooper [tldp.org] '.

Re:Why Buy? (2, Insightful)

seebs (15766) | more than 5 years ago | (#26816875)

Well, I suppose because the contents are different. We're answering different questions.

I would never dream of discouraging people from looking at and using the various free guides out there. The autoconf manual is full of useful information about portability, too.

There's more than one article or book because there's more than one topic. "Shell Scripting" is not a single topic; "portable shell" is very different from "advanced bash". It solves different problems, and is useful for different circumstances.

Patently Absurd (-1, Troll)

Anonymous Coward | more than 5 years ago | (#26816703)

Since when have shell scripts suffered from portability problems?

This is a problem made up by people who don't really know any better, for people who wouldn't know they don't need it.

Just some general comments... (5, Informative)

seebs (15766) | more than 5 years ago | (#26816719)

First off, in the interests of full disclosure, Joe MacDonald is one of my coworkers.

Anyway... The big surprise to me was the word "Beginning", which somehow showed up in the publisher's cover pages, but which I didn't know about during the writing process. My tech reviewer was Gary V. Vaughan (yes, the autoconf/libtool guy). I bounced material off a number of seasoned expert scripters during the process. Basically, my goal was to write a book that I could use as a reference, and which would teach me something.

I succeeded beyond my wildest dreams. The discovery that printf(1) is essentially universal these days was a complete shock to me; I had no idea it was portable. During my first pass on the regular expressions section, I started by writing down what I believed I knew about sed, awk, etcetera. Then I tested it... and had to revise most of it. A number of things I was used to were GNU or BSD extensions. When Gary sent the chapter back for tech review, he'd flagged most of these things, because he "knew" the same things I did.

So everything there should be pretty thoroughly checked out now -- I realized very early on that this field was full of things "everyone knows". Many of them wrong. We tested things on a field of around 30 different versions of Unix and Linux. We tested them on unusual installs, we tested them on special cases.

Why?

Because portable shell is an incredibly portable language, and sometimes that matters. Because shell is a very powerful language, too. Because sometimes shell is all you have -- and because sometimes shell is more expressive for a task than your other choices. I love me some C, I program in C by preference much of the time -- but there are a lot of tasks I'll do in shell rather than in C. There are similarly many tasks I'd rather write in shell than in perl. Shell is what make uses to run commands, and sometimes you need to write something clever in shell because make doesn't have quite the right feature.

In short, it's something I have found consistently useful, day in and day out, for about twenty years now. I just wish I'd realized how much more there was to learn years ago, I coulda saved a lot of time... :)

And, to answer a question hinted at earlier: Yes, now that this book exists, I keep a copy on my desk. I look stuff up in it about once a week.

Nobody's seriously comparing C and shell (1)

Nicolas MONNET (4727) | more than 5 years ago | (#26817999)

Saying "I love C but there are things that are better in shell" is completely anachronistic.

Seriously. The question's been settled for over 20 years.

And there are other languages, you know. The question is more whether to use Python (for example) instead of shell in some cases, and when.

Re:Nobody's seriously comparing C and shell (2, Interesting)

seebs (15766) | more than 5 years ago | (#26818333)

I compare C and shell all the time. Sometimes the answer surprises me. e.g., until I knew about printf(1), I sometimes went to C if I needed to pretty-print output. Now sometimes I don't.

I will happily mix and match multiple languages; one of my first shipping products was written in shell, perl, and C. Each did some things well that the others didn't...

I tend not to use Ruby for things that I want to be portable, because not everything has Ruby around yet. I tend to avoid Python because it just never "clicked" for me. I use perl for self-contained programs, but I don't like it very much for code that has to run other commands or manipulate files.

To this day, I've never found any of the competing languages to be as expressive as shell for things that are essentially sequences of Unix commands. If a task is something I would normally perform sitting at a prompt typing, I tend to find shell to be the language closest to the problem space.

portable shell scripting is called Python (1, Redundant)

fuzzylollipop (851039) | more than 5 years ago | (#26816817)

why deal with limited shell scripting when you have something so batteries included and universal as Python

Re:portable shell scripting is called Python (1)

seebs (15766) | more than 5 years ago | (#26817481)

Honestly, I just never got into python. I like perl (and hate it), and I like Ruby (and love it), but Python never "clicked" for me. Python and Tcl are the "scripting languages I just don't enjoy working in".

But... I also don't have it on everything I use. And I could get it, but that's another distraction from Getting The Job Done. So I do a ton of work in shell. Especially work that's entirely built around running other commands.

Re:portable shell scripting is called Python (1)

ultrabot (200914) | more than 5 years ago | (#26818841)

Honestly, I just never got into python. I like perl (and hate it), and I like Ruby (and love it), but Python never "clicked" for me. Python and Tcl are the "scripting languages I just don't enjoy working in".

I believe this is a common mindset issue. Ruby borrowed lots of stuff from perl (which they thought was good, or appealing to convert - I'm not sure), including TIMTOWTDI, whereas Python community thinks of Perl as a warning example more than anything else. My theory is that the different languages balance between simplicity and 'regularity' (python) vs. 'playfullness' and 'interesting' solutions (ruby, perl).

Re:portable shell scripting is called Python (0)

Anonymous Coward | more than 5 years ago | (#26817727)

which version of this universal python do i need?
which supplementary libraries for this universal python do i need to install?

Where shell scripts shine (1)

wdef (1050680) | more than 5 years ago | (#26817411)

Wherever you want to do an awful lot of things with the input and output of) system utilities and./or bash builtins. Look at the gargantuan effort that is the Knoppix boot scripts - I seriously doubt it would make sense to rewrite those in Python or Perl, since nearly every line is a pipe between utilities or redirection. And it works well.

Re:Where shell scripts shine (1)

ultrabot (200914) | more than 5 years ago | (#26817887)

Wherever you want to do an awful lot of things with the input and output of) system utilities and./or bash builtins.

Whenever you try to do something nontrivial in between the system utilities, you start losing the benefit of shell scripts.

Re:Where shell scripts shine (1)

wkcole (644783) | more than 5 years ago | (#26818665)

Wherever you want to do an awful lot of things with the input and output of) system utilities and./or bash builtins. Look at the gargantuan effort that is the Knoppix boot scripts - I seriously doubt it would make sense to rewrite those in Python or Perl, since nearly every line is a pipe between utilities or redirection. And it works well.

I'm not familiar with Knoppix, but that is true for all boot script systems, and beyond being hard to do in something else, there are cases where it may be impossible. There are still systems out there that boot on very small root filesystems that do not have any shared libraries available until the rest of their storage is mounted, so you've got to have something small and statically linked to run the scripts that get your whole software edifice in place. You can have similar problems in some failure states, where you might like to have something run if, say, /usr suddenly vanishes or if root needs to log in on the console of a box that has dropped off the net. There are systems where /sbin/sh exists for just such reasons and is a statically linked classical Bourne shell.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...