Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Book Review: Linux Shell Scripting Cookbook

samzenpus posted more than 3 years ago | from the read-all-about-it dept.

Books 173

vellorean writes "I have been reading Linux Shell Scripting Cookbook by Sarath Lakshman, published by Packt, for a while. While most people I know learn shell scripts themselves, I was looking to refresh my concepts a little as well as have a reference lying around on the table for fast access." Read below for the rest of vellorean's review.First of all, let me remark by saying that shell scripting is something learned more on a need basis than as a tool to solve the main problem. People would seldom write shell scripts as standalone programs (exceptions exist). However, what makes shell scripting invaluable to know is the fact that knowing some tricks can save several minutes, or hours, of work by automating and simplifying certain tasks, generally (but not restricted to) file management and data processing. Linux Shell Scripting Cookbook does go quite far in pursuing this goal, and is appropriate for both beginners who are looking to gain dexterity in shell scripting, as well as intermediate users who wish to polish their skills. The book also can double up as a quick reference, though I would argue that the "Advanced Bash Scripting Guide" would suit that more.

At the outset, the author clarifies that the focus will be on Bash. This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do. Besides, learning other shell scripting languages while knowing bash isn't too hard, since the paradigm remains the same.

The book is organized into chapters based more on utility than scripting concepts themselves, although the language aspects are brought onto the reader gradually. For instance, the examples in the first chapter focus more on the basic data elements (variables, arrays, functions etc.) as well as operators (for numbers, files etc.), and all the examples demonstrate simple usage of these concepts, and he further chapters build upon these in a gradual manner.

At the same time, if he reader has some familiarity with shell scripting and needs to only refresh or learn a certain concept, he/she needs to just read the relevant chapter. It is not too difficult to grasp the examples of the later chapters, provided some basic shell knowledge is assumed.

A positive trait in the presentation of this book is that it is all based on practical everyday examples which, with minor adaptation, can be used by many for their own daily tasks. For instance, there are several examples which describe searching for and processing files, which, I'd imagine, many users would want to do on a regular basis. Thus, providing realistic examples allows the book to double its utility. The language and approach used is simple and conversational, and the presentation is very clear, with each idea being described as a problem statement followed by a "How to do it" section with the actual code, and ending with a discussion of the nitty-gritties of the code. It is easy to go for a quick scan for those in a hurry, while those who with to read in more detail will not be disappointed either.

The book also covers a wide array of applications. For instance, there are examples on automating fetching web pages and processing them, demonstrations of parsing and simplifying and even some queries around databases wrapped around in shell. It also spans to utilities and tasks connected to statistics, backups, compression, version control and many more.

The book goes into a fair amount of detail in terms of describing the shell scripting concept under consideration. The examples used go into a fair amount of detail in order to describe to the user all the aspects involved in the method or command being used. The concepts described are fairly complete, and would be sufficient for the reader to use immediately or with just a little bit of fine tuning. In terms of breadth, the book covers most of the features of shell scripting while also describing the various facilities the shell provides access to in a Unix-like environment. Thus, the book does not disappoint in this front either.

In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts. Granted, most of the shell scripts described in the book are very simple and succinct, but a some words on how loops can be made better, or how to spot situations where pipes are not needed to solve a problem etc. might have been a nice addition. Some explanation of differences with dash, tcsh, zsh etc. might also have been nice, since a lot of users have different default shells. But all this isn't going to prevent me from giving this book a high rating, since it delivers quite well on the promises it makes at the beginning.

This is definitely a good book to have near your desk, and kudos to the author for having taken the effort to put it together. I would highly recommend it to the beginner and occasional shell user for a thorough read, and to an intermediate to have on his/her desk for borrowing the cool scripting ideas and applications the author has written in this book.

You can purchase Linux Shell Scripting Cookbook from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

cancel ×

173 comments

Sorry! There are no comments related to the filter you selected.

2 Packt reviews in 2 days? (1)

Anonymous Coward | more than 3 years ago | (#35831822)

Is Packt desperate for books sales that they have 2 shill reviews in just the last couple of days? Or is it that Slashdot is low on funds and needs another shill check from Packt?

Re:2 Packt reviews in 2 days? (1)

Anonymous Coward | more than 3 years ago | (#35831878)

Or maybe they just happened to release a couple of books relevant to the interests of the slashdot demographic close to each other?

Re:2 Packt reviews in 2 days? (0)

Anonymous Coward | more than 3 years ago | (#35832112)

Yeah, except for the fact that this book was released on Jan 25th and the other December 11th of last year. So they are both at least 5 months old now. Looks like Packt is trying to bump up shitty book sales with another shill review.

Good book, did some of it (-1, Troll)

commadot (2042338) | more than 3 years ago | (#35831876)

It includes historical differences between shells and most importantly incompatibilities between them. Really worth a read. Also includes description of exotic shells like the bsh [tinyurl.com]

Boy Goatsex is out in force today... (2)

Lead Butthead (321013) | more than 3 years ago | (#35831958)

Every topic is littered with them...

Thanks (0)

Anonymous Coward | more than 3 years ago | (#35832430)

I have a quad core phenom-II. I had no idea how fast it was until I clicked on that link. Well done.

BTW it's a while since I've seen goatse.cx. Very funny.

Why are there still shell scripts anyways? (1, Informative)

i kan reed (749298) | more than 3 years ago | (#35831882)

Why does anyone still use shell scripts anymore? Every major(and most minor) distro ships with python, ruby, and perl either built in or a trivial statement away. These languages give you access to basic OS functions and much more sane syntaxes. Shell scripting is something I haven't felt the need to do in 5 years.

Re:Why are there still shell scripts anyways? (5, Insightful)

mugnyte (203225) | more than 3 years ago | (#35831924)

wow. how many ways can i answer that?

  - there is no one true way
  - people learn a skill and want to continue using it
  - concise syntax / small footprint / fast load times
  - high confidence shell scripts will run on unknown systems
  - can be layered on years of useful scripts
  - distrust of your options for doing work

Re:Why are there still shell scripts anyways? (0)

Anonymous Coward | more than 3 years ago | (#35832140)

Sure you can use a wrench to hammer in a nail but a hammer is much more appropriate. Modern scripting languages have concise syntax, small footprints and fast load times. There is a high confidence that such languages will be on unknown systems and they have been around for years sporting vast libraries cough... CPAN.. cough that rival and exceed most anybody's bag of shell tools.

The real point is that modern scripting languages are languages and as such offer sophisticated and proper design for engineering eloquent and useful solutions to real world problems. There's a reason why languages are designed and there are reasons for each and every bit of functionality and concepts that they put in them - even if you fail to grasp them. As such there is good reason to move to them instead of arcane shell incantations that do not promote sound software engineering principals.

That said I will whip out a quick bash script to do some research or investigative type stuff but if I ever find it being run frequently, growing bigger or being put into production I usually take the time to re-write it properly in Perl.

Re:Why are there still shell scripts anyways? (2)

sdguero (1112795) | more than 3 years ago | (#35832690)

For me its usually;

- too lazy to learn another language or...
- not enough time to learn another language

Re:Why are there still shell scripts anyways? (5, Insightful)

beatle42 (643102) | more than 3 years ago | (#35831972)

I find it useful to do shell scripting when I've been doing something by hand, and want to automate it. It's pretty easy to just echo the command I've been doing into a file and touch it up from there. Then I can take advantage of the organic growth of the command as it usually happens, and can also use it to run that same command on lots of systems if needed.

Re:Why are there still shell scripts anyways? (1)

Desler (1608317) | more than 3 years ago | (#35832124)

Which are all things you can also do in perl and python...

Re:Why are there still shell scripts anyways? (0)

Anonymous Coward | more than 3 years ago | (#35832698)

True, but they come out of doing them at a command prompt, so why would I write it twice? I could do it all in assembly too, but that doesn't mean its a good choice.

Re:Why are there still shell scripts anyways? (1)

Paracelcus (151056) | more than 3 years ago | (#35832758)

Rah! A really good comment, saves time, works with all shells.

Re:Why are there still shell scripts anyways? (5, Insightful)

Anonymous Coward | more than 3 years ago | (#35831984)

Sysadmins who can't work in plain shell scripts are dangerous. What will you do if you lose your /usr filesystem ? Be stuck with no automation tools at all as you fix by hand ? Shell scripts have little or not dependencies, and can function where other languages aren't available.
More-over they are ubiquitious. Perl, python and ruby are common in Linux, but what will you do if you're on a different Unix altogether ? A sysadmin who can use shells efficiently can easily transfer his skills to solaris, hp-ux or several others (I did just that, and now I'm paid quite a lot more than a Linux sysadmin earns as a Unix sysadmin because the skillset is rarer). Of course there are other skills you'll need to learn (package management for example is vastly different between various unixes and most are frankly primitive if you are used to Linux) but the core Unix fundamentals remain the same, and if you don't know powerful and flexible shell scripting, in my book, you're no sysadmin at all.

Re:Why are there still shell scripts anyways? (1)

Desler (1608317) | more than 3 years ago | (#35832148)

but what will you do if you're on a different Unix altogether ?

You do realize that Perl and Python exist on AIX, HP-UX, Solaris etc as well, right? If you are on a different Unix... you just use them as well.

Re:Why are there still shell scripts anyways? (1)

goombah99 (560566) | more than 3 years ago | (#35831986)

You phrase the question differently than I would. I would ask why is perl not the default shell language.

I have a hard time understand the need for bash when perl is around. I can't quite make the same argument for Python and Ruby. THose are arguably better programming languages but not better administrative script languages. Perl on the otherhand is still very close to the common shell languages while being much more powerful. It's also faster than shell for almost every purpose where time matters. But this is achieved without making simple things difficult.

Indeed that was Larry's mantra: simple things should be simple, and hard things possible. So perl is the glue language of choice to supplant shell.

Perl's name was derived from the acronym Practical Extraction and Report Language. That's what Linux administrative scripting is all about. If you want to write a GUI word processor, use python.

Re:Why are there still shell scripts anyways? (0)

Anonymous Coward | more than 3 years ago | (#35832100)

PERL is dead, get over it. It had uses over a decade ago, now almost nothing serious is developed using it, just admins' glue for when shell scripting is too much of a PITA.

Re:Why are there still shell scripts anyways? (1)

h4rr4r (612664) | more than 3 years ago | (#35832232)

So nothing serious, just the stuff no business can live without holding everything together. Ok got it.

Re:Why are there still shell scripts anyways? (1)

tnk1 (899206) | more than 3 years ago | (#35833256)

If you are talking about actual commercial enterprise software that is sold for money, perl probably never had anything "serious" developed using it. It's not really a language for that.

Perl is the scripting language you use when you want more than you get with shell, but don't really care if it is optimized to the point it can be used for an enterprise app. Just like bash, perl is on every UNIX-like host out there and if you write your scripts using only perl and perl modules, the code is entirely portable to anything you can get perl to run on, including Windows. With things like PAR, you don't even need to load the modules on each host, it will create a self-extracting archive with the necessary modules and shared libraries once you archive up your script.

The other nice thing about perl is that it works quite a bit like shell already. You can run command line apps directly from perl just like you can from shell. All you have to do is use system() or the dreaded backticks. I avoid that, because it can affect portability, but if you need to upgrade an existing shell script quickly, it's easy to do. You can upgrade a shell script, whose data source is a text file, to a script that does the same thing to data stored in a database or an XML file, or on the web with trivial effort in perl and not even have to re-write much of the portion that you used in the original bash script.

Perl is always there, has modules for everything, and has been used for so long that you can find documentation for best practices to do anything with it. Ruby and Python have their uses, but they aren't solving any problems for me that haven't already been solved by perl long ago. There's simply no way to go wrong with perl if you understand it's limitations, have already taken the time to learn it and have become proficient in it. It is also possible to write readable, well-structured code with perl, they just don't force that on you.

I use:

bash = startup scripts, quick scripts where portability is a non-issue.
perl = creation of in-house tools, advanced scripts
java = business enterprise apps for customers that can be developed fairly quickly
c++ = applications that need less portability and a lot more optimization.
ruby = we had great success using ruby for making great looking reports for customers

Perl may not be exciting or the next thing, but its far from dead. It has a niche that it fills very well.

Re:Why are there still shell scripts anyways? (2)

Hatta (162192) | more than 3 years ago | (#35832186)

You phrase the question differently than I would. I would ask why is perl not the default shell language.

Because Perl is huge and sh is tiny. Would you be able to port Perl to BusyBox without dramatically increasing the size of the executable?

Re:Why are there still shell scripts anyways? (1)

h4rr4r (612664) | more than 3 years ago | (#35832322)

Even my phone has 256MB of ram, and its replacement will have at least 1GB. Remind me again why this is a problem?

What is up with embedded devices having less than 64MBs of ram?

Re:Why are there still shell scripts anyways? (1)

cloudmaster (10662) | more than 3 years ago | (#35833340)

Because I don't want to pay for more RAM in my device just because you're too lazy to use the right tool for the job? :)

Re:Why are there still shell scripts anyways? (0)

Anonymous Coward | more than 3 years ago | (#35833590)

If my embedded device is a sensor on a pipeline then I will want the smallest size and footprint possible. If I can get them at $5/unit with 8 MB of RAM and no local storage or $20/unit with 256 MB RAM and a little storage I will, without a doubt, use the 8 MB unit since I can stick 3x as many sensors on the pipeline for the same cost at the 20/unit (once you actualize labour costs).

For a device that only collects temperature, pressure, solids PPM every 180 ms having fast running application regardless of the hardware is essential (I'd probably use assembler in this case, but YMMV). You wouldn't use Perl (or Python or Ruby) in development in these cases.

You need to expand your mind as to what an embedded device actually is.

Re:Why are there still shell scripts anyways? (1)

h4rr4r (612664) | more than 3 years ago | (#35833650)

I mean consumer embedded devices. I would probably just stick with actual compiled code on devices like that.

Re:Why are there still shell scripts anyways? (1)

codepunk (167897) | more than 3 years ago | (#35832234)

In my shop it is either bash or python take your pick, perl is not to be used on company systems.

Re:Why are there still shell scripts anyways? (2)

h4rr4r (612664) | more than 3 years ago | (#35832356)

I take option 3, not working there.

Python could be nice, if it ignored whitespace/indentation. Also if it had its own version of CPAN.

Re:Why are there still shell scripts anyways? (1)

Bigbutt (65939) | more than 3 years ago | (#35833576)

I'm curious as to the reasons behind this. Is there some inherent problem with perl that it's banned or is it just a requirement to maintain consistency?

[John]

Re:Why are there still shell scripts anyways? (1)

maxwell demon (590494) | more than 3 years ago | (#35832364)

You phrase the question differently than I would. I would ask why is perl not the default shell language.

Because a language which compiles the full file before execution simply isn't usable for an interactive shell. When doing interaction you can't wait whether a function called now but not yet defined will be defined later. You need to execute a command as soon as it is issued, or give an error if you can't. And it has to be possible to change the definition of a function later.

Also, the primary use of the shell is to start other programs from it. Therefore something which needs an extra command to do that simply isn't acceptable as shell. A shell has to take any command it doesn't know as call of a program.

Re:Why are there still shell scripts anyways? (1)

erice (13380) | more than 3 years ago | (#35832486)

You phrase the question differently than I would. I would ask why is perl not the default shell language.

I have a hard time understand the need for bash when perl is around.

[snip] Larry's mantra: simple things should be simple, and hard things possible.

Perl isn't simple enough for basic scripting. Most shell scripts are little more that lists of commands with minimal control flow. For that, Perl is too verbose. The extra syntax means more places for bugs and it gets extra messy if you are generating scripts programatically. Now, I agree that more complex scripting should be done is something like Perl. "Advanced Bash" makes about as much sense to me as "Advanced duck tape application"

It's too bad that Unix shell scripting languages are so dreadful. They didn't have to be. REXX was/is an awesome scripting language for VM/CMS. Negligible syntax overhead for simple operations and you could still do complex things with a clean syntax. Perl is better for bigger things but REXX effortlessly spanned the zone from simple list to substantial program. REXX is available for Unix but it doesn't really have the same effect because the close coupling between language syntax and the default shell isn't there.

Re:Why are there still shell scripts anyways? (0)

Anonymous Coward | more than 3 years ago | (#35833180)

I would buy the book "Advanced duck tape application".

Re:Why are there still shell scripts anyways? (1)

turgid (580780) | more than 3 years ago | (#35832494)

I have a hard time understand the need for bash when perl is around.

Conversely, I have a hard time understanding why anyone would use perl when bash, Ruby and Python are around.

A shell should be (needs to be) light-weight. Perl is not. Perl's syntax is horrendously complicated. Bash (and sh) syntax is not.

Re:Why are there still shell scripts anyways? (1)

Waffle Iron (339739) | more than 3 years ago | (#35832702)

You phrase the question differently than I would. I would ask why is perl not the default shell language.

For one reason, you wouldn't want to have to type in something like

system("ls -l");

instead of

ls -l

for everything you do.

However, the exact same syntax features that make sh good for interactive prompts make it a horrible programming language. IMO, it's tied with COBOL for the worst language still in general use. I'm familiar with a couple of dozen languages, and sh is the one I find hardest to use without constantly referring back to the documentation. I can never remember its crazy variable expansion rules and escapes, its weird logic and test operators, or its cryptic built-in variable names.

Lately, if a script looks like it's going to be longer than a single line, I use Python and import the "subprocess" module. This greatly helps my sanity.

Re:Why are there still shell scripts anyways? (1)

SuseLover (996311) | more than 3 years ago | (#35832006)

Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base. Or maybe your security requirements disallow python, or ruby or whatever to be installed. Or that running shell scripts using many built-ins run faster than the interpreted languages? Bash scripting features seem to stay more stable over time vs. other scripting languages that are constantly changing. The app teams within your company may want to constantly upgrade the installed version that might break you scripts/programs. And bash is dirt simple to learn and implement for non-programmers.

Re:Why are there still shell scripts anyways? (2)

goombah99 (560566) | more than 3 years ago | (#35832168)

Maybe because bash is installed by default on most Linux/UNIX systems and is portable across all yoru installed base.

perhaps they exist but in 20 years I've not see a Linux distro without Perl.

Or maybe your security requirements disallow python, or ruby or whatever to be installed.

if they disallow perl then shell needs to be disallowed too

Or that running shell scripts using many built-ins run faster than the interpreted languages?

this is patently false. Perl is vastly faster than shell even taking into account load times when you do anything significant. Bash scripts that do anthing important generally are calling things like awk and grep and sed to parse things and these invocations are dramatically slower in a loop than a single invocation of perl. Additionally, perl I/O is in many case faster than standard unix operations that move large blocks of data.

finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.

So don't try to argue this on speed or system resource utilization.

Bash scripting features seem to stay more stable over time vs. other scripting languages that are constantly changing.

huh? perl is very static and backward compatible.
moreover, it's the helped commands that bash needs like grep and awk and so many other system resource introspection calls that one cannot count on being present or taking exactly the ame arguments.

even a braindead command like "cp" is not only different on Linux and BSD but it's command arguments have changed over time. Whereas is perl the commands needed to copy files have not changed in 20 years.

The app teams within your company may want to constantly upgrade the installed version that might break you scripts/programs.

the only time i've seen broken perl was the 5.8 to 5.10 switch changed the timing of a few things.

And bash is dirt simple to learn and implement for non-programmers.

Perl is very very very close to bash in syntax

Re:Why are there still shell scripts anyways? (0)

SuseLover (996311) | more than 3 years ago | (#35832288)

I see. So when I am doing disaster recovery of a system where nothing but /usr/sbin and bash is available, I can use perl how? All of the DR scripts I have seen are written in C shell or Bash.

Re:Why are there still shell scripts anyways? (1)

Anonymous Coward | more than 3 years ago | (#35832368)

Yeah, how can you possibly work when I invent a contrived example that is set up so I can win!

Re:Why are there still shell scripts anyways? (1)

Desler (1608317) | more than 3 years ago | (#35832490)

So the best you can do to counter him smashing all your arguments to bits is something as lame as that?

Re:Why are there still shell scripts anyways? (1)

hawguy (1600213) | more than 3 years ago | (#35833406)

finally invoking and disposing a command like awk many times in a row makes for very poor memory management and system resource utilization compared to keeping one program like perl resident.

Poor memory management? The awk binary and its libraries will stay resident in the pagecache between invocations, and awk+bash has a lower memory footprint than perl (on my system, running perl uses around 19MB of Virtual memory, awk+bash is around 14MB).

So, it could be argued that running awk+bash is better for memory overhead.

fork+exec overhead is so low that unless the script is calling awk thousands of times, it's not worth the time to even think about it. It takes around 2ms for me to call awk from a script.

Re:Why are there still shell scripts anyways? (1)

jandrese (485) | more than 3 years ago | (#35832032)

Because there are lots of things that are more awkward with Ruby/Python/Perl than they are with shell scripts?

If you're running a bunch of commands on a system and just verifying return codes, a shell script is the right tool for the job. It's also the right tool for those times when you're working on a very stripped down machine that may or may not have your favorite interpreter installed.

Re:Why are there still shell scripts anyways? (1)

Hatta (162192) | more than 3 years ago | (#35832054)

Because I've always got two or three shells open anyway. If I'm using Bash all day anyway, why not write my scripts in it?

There are definitely better scripting languages, but I get a lot more Bash practice than I do Perl or Python practice. That's all there is to it.

Re:Why are there still shell scripts anyways? (1)

Anonymous Coward | more than 3 years ago | (#35832056)

There are TONS of very valid uses for shell (bash) scripts on modern machines. Take a look at NVidia's proprietary driver for instance, the file you download from Nvidia for linux is "self extracting", this was accomplished using a BASH script...don't believe me, take a look at the file yourself using vim (or emacs if you must!), pretty nifty and an excellent example of a valid use of an advanced bash script!

Re:Why are there still shell scripts anyways? (1)

kale77in (703316) | more than 3 years ago | (#35832060)

Since I use the shell as my preferred work environment, rolling up commands into scripts simply multiplies my existing effectiveness with very little extra effort.

Correspondingly, what I learn from scripting improves my ability to act quickly in a 'live' shell.

Since I work in PHP (mostly), I use it for anything that BASH can't do, since then I have access to my application libraries. If my app was in Ruby I'd use that.

Re:Why are there still shell scripts anyways? (1)

smcdow (114828) | more than 3 years ago | (#35832066)

initscripts. If you're not writing them, you aren't a software developer.

Re:Why are there still shell scripts anyways? (1)

maxwell demon (590494) | more than 3 years ago | (#35832466)

initscripts. If you're not writing them, you aren't a software developer.

And I always thought it's the sysadmins who write the init scripts ...

I guess anyone who writes software on Windows or OS X, or writes application software which is started by the user, not by the system at startup, as well as anyone doing embedded development on specialized devices which don't run a traditional operating system at all, isn't a software developer, then :-)

Re:Why are there still shell scripts anyways? (1)

Bork (115412) | more than 3 years ago | (#35832082)

Because from a shell script I can run anything else. From a shell script I can tie the other together into one script. Because it does what I want it to do.

Re:Why are there still shell scripts anyways? (1)

h4rr4r (612664) | more than 3 years ago | (#35832394)

Or you could use perl and `` to run whatever you want.
I use bash all the time, but sometimes perl is the way to go.

Perl server pages. (1)

bobs666 (146801) | more than 3 years ago | (#35832084)

Yes, I write web applications with little to no programing. Yes I use scripts, and even some unix commands called from my HTML+scripts.

<<date +%d/%m/%Y>>

works on my pages. ya it prints the date... Powered by PERL.
PS. I can read an write SQL on my web pages. no programing needed.

Re:Why are there still shell scripts anyways? (2)

rallymatte (707679) | more than 3 years ago | (#35832156)

#!/bin/sh
echo "Lot of the time it's just quick and easy"

Re:Why are there still shell scripts anyways? (1)

Ruke (857276) | more than 3 years ago | (#35832844)

#!/usr/local/bin/perl
print 'I see your point. Perl is just too unwieldy in this case.';

Re:Why are there still shell scripts anyways? (0)

Anonymous Coward | more than 3 years ago | (#35833204)

% ./slashdot.pl
./slashdot.pl: bad interpreter: /usr/local/bin/perl: no such file or directory

Re:Why are there still shell scripts anyways? (1)

cptdondo (59460) | more than 3 years ago | (#35832178)

Try working in an embedded environment. When you have 4 MB of flash and 8 MB of RAM, and python (even a barely usable stripped python) takes 3 MB, you use shell. Most of my programming is on embedded platforms; if I have 32 MB of flash I don't know what to do with it. I use a lot of shell and awk.

Don't even get me started on PERL; it works for some and for others it is the most obtuse incomoprehensible glop you can imagine. Instant cruft. And it takes megabytes of storage.

Just because you can't see it from your house doesn't mean it has ceased to exist.

Re:Why are there still shell scripts anyways? (1)

sjaskow (143707) | more than 3 years ago | (#35832242)

Don't forget that there's still a lot of commercial unix in the work place. Most of those OS's don't come with python or ruby and the perl that does come with them tends to be old and not have very many useful modules already. Learn ksh/bash and you can write scripts on any modern Unix-like OS; only learn perl/ruby/python/current-flavor-of-the-day-language and you won't.

Why do people thing there's only Linux distros? (4, Interesting)

PetiePooo (606423) | more than 3 years ago | (#35832444)

One of my fondest hacks was a shell script. It was for an embedded Solaris box where we didn't have the ability (through written policy) to add any other programs or packages, but scripting was OK. Ultimately, it was a job for expect, as I was communicating via telnet to another platform to upload firmware. But I only had sh, csh, or ksh. I chose vanilla Bourne shell, and wrote a script that accepted as inputs the IP address to upload the file, and the name of the file to upload. It set some variables relaunched itself in the format

exec $0 | telnet $DEST_IP | $0

The script receiving stdout from telnet would watch for input lines, keeping track of what response was expected, and send either a USR1 or a USR2 back to the script outputting commands into telnet's stdin, which would then either bail or send the next command. It was rudimentary pass/fail error checking, but much better and faster than echo a; sleep 1; echo b; sleep 1; echo c...

I often prefer #!/bin/sh as I know it's a common denominator on all *nix systems, and often my scripts work even if the shell is busybox. Yes, perl or python may handle the task more efficiently and with less code, but keeping fresh on Bourne shell makes it easier to fiddle with initramfs images and embedded systems with limited space and no advanced shells. In fact, I'll often stick with it just for the challenge. I like making the original Bourne shell truly sing...

Open your eyes. There's much more to the world than just "every major (and most minor) distros." Even more than just Linux and the BSDs.

Crikey. If nobody learned assembly, we wouldn't even have operating systems.

Re:Why do people thing there's only Linux distros? (1)

turgid (580780) | more than 3 years ago | (#35832590)

You rule.

Re:Why are there still shell scripts anyways? (4, Insightful)

Zenin (266666) | more than 3 years ago | (#35832496)

In particular Bourne shell makes gluing other programs together far easier, cleaner, and more reliably then can be done with any of the languages you list...and I say that even given that I'm extremely fluent in at least two of them.

Correctly and reliably handling even a simple "foo | bar | baz" construct in the languages you list can be done, but you're talking about an LOT of non-trivial systems level programming to do it, effectively "coding C inside perl/python/ruby". No, really. Sure, it's very easy to do it wrong and fragile (just call system(), what's so hard right?), but if you actually want to handle it as correctly and reliably as Bourne does out of the box...it's going to take a hell of a lot more code and detailed systems programming knowledge then even most "Sr" Unix sysadmins tend to have. Almost without fail ever attempted I've ever seen in the industry gets it wrong...typically very, very wrong.

Bourne is built to run other programs, to manage the interactions between them, programming logic being the exceptional use. All of the languages you list are built to be self-contained, programming logic being the primary task and communicating with external programs a very secondary use.

Now of course...if your script/program is self-contained and not just wrapping a bunch of exec()s of other programs, then sure Bourne shell is one of the least favorable options.

There's also nothing saying you can't freely mix and match. Most anyone worth their salt does so all the time. To do otherwise would be like saying a web coder could only work in HTML or only work in JavaScript or only work in PHP. We're talking about complimentary languages here, with some overlap.

Use the right tool for the job.

Re:Why are there still shell scripts anyways? (1)

blair1q (305137) | more than 3 years ago | (#35832854)

Shell scripting lets you turn things you do on a shell command-line into scripts, without changing them.

That you can then cruft them up with reams of flow-of-control syntax, variables out the wazoo, optargs for days, and trap statements, is just a bonus.

Sure you would have done it better in Python. But if you don't know Python you take what you type into the shell more than once, and make a script out of it.

Re:Why are there still shell scripts anyways? (1)

hawguy (1600213) | more than 3 years ago | (#35832862)

I still use Bash for most automation tasks because there's no guarantee that the next person in my job is going to be familiar with Perl, Ruby, and Python, but every unix admin is going to know Bash.

If all I need to do is start up a job, make sure it created a non-zero length output file, and log a message if it ends with a non-normal return code, bash is simple and easy and has no real drawbacks.

For anything non-trivial (i.e. a monitoring plugin to do a transaction on our website) than I would use Perl (because it's what I'm most comfortable with, someone else might choose Python or Ruby), but 90% of the time when I need a script to do something, Bash is just fine and I know that the next guy in my job is going to be able to maintain the script.

Re:Why are there still shell scripts anyways? (1)

mangu (126918) | more than 3 years ago | (#35832952)

Why does anyone still use shell scripts anymore?

Because they are the simplest and quickest way to do a lot of things one needs to do when using a computer. Try this in python, ruby, or perl:

the disk is nearly full, find who is the hog

in bash:

du -x / | sort -n

Now do it in another language

Re:Why are there still shell scripts anyways? (1)

cloudmaster (10662) | more than 3 years ago | (#35833316)

You listed the options backwards. Most UNIXes (you might be shocked to find that Linux isn't the only Unix variant people use) do not ship with Python, and I do everything I can to avoid having Ruby installed on even my Linux systems. And while I love perl, you are the first person I've heard describe its syntax as sane. :)

Re:Why are there still shell scripts anyways? (1)

Bigbutt (65939) | more than 3 years ago | (#35833522)

Why does anyone still use shell scripts anymore?

Because I'm a Unix Sysadmin?

Shell scripting is something I haven't felt the need to do in 5 years.

I'm guessing you're not a Unix Sysadmin any more?

If we were interviewing someone who said they hadn't written a shell script in 5 years, I'd have to spend some extra time querying the reasons and then think twice about hiring him. See, we use shell scripts. For init scripts, for simple tasks, for data gathering; loop through 300 systems getting the uptime for instance. Someone we hire would need to be able to perform maintenance on the scripts and hopefully not break an existing script or start using something like Python that the other sysadmins may not know and have to either learn in order to maintain your scripts or convert to regular shell scripts again.

See I've dealt with people who have their own little way of doing things. We had one sysadmin who felt Rexx was a great scripting language. Which was fine for him but when he left, we had to convert his scripts into a shell script in order to continue to maintain things.

So we have a rule that system scripts are done using a specific template, which is editable of course in case of need, which manages several aspects of system scripts including checking for multiple copies of the script and killing sessions if scp to a system hangs for some reason.

[John]

Who uses shell scripts today? (-1, Troll)

backslashdotcomma (2042342) | more than 3 years ago | (#35831916)

With wealth of languages like the Python for instance, who needs them? Besides, if you look at History of UNIX [tinyurl.com] you'll see that shell scripts weren't intended for programming

Re:Who uses shell scripts today? (2, Interesting)

Anonymous Coward | more than 3 years ago | (#35832046)

OT: What is it with url shortening services? At best, the damn service works and it's a blind link to what the author claimed (which is inferior to just LINKING). At worst it's goatse. Somewhere in between is someone who f-ed up using it (like parent). Can Slashdot please ban links to url shortening services?

Re:Who uses shell scripts today? (1)

h4rr4r (612664) | more than 3 years ago | (#35832630)

It really should just expand them when the comment is submitted.

Re:Who uses shell scripts today? (0)

Anonymous Coward | more than 3 years ago | (#35832780)

I move they ban any user that clicks them without first expanding them.

More efficient scripts (0)

Anonymous Coward | more than 3 years ago | (#35831948)

In summary, probably the only thing I'd have liked to see more of is some emphasis on how to write more efficient shell scripts.

For that, you'd want a book on Perl, Python or Ruby.

Really, as soon as you've got more than a screenful of shell code, you're better off rewriting it in a regular scripting language. You can do amazing things with Unix utilities, but a huge amount of it winds up requiring launching 6 processes to, e.g., read the greatest value of the third field of the output of 'ls'. That's nuts.

Re:More efficient scripts (1)

Anrego (830717) | more than 3 years ago | (#35832114)

Agreed!

This book is probably interesting, but in my opinion if you are doing anything beyond a basic set of commands run in order (with _maybe_ some very primitive control statements) .. you are far better off using a more powerful scripting language... or even a real programming language.

Personally I don't like shell scripts at all for anything more complex than a temporary convinience. From what I've seen they are usually poorly written (lacking proper error handling and logging), are usually riddled with dependancies, and fragile. Oh, they added a few spaces in the output of some command you call? And your script didn't notice and just kept right on doing? Well now you're screwed!

Obviously you can have the same kind of problems with a real programming language.. but shell scripts just seem to encourage bad things to happen.

Re:More efficient scripts (1)

bigredradio (631970) | more than 3 years ago | (#35832146)

Is this what you were thinking of?

ls -l | awk {'print $5'} | sort | tail -n1

Re:More efficient scripts (1)

prgrmr (568806) | more than 3 years ago | (#35832222)

nope, it only takes three:

ls -l |sort -k3|tail -1

Re:More efficient scripts (1)

Tsiangkun (746511) | more than 3 years ago | (#35833036)

ls -l | perl -ane 'print "$F[2]\n" '
Two processes.

Remember the torture (-1, Troll)

backslashdotperiod (2042346) | more than 3 years ago | (#35831978)

What is like learning to code in C-shell? Hate shells, all of them, even this one [tinyurl.com]

Re:Remember the torture (0)

Anonymous Coward | more than 3 years ago | (#35832026)

my eyes are burning... argh! Damn you!

Re:Remember the torture (0)

Anonymous Coward | more than 3 years ago | (#35832040)

MY EYES... dude i am at work here "S

MOD DOWN (0)

Anonymous Coward | more than 3 years ago | (#35832064)

mod down, link is gotse

BAN "backslashdotperiod (2042346)". (1)

mmell (832646) | more than 3 years ago | (#35832126)

=====> You must be this intelligent to ride the internet.

Apparently, backslashdotperiod is a model and wants us all to see his picture.

Re:Remember the torture (1)

Zontar The Mindless (9002) | more than 3 years ago | (#35832546)

That's somewhat clever, but some of us do know what base-64 encoding is.

Useful (1)

MonsterTrimble (1205334) | more than 3 years ago | (#35832036)

This would actually be pretty useful for total noobs (to Bash) like me. I haven't played with programming since 1997 and Fortran. Before that it was C64 Basic. I find that lots of the resources online about Bash scripts are not very good at explaining what the heck is what, syntax and how it all goes together. I can muddle my way through it, but not at the level I like.

Re:Useful (4, Informative)

StuartHankins (1020819) | more than 3 years ago | (#35832094)

http://www.tldp.org/LDP/abs/ [tldp.org] is helpful -- I printed and older version of it 2-up and duplexed it, then comb-bound it. It's been very handy.

Re:Useful (1)

MonsterTrimble (1205334) | more than 3 years ago | (#35832372)

You Sir, I owe a coke.

reviewer isn't a sysadmin (4, Insightful)

prgrmr (568806) | more than 3 years ago | (#35832160)

First of all, let me remark by saying that shell scripting is something learned more on a need basis than as a tool to solve the main problem. People would seldom write shell scripts as standalone programs

Seriously? So you write your application's /etc/init.d scripts in something other than bash (or csh if on solaris, or ksh if on aix)? Granted not everyone would want, let alone try, to write 500+ line bash scripts like I occasionally do; but, there are a ridiculous number of 100+ line perl scripts that could have been done in bash or ksh in fewer lines, and with more clarity, and without the overheard of loading dozens of perl modules due to interdependencies among them.

Re:reviewer isn't a sysadmin (1)

Anonymous Coward | more than 3 years ago | (#35832384)

Seriously? So you write your application's /etc/init.d scripts in something other than bash (or csh if on solaris, or ksh if on aix)? Granted not everyone would want, let alone try, to write 500+ line bash scripts like I occasionally do; but, there are a ridiculous number of 100+ line perl scripts that could have been done in bash or ksh in fewer lines, and with more clarity, and without the overheard of loading dozens of perl modules due to interdependencies among them.

You're right. Sorry to have said that; I had myself in mind when I wrote it.

I stand corrected. :-)

Kumar

Re:reviewer isn't a sysadmin (1)

Culture20 (968837) | more than 3 years ago | (#35833432)

csh if on solaris

Speaking of csh on solaris, I had a devil of a time figuring out why my csh scripts never worked until I realized that I needed to always put a newline character on the last line (the system wouldn't parse the last line without it). I slowly learned to hate csh's oddities as I learned tcsh (and later, [ba]sh).

i can forkgive if you can forkget (2)

digitalsushi (137809) | more than 3 years ago | (#35832182)

So the bash fork bomb got popular back while I was at UNH. If you're not familiar with it, it's similar to this: ;(){ :|:& };: (I changed one character so that you dont paste it on recommendation) The way it works is trivial, and it's mystique is only in that it uses punctuation marks in lieu of letters for its own name: make a new function called :, run it and force the output into a new fork of itself in the background; then a final call to the new function. h4rdc0r3 1337.

So, we had this UNH policy that if you hung a shared unix dev box, you got evicted from CEPS (the college for the engineering/cs kids). They could give you a warning or go straight to evict. Their call.

I wrote to their head unix admin once, and with some humility, cause I knew they knew this, pointed out that they could just change the system's ulimit values to disallow casual fork bombing. The default system ulimits on the distro back then (still?) allowed a single user to consume all memory. So we saw a couple kids run the fork bomb and they never actually got evicted, cause maybe 40 grand tuition outvotes a cranky unix admin's wanton lust for cruelty.

Anyways, as a student, my issue was this - set the damned ulimit so that my editor session doesn't get wiped with my homework -- you can't argue for frequent backups when more than 0 seconds of hard work are arbitrarily on the line solved by a simple config change.

But the UNH unix admin guy replied to me and told me that yeah, he knew about ulimit back when I was in diapers, but he wasn't going to change it up. And a few more times that semester, I lost a couple lines of uncommitted code due to some clown pressing enter on a dare.

Fork ya later,

Obviously, these guys haven't used UNIX. (4, Insightful)

mmell (832646) | more than 3 years ago | (#35832196)

There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.

If I stick it in root's cron on, say, an AIX system and it pukes I'm not interested in rewriting somebody's bash script to handle running under sh, csh or ksh. I want it written by its original author in sh. And I don't want to hear about a shebang line (#! /bin/bash) - if I haven't installed bash, it'll still puke.

Re:Obviously, these guys haven't used UNIX. (1)

grub (11606) | more than 3 years ago | (#35832436)

A longstanding pet peeve of mine are systems that symlink /bin/bash to /bin/sh

Pretty much any Linux system, for example.

Re:Obviously, these guys haven't used UNIX. (0)

Anonymous Coward | more than 3 years ago | (#35832534)

Here is how new I am to LINUX / UNIX. What is the difference?
Please explain.

Re:Obviously, these guys haven't used UNIX. (0)

Anonymous Coward | more than 3 years ago | (#35832622)

Pretty much any Linux system, for example.

On Debian (and Ubuntu?) /bin/sh is a link to dash, not bash.

Re:Obviously, these guys haven't used UNIX. (1)

Culture20 (968837) | more than 3 years ago | (#35833504)

A longstanding pet peeve of mine are systems that symlink /bin/bash to /bin/sh

Doesn't bash emulate sh if it's called as sh?

Re:Obviously, these guys haven't used UNIX. (0)

Anonymous Coward | more than 3 years ago | (#35833034)

Oh my god, bash is not available in a fantasy situation!
What are we gonna do?

Re:Obviously, these guys haven't used UNIX. (1)

tnk1 (899206) | more than 3 years ago | (#35833390)

There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.

If I stick it in root's cron on, say, an AIX system and it pukes I'm not interested in rewriting somebody's bash script to handle running under sh, csh or ksh. I want it written by its original author in sh. And I don't want to hear about a shebang line (#! /bin/bash) - if I haven't installed bash, it'll still puke.

That makes good sense if you are writing scripts that you will have the need to run from maintenance mode, but that's a pretty small amount of administrative scripting when it comes down to that.

However, if you are writing all of your shell scripts with the assumption that you will not be able to access something as widely available on all UNIX platforms as bash, even when those scripts are not needed in maintenance mode and/or make reference to things that wouldn't be available in maintenance mode anyway, that's just masochistic.

Re:Obviously, these guys haven't used UNIX. (1)

DiegoBravo (324012) | more than 3 years ago | (#35833572)

> mainly it's available on ALL UNIX systems

Mainly it doesn't matter since this book is about LINUX shell scripting.

Besides Bash, a lot of tricks and utilities are only available if you assume a Linux environment.

Re:Obviously, these guys haven't used UNIX. (1)

Culture20 (968837) | more than 3 years ago | (#35833578)

There's a reason we use /bin/sh (Bourne) to write scripts, and it's the same reason we use vi to edit them - mainly it's available on ALL UNIX systems. Despite widespread adoption, bash is not universally available - sorta like EMACS.

But not all GNU/Linux systems. Gentoo is a prime example. Nano instead of vi. And as sibling stated: almost no GNU/Linux system has true sh, they symlink sh to bash.
Oh, and don't stick things in a crontab on Mac OS X systems. Mac OS X prefers XML plist files for scheduling and startup.

In Soviet Russia (0)

Revek (133289) | more than 3 years ago | (#35832260)

Script replaces you.

I dare you to diagram this sentence! (0)

Anonymous Coward | more than 3 years ago | (#35832474)

This, people may or may not like, but the fact that bash has become ubiquitous in terms of the available shells on Unix-like systems today, starting out with bash is not a bad thing to do.

Go on, I dare you to diagram that sentence!

This book gets the details critically wrong. (4, Informative)

lhunath (1280798) | more than 3 years ago | (#35832608)

I've just had a look through the book via Amazon's Look Inside.

I simply cannot fathom how so many authors manage to get a book published on a subject they don't even master properly. "Shell scripting" is by far the worst in this category. So much that I recommend anyone to run screaming in the other direction when you see mention of "shell scripting" in any title. If it mentions "Linux", it's usually also an excellent indicator of junk. Not to take away from the beauty of the Linux kernel (which obviously has nothing at all to do with shell scripting, but let's ignore that fact).

I obviously haven't read the book, but when I skim through it and I see failure to quote parameters, failure to recognize the difference between executing a script with /bin/bash in the hashbang and running it by passing it as an argument to sh, and more, I know for a fact that I'm going to be sitting in #bash trying to re-educate poor misguided souls for as long as I can suffer it.

To those interested in the subject: Do not read this book. Do not read this "Advanced Bash Scripting Guide" mentioned in this review. You will end up writing junk.
Go see Greg Wooledge's wiki, it's got an extensive FAQ of actually useful issues (http://mywiki.wooledge.org/BashFAQ), it's got a great newbie guide (http://mywiki.wooledge.org/BashGuide), it's got a great cheat sheet (http://mywiki.wooledge.org/BashSheet), and much more. Best of all: Unlike that broken ABS, they're all community supported and unlike this book, the knowledge is free.

Re:This book gets the details critically wrong. (1)

turgid (580780) | more than 3 years ago | (#35832688)

What's wrong with the Advanced Bash Scripting Guide at tldp.org?

It's not perfect, but it will get you along way.

Re:This book gets the details critically wrong. (1)

lhunath (1280798) | more than 3 years ago | (#35833462)

"It's not perfect" is an understatement. Firstly, it is a key learning resource for a lot of beginners and more knowledgeable users. That means it has a certain responsibility: To be correct.

Bash is a very lax interpreter, most sh's more so. As a result, they allow you to do a lot of stuff you really, *really* shouldn't be doing, and your test cases will make you think that doing it that way actually "works". Combine that with teaching sloppy practices and you naturally get bad results: People don't know what quoting really is and how crucially necessary it is. People don't pay attention to separating data from code which means most scripts are very vulnerable to arbitrary code execution, but more importantly, bugs such as receiving unexpected data can have devastatingly destructive results, and you might not even notice. Most people don't even understand what the difference is between running "bash" and running "sh" (while I'm sure more understand the similar difference between C and C++).

The ABS is very popular. A lot of people read it and learn from it. The problem is that the code in its examples have terrible practices, important key details are not explained, let alone in sufficient detail, and beginners learning from this will not notice until they shoot themselves in the foot (and will end up blaming bash for it). The resources I enumerated are actively monitored by very knowledgeable people. They are extensive, detailed, clear and useful. But above all, they are correct.

Re:This book gets the details critically wrong. (4, Funny)

Greg W. (15623) | more than 3 years ago | (#35833226)

Go see Greg Wooledge's wiki

Just not all at once, please.

yahoo (1)

Hazel Bergeron (2015538) | more than 3 years ago | (#35832808)

The only interesting thing about this book was to find the author's web site and see how many hoops there are nowadays to get through the Yahoo interview process. I've a friend who managed to get a senior sysadmin role at Yahoo a few years ago who can barely program and has no computer science knowledge whatever - his only previous experience was some ISP admin work, although he was a tremendous suck-up to^W^W^Wgood friends with a few core FreeBSD developers. Anyway, I had the impression that the company was a has-been which runs on inertia.

Alternative book (0)

Anonymous Coward | more than 3 years ago | (#35833110)

This text shows how to write portable Bourne shell, a virtue IMO:

http://www.amazon.com/Portable-Shell-Programming-Extensive-Collection/dp/0134514947

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>