Shell scripting: the silent hero

In 5+ years of using Linux I’ve never really taken the time to learn shell scripting. A combination of using mostly graphical tools, Emacs and other scripting languages means that I’ve rarely found myself in a position where I’ve had to depend on a shell script. In fact the most I’ve done is writing short aliases for longer commands I use on a regular basis. When I needed to do automation I generally fell back on Python (and more recently Ruby). I prefer using a small, uniform set of tools so I tried to do as much as I could within a bunch of Python scripts.

I’ve been doing some data collection work for my thesis and normally I would have been perfectly fine using Ruby and Rake for automation. Rake is Ruby task runner, similar to good old Make but with all the power of Ruby behind it. But I needed to run the same set of tasks under different Ruby VMs (specifically MRI and JRuby). So some of the automation needed to be outside of Ruby. I currently have a shell script that uses RVM to set the Ruby VM, runs a set of Rake tasks, changes over to another VM and repeats the Rake tasks. Each VM sends output to a different file. In between each iteration of Rake tasks, the shell script drops in the iteration number into the output file for that VM. I could have had this happen inside Ruby by passing the iteration number to the Rake task, but it’s easier to put it in from the shell script.

I’m using the BASH shell, mostly because it’s what comes default on Linux boxes, but I think what I’m going to say will be true of other shells like zsh as well. Shell scripting is a powerful tool for lots of reasons. Firstly the scripting language itself is pretty full featured. It may not be as clean and “batteries included” as Python or Ruby, but there’s enough there to get a lot done. Variables, conditionals, loops and functions are all there and the syntax is pretty relaxed compared to some languages (especially when it comes to interpolating strings or commands with variables). BASH also plays to the strengths of the Unix philosophy. It works well with the small, powerful UNIX utility programs and the idea that “everything is a file”. Being able to directly run programs and then pipe the results around is an extremely flexible way to get things done. IO redirection is also very powerful, especially if you’re working with files (like I am).

Perhaps the best thing about Bash scripting (that has been inherited by Perl, Python, Ruby and the rest) is that you can get stuff done with very little knowledge of shell scripting. It’s just as easy to pick up new tricks and tools as you go along. Personally, though I had only used aliases for years I was able to move up to functions, loops and conditionals in just a few hours. I wouldn’t say I’m a proficient shell scripter, but I could certainly hold my own against some ugly automation task at this point. Admittedly having a few years experience in similar languages helped as I knew what to look for.

I’m very glad I got a chance to add shell scripting to my personal arsenal of programming tools. For me, I’m not going to sit down with a book to study shell scripting in any formal way. I’ll keep my current knowledge tucked away in my brain for future reference and pulling it out (maybe adding to it) when I need to. It’s definitely a learn-as-you-go skill for me, but one that I’m glad to have.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s