Python Scripts as a Replacement for Bash Utility Scripts

Often in Python scripts that are used on the command line, arguments are used to give users options when they run a certain command. For instance, the head command takes a -n argument that takes the number following it and prints only that number of lines. Each argument that is provided to a Python script is exposed through the sys.argv array, which can be accessed by first importing sys. The code below shows how to take single words as arguments. This program is a simple adder, which takes two number arguments and adds them, and prints that out to the user. However, this format of taking in command-line arguments is rather basic. It is easy to make mistakes—for instance, pass two strings, such as hello and world, to this command, and you will start to get errors:

#!/usr/bin/env python
import sys

if __name__ == "__main__":
    # The first argument of sys.argv is always the filename,
    # meaning that the length of system arguments will be
    # more than one, when command-line arguments exist.
    if len(sys.argv) > 2:
            num1 = long(sys.argv[1])
            num2 = long(sys.argv[2])
            print "This command takes two arguments and adds them"
            print "Less than two arguments given."
    print "%s" % str(num1 + num2)

Thankfully, Python has a number of modules to deal with command-line arguments. My personal favorite is OptionParser. OptionParser is part of the optparse module that is provided by the standard library. OptionParser allows you to do a range of very useful things with command-line arguments:

  • Specify a default if a certain argument is not provided.

  • It supports both argument flags (either present or not) and arguments with values (-n 10000).

  • It supports different formats of passing arguments—for example, the difference between -n=100000 and -n 100000.

Let's use the OptionParser to enhance the sending-mail script. The original script had a lot of variables hard-coded into place, such as the SMTP details and the users' login credentials. In the code provided below, command-line arguments are used to pass in these variables:

#!/usr/bin/env python
import smtplib
import sys

from optparse import OptionParser

def initialize_smtp_server(smtpserver, smtpport, email, pwd):
    This function initializes and greets the SMTP server.
    It logs in using the provided credentials and returns the
    SMTP server object as a result.
    smtpserver = smtplib.SMTP(smtpserver, smtpport)
    smtpserver.login(email, pwd)
    return smtpserver

def send_thank_you_mail(email, smtpserver):
    to_email = email
    from_email = GMAIL_EMAIL
    subj = "Thanks for being an active commenter"
    # The header consists of the To and From and Subject lines
    # separated using a newline character.
            from_email, subj)
    # Hard-coded templates are not best practice.
    msg_body = """
    Hi %s,

    Thank you very much for your repeated comments on our service.
    The interaction is much appreciated.

    Thank You.""" % email
    content = header + "\n" + msg_body
    smtpserver.sendmail(from_email, to_email, content)

if __name__ == "__main__":
    usage = "usage: %prog [options]"
    parser = OptionParser(usage=usage)
    parser.add_option("--email", dest="email",
            help="email to login to smtp server")
    parser.add_option("--pwd", dest="pwd",
            help="password to login to smtp server")
    parser.add_option("--smtp-server", dest="smtpserver",
            help="smtp server url", default="")
    parser.add_option("--smtp-port", dest="smtpserverport",
            help="smtp server port", default=587)
    options, args = parser.parse_args()

    if not ( or options.pwd):
            parser.error("Must provide both an email and a password")

    smtpserver = initialize_smtp_server(options.stmpserver,
            options.smtpserverport,, options.pwd)

    # for every line of input.
    for email in sys.stdin.readlines():
            send_thank_you_mail(email, smtpserver)

This script shows the usefulness of OptionParser. It provides a simple, easy-to-use interface for command-line arguments, allowing you to define certain properties for each command-line option. It also allows you to specify default values. If certain arguments are not provided, it allows you to throw specific errors.

So what have you learned? Instead of replacing a series of bash commands with one Python script, it often is better to have Python do only the heavy lifting in the middle. This allows for more modular and reusable scripts, while also tapping into the power of all that Python offers. Using stdin as a file object allows Python to read input, which is piped to it from other commands, and writing to stdout allows it to continue passing the information through the piping system. Combining information like this can make for some very powerful programs. The examples I have given here are all for a fictional service that logs to a file.

As a real-world example, recently I have been working with gigabytes of CSV files that I have been converting using a Python script to a file that contains SQL commands to insert the information. To understand the sort of data I'm concerned with here, I ran the data for a single table, and the script took 23 hours to execute and generated an SQL file that was 20GB in size. The advantage of using a Python script in the fashion described in this article is that the whole file does not need to be read into memory. This means that an entire 20GB+ file can be processed one line at a time. Also it is easier to think about a problem when each step (reading, sorting, manipulation and writing) is separated into these logical steps. The guarantee that each of these commands, which are part of the core utilities of UNIX-like environment, is efficient and stable helps the entire experience to be more stable and secure.

The other benefit is that there is no hard-coded file that is read in. Often having the flexibility to pass it strings rather than the concept of files is very powerful. For instance, if 20,000 lines through a certain file, the script breaks, instead of re-running the script from the start, tail can be used to read only from the line on which the script failed.

There are a lot of aspects to Python in the shell that go beyond the scope of this article, such as the os module and the subprocess module. The os module is a standard library function that holds a lot of key operating system-level operations, such as listing directories and stating files, along with an excellent submodule os.path that deals with normalizing directories paths. The subprocess module allows Python programs to run system commands and other advanced operations, such as handling piping as described above within Python code between spawned processes. Both of these libraries are worth checking out if you intend to do any Python shell scripting.




Comment viewing options



Anonymous's picture


Great post. I must say thanks

Raza's picture

hot girls

WONDERFUL Post.thanks for

dennishobson's picture


Between that system, the

laulan's picture


This article opens my mind

Rumah Inicio's picture


Reply to comment | Linux Journal's picture

This post will help the internet visitors for
setting up new website or even a weblog from
start to end.


how to last longer in bed's picture


more this great post

tipsinfomuda's picture


Very efficiently written post

how to last longer in bed's picture


It will do all the logging

alextony's picture


Perl is used that way for

CK's picture


Incorporating is one of the

incorporate's picture


Thanks for sharing this useable article

raspberryketonesideeffects's picture

raspberryketonesideeffects | meratol reviews | proactols | capsiplex plus reviews |

Unconvencing Objective

Anonymous's picture


Thanks for sharing this useable article

raspberryketonesideeffects's picture

raspberryketonesideeffects | meratol reviews | proactols | capsiplex plus reviews |

Thanks for sharing this useable article

raspberryketonesideeffects's picture

raspberryketonesideeffects | meratol reviews | proactols | capsiplex plus reviews |

It sound that this is a

obdii's picture


It seems interesting thanks

Web Design Firm's picture

It seems interesting thanks for sharing this useful information.

It allows you to try the new

car diagnostic tool's picture


That is very interesting

reillyariel's picture

free cell phone spy software
cell spyware

Aftermarket products can

Tony lincle's picture


Very well made and very good

Voyance sérieuse par mail's picture


new text to read

Anonymous's picture


Reply to comment | Linux Journal

Cathern's picture


Please let me know where you got your theme. Appreciate it

Reply to comment | Linux Journal

how pick up women at the club's picture

Your way of telling everything in this paragraph is

Reply to comment | Linux Journal

Port Phillip Bay Fishing Charters's picture

Thnx for providing these details within your website.

Reply to comment | Linux Journal

how to pick up girls san andreas's picture


Reply to comment | Linux Journal

skin disorders hypothyroidism's picture

Hi there! I know this iѕ somеwhаt off topic but


AWK as a half-way solution

Magic Banana's picture

As other readers mention, the proper solution is:
$ sort names.log | uniq -c

$ awk '{ ++names[$0] }

Pretty straightforward, isn't it?

tiny bug

perfectionatic's picture



aishavax's picture

Disgrace on Google for now not positioning this submit upper!


Felipe's picture




Reply to comment | Linux Journal

try this blog page's picture

a user of internet thus from now Ι am usіng net
for агticlеs or reνiews, thanks to web.

Perl is used that way for

hmepas's picture



Excelent article. I will be

Anonymous's picture


why not +x it's picture


cat nameslog |

Would seem a bit more elegant to me.

Guys, it's really strange to

Anonymous's picture


Thanks for the tips

anonymous's picture




I see the point in showing

Anonymous's picture


Hey of course you are

Richy Delaney's picture

Hey of course you are right,

this was written only to illustrate piping.


python -s

Roberto S.'s picture





Anonymous's picture



could be done in one line with perl

Anonymous's picture

without even writing a script, using "perl -ne"

sort -u

Anonymous's picture

$ cat names.log | sort | uniq | wc -l


Are you trying to make things look complicated?

sort -u names.log | wc -l

is all you need.

Hey, Thanks for reading the

Richy Delaney's picture


Thanks for reading the article.

sort -u names.log | wc -l





An excellent post. The post

Raza's picture


Command line argument options

Anonymous's picture


This site is, is it not?



uniq -c

Hermann Schwärzler's picture


But for the records: You write on Page 1




You can try install IPython

Anonymous's picture

You can try install IPython for shell, it very easy

Another cool feature i used

kle_py's picture

Another cool feature i used some time ago is