If you need to execute a shell command with Python, there are two ways. You can either use the subprocess module or the RunShellCommand() function.
The first option is easier to run one line of code and then exit, but it isn’t as flexible when using arguments or producing text output. The second option is more complicated to code, but it will allow for more customization in your script.
I’ll’ be going through how to do both in this blog post!
I strongly recommend you check out Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming. It helped me expand my understanding of shell scripts.
If you're working with Python and shell scripts you may also want to check the four different ways to run Python scripts inside a terminal. It's the opposite of what we have in this post.
What is the purpose of shell commands?
Shell is a command-line interface (CLI) for interacting with the operating system. It provides an environment for running text-based commands and typically offers several features not available in graphical user interfaces (GUIs), such as piping and redirection.
Shell environment is often referred to as the command prompt in Windows and terminal in Linux.
To open the command prompt in Windows, press Win+R, type cmd, and hit Enter. On Linux, you can open the terminal by pressing Ctrl+Alt+T. Press Cmd+Space, type terminal, and hit Enter if you are on Mac OS.
While most users prefer GUI-based tools, advanced operations often command prompt usage. For instance, data scientists and programmers use shell commands to manage files, directories, and software. Additionally, CLIs can be used in batch scripts or cron jobs to automate repetitive tasks.
How to execute a shell command in Python?
Programmers often want to execute a shell command as inside a more extensive Python code. There are two options to do so:
Use the subprocess module.
The subprocess module is a part of the standard library in Python. It provides facilities for executing shell commands and capturing their output or error.
Here’s how to run a shell script using Python. In this example, we’re copying a file from ‘dirA’ to ‘dirB.’
import subprocess # This is our shell command, executed in subprocess. p = subprocess.run("cp", "dirA/file", "dirB")
run() is a much quicker way to execute shell scripts, you should use the Popen constructor for more flexibility.
The Popen constructor allows you to pass in additional arguments, such as shell=True, which will cause the command to run in a sub-shell.
Additionally, you can use
stdout=PIPE to have Popen capture the returned output instead of simply displaying it on the screen.
Here's a quick example of how to run a shell command and capture its output:
import subprocess # This is our shell command, executed by Popen. p = subprocess.Popen("ls -lh", stdout=subprocess.PIPE, shell=True) print(p.communicate())
The above Python program executes a shell command,
ls -lh and gets an output. Note: If we don’t specify
shell=True, Popen will only capture the first output line.
It's often very convenient to start Popen objects as context managers. We can use the
with statement in Python. Here's an example.
with Popen(["ifconfig"], stdout=subprocess.PIPE) as proc: print(proc.stdout.read())
In the above code snippet, we use
ifconfig as our shell command. The
with statement ensures that the proc object is closed properly when we exit the block.
This is especially important when you are working with large amounts of data. If you don’t close the proc object properly, your program will close before all the data is read from stdout. Additionally, it will leave a file descriptor open for your OS.
Use the Command module
The Command library is another convenient way to execute shell scripts inside Python. The Command module is a wrapper around the Popen method of the subprocess module. But, Command has outstanding capabilities for logging and debugging.
You can install the Command library from PyPI using the below command.
pip install Command
The following Python script executes a shell command.
import command res = command.run(['ls']) print(res.output) print(res.exit)
Also, you could set the debug parameter to a function to handle the logs differently. The following example prints the debug logs on the screen.
import command def debugger(text): print(text) res = command.run(['ls'], debug=debugger) print(res.output) print(res.exit)
Use the os module.
os module in Python provides access to many low-level features of the operating system. You can use it to list directories, change directories ( cd ), list information about files ( stat ), create files, and more.
Here’s an example of how to use
os to get the list of files in the current directory.
import os directories = os.system("ls -lh") print(directories) # You could also use the os.popen() method with os.popen("ls -lh") as f: print(f.readlines())
os.popen returns a file object, and we use the
read() method to get the data from stdout.
How to run a script with Python?
Often times you might have to run other scripts inside a python program as you're running it from the shell. To do this, you can use the subprocess module as well.
In this example, we'll show how to run a script called
myscript.sh that takes an input filename as an argument.
import subprocess subprocess.run(["myscript.sh", "--input_file=input.txt"])
If you're using an older version of Python (prior to 3.5) you should instead use the
Thanks for the read, friend. It seems you and I have lots of common interests. Say Hi to me on LinkedIn, Twitter, and Medium. I’ll break the ice for you.
Not a Medium member yet? Please use this link to become a member because I earn a commission for referring at no extra cost for you.