If you need to execute a shell command with Python, there are two ways. You can either use the subprocess module or the run command(
command.run() ) function.
The first option is easier to run one line of code and exit, but it isn’t as flexible when using arguments or producing text output. The second option is more complicated to code, but it will allow for more customization in your script.
I’ll be going through how to do both in this blog post!
I recommend you check out Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming. It helped me expand my understanding of shell scripts.
If you’re working with Python and shell scripts, you may also want to check the four different ways to run Python scripts inside a terminal. It’s the opposite of what we have in this post—how to run Python scripts from a terminal. Also, check this if you want to run scripts over ssh on a remote computer.
What is the purpose of shell commands?
Shell is a command-line interface (CLI) for interacting with the operating system. It provides an environment for running text-based commands and typically offers features not available in graphical user interfaces (GUIs), such as piping and redirection.
Shell environment is often called the command prompt in Windows and terminal in Linux systems.
While most users prefer GUI-based tools, advanced operations often command prompt usage. For instance, data scientists, programmers, and system engineers use shell commands to manage files, directories, and software. Additionally, CLIs can be used in batch scripts or cron jobs to automate repetitive tasks.
How to execute a shell command in Python?
Programmers often want to execute a shell command in more extensive Python code. There are two options to do so:
Use the subprocess module.
The subprocess module is a part of the standard library in Python. It provides facilities for executing shell commands and capturing their output or error.
Here’s how to run a shell script using Python. In this example, we’re copying a file from ‘dirA’ to ‘dirB.’
run() is a much quicker way to execute shell scripts, the Popen constructor could be more flexible.
The Popen constructor allows you to pass in additional arguments, such as
shell=True, which will cause the command to run in a sub-shell.
Additionally, you can use
stdout=PIPE to have Popen capture the returned output instead of simply displaying it on the screen.
Here’s a quick example of how to run a shell command and capture its output:
The above Python program executes the shell command ls -lh and prints the result. Note: If we don’t specify
shell=True, Popen will only capture the first output line.
It’s often very convenient to start Popen objects as context managers. We can use the
with statement in Python. Here’s an example.
In the above code snippet, we use
ifconfig as our shell command. The
with statement ensures that the proc object is closed correctly when we exit the block.
This is especially important when you are working with large amounts of data. If you don’t close the proc object, your program will close before all the data is read from stdout. Additionally, it will leave a file descriptor open for your OS.
Use the Command module.
The Command library is another convenient way to execute shell scripts inside Python. The Command module is a wrapper around the Popen method of the subprocess module. But, Command has outstanding capabilities for logging and debugging.
You can install the Command library from PyPI using the below command.
The following Python script executes a shell command.
Also, you could set the debug parameter to a function to handle the logs differently. The following example prints the debug logs on the screen.
Use the os module.
os module in Python provides access to many low-level operating system features. You can use it to list directories, change directories ( cd ), list information about files ( stat ), create files, and more.
Here’s an example of using the
os module to get the list of files in the current directory.
Warning: This Python method to execute bash commands is now deprecated.
os.popen returns a file object, and we use the
read() method to get the data from stdout.
os.system vs. subprocess: What should you use?
The subprocess module is always preferable compared to the
os.system one. This is because the latter has a hike risk of shell injection attack.
You can pass more than one command to the system utility by operating them out using a pipe. The following is an example of listing the files and creating one there.
In addition to getting the list of files and folders in the current directory, the above command also creates a file. Attackers may use this vulnerability to access your servers and do harmful activities.
os.system unless it’s necessary and you understand the risks—it’s not the answer to exec shell scripts.
How to run a script with Python?
Often you might have to run other scripts inside a Python program as you’re running it from the shell. To do this, you can use the subprocess module as well.
In this example, we’ll show how to run a bash script file called
myscript.sh that takes an input filename as an argument.
If you’re using an older version of Python (before 3.5), you should instead use the
Not a Medium member yet? Please use this link to become a member because I earn a commission for referring at no extra cost for you.