Python is a language that is known for its scripting and automating workflows. Python also comes with packed useful tools out of the box with Python Standard Library. A common thing to do in Python execute a shell command. But it doesn’t actually work every time. It usually ends up being in a bash or batch file. You will know how to Python execute shell command here with the Os and Subprocess Modules.
Also Read: Pointers in python: Basic knowledge you need
What is Python?
Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development. As well as for use as a scripting or glue language to connect existing components together. Python’s simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance. Python supports modules and packages, which encourages program modularity and code reuse.
Features:
- Open Source and Free
- Support for GUI
- Object-Oriented Approach
- High-Level Language
- Integrated by Nature
- Highly Portable
- Highly Dynamic
Using theos Module:
The first thing you have to do when you approach to run the shell command is by using os.system():
import osos.system('ls -l')
You have to do only one simple thing is that to save this script and then run it. Then you will see the output window in a command line. So, the problem with this approach is that in its inflexibility. Because you can’t even get the resulting output as a variable, if you want to read more about this function, you can visit the documentation from here and read and learn more about this information.
But let us tell you something if you run this function in Jupyter Notebook, you will not get an output inline. Instead, you will get an inline output with the return code of the executed program( 0 for successful and -1 for unsuccessfully). You can find the output of the command line where you started the Jupyter Notebook.
Now the os.popen() command line will open a pipe from or to the command line. This means you can get direct access to the stream within Python. This is useful since you can now get the output as a variable:
import os
stream = os.popen('echo Returned output')
output = stream.read()
output'Returned output\n'So when you see the .read() function, you can get a whole output as one string. You can also use the function .readlines(), which will help you split each line( Including a trailing \n).
Note: You can only run them once.
You can also write the stream using the mode='w' argument. To delve deeper into this function, have a look at the documentation.
In this example and the following examples, you will see that you always have trailing line breaks in the output. To remove them (including blank spaces and tabs in the beginning and end), you can use the .strip() function like with output.strip(). To remove those characters only in the beginning, use .lstrip() and for the end. .rstrip().
Using the subprocess Module
This approach is more versatile and simple one than the other one and this one is recommended more than the other.
The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes. (Source)
The main thing you want to know and keep in mind that if you use Python >=3.5 is subprocess.run(), but before we get there let’s go through the functionality of the subprocess module. The subprocess.Popen() is the one responsible for the creation and management of the executed process. Unlike in the previous function this class is executed with only a single line of command with arguments as a list. That means you won’t be able to pipe commands.
import subprocess
process = subprocess.Popen(['echo', 'More output'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = process.communicate()
stdout, stderr(b'More output\n', b'')You can notice that we have set an stdout and stderr to subprocess.PIPE. This indicated the exceptional value of subprocess.Popen. It means that a pipe should be opened that you can then read with the .communicate() function. It is also possible to use a file object as with:
with open('test.txt', 'w') as f:
process = subprocess.Popen(['ls', '-l'], stdout=f)There is another thing that you can notice is the type of output is bytes. You can solve this by doing one simple thing by typing stdout.decode('utf-8') or by adding universal_newlines=True when calling subprocess.Popen.
Then after that you run .communicate(), it will wait until the process is complete. But if you have a long program that you want to run and want to continuously check the status in real time while doing something else you can run the following code and do it like this.
process = subprocess.Popen(['ping', '-c 4', 'python.org'],
stdout=subprocess.PIPE,
universal_newlines=True)
while True:
output = process.stdout.readline()
print(output.strip())
# Do something else
return_code = process.poll()
if return_code is not None:
print('RETURN CODE', return_code)
# Process has finished, read rest of the output
for output in process.stdout.readlines():
print(output.strip())
breakPING python.org (45.55.99.72) 56(84) bytes of data.
64 bytes from 45.55.99.72 (45.55.99.72): icmp_seq=1 ttl=51 time=117 ms
64 bytes from 45.55.99.72 (45.55.99.72): icmp_seq=2 ttl=51 time=118 ms
64 bytes from 45.55.99.72 (45.55.99.72): icmp_seq=3 ttl=51 time=117 ms
64 bytes from 45.55.99.72 (45.55.99.72): icmp_seq=4 ttl=51 time=118 ms
--- python.org ping statistics ---
RETURN CODE 0
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 117.215/117.874/118.358/0.461 msPoll function:
You can use the .poll() this is the function that you can use to check the return code. It will return None if the function is still running. If you want to get an output you can use the single command line of process.stdout.readline() to read.
Also you can conversely use process.stdout.readlines(). It will read all the line of your command and it also waits for the program to finish. If you want to read and get more information on the matter and on the functionality of subprocess.Popen, have a look at the documentation.
Also note, that you won’t need quotations for arguments with spaces in between like '\"More output\"'. If you are unsure how to tokenize the arguments from the command, you can use the shlex.split() function:
import shlex
shlex.split("/bin/prog -i data.txt -o \"more data.txt\"")
['/bin/prog', '-i', 'data.txt', '-o', 'more data.txt']
You can also use a subprocess.call() which is at your disposal if you want it.
process = subprocess.run(['echo', 'Even more output'],
stdout=subprocess.PIPE,
universal_newlines=True)
process
CompletedProcess(args=['echo', 'Even more output'], returncode=0, stdout='Even more output\n')
You can also find other outputs in this variable:
process.stdout
'Even more output\n'
Like the subprocess.call() and also the previous .communicate() function it will wait for your command line to execute. But here is an advanced example on how you can access a server with ssh and the subprocess module:
import subprocess
ssh = subprocess.Popen(["ssh", "-i .ssh/id_rsa", "user@host"],
stdin =subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True,
bufsize=0)
# Send ssh commands to stdin
ssh.stdin.write("uname -a\n")
ssh.stdin.write("uptime\n")
ssh.stdin.close()
# Fetch output
for line in ssh.stdout:
print(line.strip())
Here you can write to input the process. Now in this case you need to set bufsize=0 in order to have a unbuffered output.
Conclusion:
Now we have given you two methods of how Python execute shell command. The effective of ways is to use the Subprocess module. It offers different kind of functionality. Hope you find this useful. Thank You for the read.