ssh remote run of a local file

There are cases when we want to test our local code if it runs on remotely. Note that we run a local code not run a remote code after loading it at remote place. Also, we want to monitor the output from the remote run. So, in this chapter, we'll see how we can run our code remotely and get the outputs from it.

In this chapter, I used my server at http://www.bogotobogo.com/ as a remote location, and the virtual CentOS (Hypervisor: VMWare Workstation 10 ) is used as my local.

In summary, we have list of commands to be executed remotely and get the output file with log.

Here is our simple shell script:

# s.sh
uname -r
uptime

As a humble start, let's try a local run first:

$ ./s.sh
2.6.32-358.el6.x86_64
 14:25:38 up 4 days, 15:59,  2 users,  load average: 0.00, 0.00, 0.00

Then, remote run:

$ ssh bogotob1@bogotobogo.com < ./s.sh
Pseudo-terminal will not be allocated because stdin is not a terminal.
bogotob1@bogotobogo.com's password:
3.12.26.1407184750
 23:14:59 up 67 days, 14 min,  1 user,  load average: 42.36, 30.37, 28.13

Our python code:

# s.py
#!/usr/bin/python
import subprocess

cmd_list = ['uname -r', 'uptime']

out = []
err = []

for cmd in cmd_list:
    args = cmd.split()
    proc = subprocess.Popen(args,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE)
    (stdoutdata, stderrdata) = proc.communicate()
    out.append(stdoutdata)
    err.append(stderrdata)

print 'out=',out
print 'err=',err

The following way of remote run won't work:

$ ssh bogotob1@bogotobogo.com python s.py
bogotob1@bogotobogo.com's password:
python: can't open file 's.py': [Errno 2] No such file or directory

However, if we run it this way, we can do remote run:

$ cat s.py  | ssh bogotob1@bogotobogo.com python -
bogotob1@bogotobogo.com's password:
out= ['3.12.26.1407184750\n', ' 16:01:11 up 67 days, 17:00,  1 user,  load average: 19.17, 22.79, 22.32\n']
err= ['', '']

Or another remote run will work if we run it this way:

$ ssh bogotob1@bogotobogo.com python < ./s.py
bogotob1@bogotobogo.com's password:
out= ['3.12.26.1407184750\n', ' 16:48:29 up 67 days, 17:47,  1 user,  load average: 28.73, 23.32, 20.98\n']
err= ['', '']

This time, we run a list of three scripts in a python (s.py):

['python uname.py', 'sh uptime.sh', 'perl hello.perl']

The s.py code looks like this:

# s.py 
#!/usr/bin/python
import subprocess

cmd_list = ['python uname.py', 'sh uptime.sh', 'perl hello.perl']

out = []
err = []

for cmd in cmd_list:
    args = cmd.split()
    print 'args=',args
    proc = subprocess.Popen(args,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE)
    (stdoutdata, stderrdata) = proc.communicate()
    out.append(stdoutdata)
    err.append(stderrdata)

print 'out=',out
print 'err=',err

If we try to run this remotely, we get an error like this:

$ cat s.py  | ssh bogotob1@bogotobogo.com python -
bogotob1@bogotobogo.com's password:
args= ['python', 'uname.py']
args= ['sh', 'uptime.sh']
args= ['perl', 'hello.perl']
out= ['', '', '']
err= ["python: can't open file 'uname.py': [Errno 2] No such file or directory\n", 'sh: uptime.sh: No such file or directory\n', 'Can\'t open perl script "hello.perl": No such file or directory\n']

The error occurred because the code thinks the scripts reside locally. So, we need to run each one remotely using ssh.

So, each command should be run like this:

$ ssh bogotob1@bogotobogo.com python < uname.py
bogotob1@bogotobogo.com's password:
3.12.26.1407184750

However, still the following program does not work. It remote shell still thinks the three codes (['uname.py', 'uptime.sh', 'hello.pl']) are local to it (meaning, they reside at remote location):

# s2.py
#!/usr/bin/python
import subprocess, sys, argparse

def check_arg(args=None):
    parser = argparse.ArgumentParser(description='Script to learn basic argparse')
    parser.add_argument('-H', '--host',
                        help='host ip',
                        required='True',
                        default='localhost')
    parser.add_argument('-u', '--user',
                        help='user name',
                        default='root')

    return parser.parse_args(args)

def remote_run(arguments):
    host = arguments.host
    user = arguments.user

    cmd_list = ['uname.py', 'uptime.sh', 'hello.pl']
    exe = {'py':'python', 'sh':'sh', 'pl':'perl'}

    out = []
    err = []

    ssh = ''.join(['ssh ', user, '@',  host, ' '])
    for cmd in cmd_list:
        type = exe[cmd.split('.')[1]]
        cmd = ssh + type + ' < ' + cmd
        args = cmd.split()
        print 'args=',args
        proc = subprocess.Popen(args,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        (stdoutdata, stderrdata) = proc.communicate()
        out.append(stdoutdata)
        err.append(stderrdata)

    print 'out=',out
    print 'err=',err

if __name__ == '__main__':
    args = check_arg(sys.argv[1:])
    remote_run(args)

Output:

$ python s2.py -H 173.254.28.78 -u bogotob1
args= ['ssh', 'bogotob1@173.254.28.78', 'python', '<', 'uname.py']
bogotob1@173.254.28.78's password:
args= ['ssh', 'bogotob1@173.254.28.78', 'sh', '<', 'uptime.sh']
bogotob1@173.254.28.78's password:
args= ['ssh', 'bogotob1@173.254.28.78', 'perl', '<', 'hello.pl']
bogotob1@173.254.28.78's password:
out= ['', '', '']
err= ['bash: uname.py: No such file or directory\n', 'bash: uptime.sh: No such file or directory\n', 'bash: hello.pl: No such file or directory\n']

But this simple code using os.system() instead of subproccee.Popen() seems to be working:

# s3.py
#!/usr/bin/python
import os

os.system('ssh bogotob1@bogotobogo.com python < uname.py')
os.system('ssh bogotob1@bogotobogo.com sh < uptime.sh')
os.system('ssh bogotob1@bogotobogo.com perl < hello.pl')

$ python s3.py
bogotob1@bogotobogo.com's password:
3.12.26.1407184750
bogotob1@bogotobogo.com's password:
 22:40:06 up 67 days, 23:39,  1 user,  load average: 21.00, 23.46, 22.55
bogotob1@bogotobogo.com's password:
Hello world from perl

If we want to get std output and error as a file from the run, the following code will do that for us ($ python s4.py):

# s4.py
#!/usr/bin/python
import os

os.system('ssh bogotob1@bogotobogo.com python < uname.py >> mylog.txt 2>&1')
os.system('ssh bogotob1@bogotobogo.com sh < uptime.sh >> mylog.txt 2>&1')
os.system('ssh bogotob1@bogotobogo.com perl < hello.pl >> mylog.txt 2>&1')

'mylog.txt' looks like this:

3.12.26.1407184750
 13:51:37 up 68 days, 14:50,  0 users,  load average: 12.23, 18.70, 19.22
Hello world from perl

# s5.py
#!/usr/bin/python
import os, subprocess, sys, argparse

def check_arg(args=None):
    parser = argparse.ArgumentParser(description='Script to learn basic argparse')
    parser.add_argument('-H', '--host',
                        help='host ip',
                        required='True',
                        default='localhost')
    parser.add_argument('-u', '--user',
                        help='user name',
                        default='root')

    return parser.parse_args(args)

def remote_run(arguments):
    host = arguments.host
    user = arguments.user

    cmd_list = ['uname.py', 'uptime.sh', 'hello.pl']
    exe = {'py':'python', 'sh':'sh', 'pl':'perl'}

    ssh = ''.join(['ssh ', user, '@',  host, ' '])
    for cmd in cmd_list:
        type = exe[cmd.split('.')[1]]
        cmd = ssh + type + ' < ' + cmd + '>> mylog.txt 2>&1'
        '''
        args = cmd.split()
        print 'args=',args
        proc = subprocess.Popen(args,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE)
        (stdoutdata, stderrdata) = proc.communicate()
        out.append(stdoutdata)
        err.append(stderrdata)
        '''
        os.system(cmd)

if __name__ == '__main__':
    args = check_arg(sys.argv[1:])
    remote_run(args)

Output:

$ python s5.py -H 173.254.28.78 -u bogotob1
cmd= ssh bogotob1@173.254.28.78 python < uname.py>> mylog.txt 2>&1
bogotob1@173.254.28.78's password:
cmd= ssh bogotob1@173.254.28.78 sh < uptime.sh>> mylog.txt 2>&1
bogotob1@173.254.28.78's password:
cmd= ssh bogotob1@173.254.28.78 perl < hello.pl>> mylog.txt 2>&1
bogotob1@173.254.28.78's password:

# s6.py
#!/usr/bin/python
import os, subprocess, sys, argparse

def check_arg(args=None):
    parser = argparse.ArgumentParser(description='Script to learn basic argparse')
    parser.add_argument('-H', '--host',
                        help='host ip',
                        required='True',
                        default='localhost')
    parser.add_argument('-u', '--user',
                        help='user name',
                        default='root')

    return parser.parse_args(args)

def remote_run(arguments):
    host = arguments.host
    user = arguments.user

    cmd_list = []
    with open('commands.txt', 'rb') as f:
        for c in f:
            cmd_list.append(c.replace('\n','').replace('\r','').rstrip())

    exe = {'py':'python', 'sh':'sh', 'pl':'perl'}

    ssh = ''.join(['ssh ', user, '@',  host, ' '])
    for cmd in cmd_list:
        type = exe[cmd.split('.')[1]]
        cmd = ssh + type + ' < ' + cmd + '>> mylog.txt 2>&1'
        os.system(cmd)

if __name__ == '__main__':
    args = check_arg(sys.argv[1:])
    remote_run(args)

Output:

]$ python s6.py -H 173.254.28.78 -u bogotob1
bogotob1@173.254.28.78's password:
bogotob1@173.254.28.78's password:
bogotob1@173.254.28.78's password:

And the file output, 'mylog.txt':

3.12.26.1407184750
 16:16:23 up 68 days, 17:15,  0 users,  load average: 10.43, 12.55, 15.23
Hello world from perl