Turn to Python from Perl: example of SSH in multi-threads
I need to execute commands on several remote servers at the same time via ssh. Writing shell script to send commands to servers with code below is quite simple but has trouble to know when all the tasks are finished on all servers.
ssh $ip “nohup sh go.sh > log 2>&1 &”
Then I turn to perl, write following perl code:
#!/usr/bin/perl
use threads;
use Net::SSH::Perl;
use strict;
my @thread_array;
system( "touch $result_file" );
open( ResultHandle, "> $result_file" ) || die "Open result file error...\n";
sub Test
{
my $host = shift;
chomp( $host );
$current_host = $host;
print "\nDebug:Try to connect $host for sequence write test...\n";
eval
{
alarm $time_out;
my $ssh = Net::SSH::Perl->new( $host );
$ssh->login($user, $pass);
my($out,$err,$exit ) = $ssh->cmd($myCmd);
alarm 0;
print ResultHandle " Test\n $host \n $out\n \n $err";
print "$host\n$out\n$err\n$exit\n";
};
}
my @clients = ("10.0.0.4","10.0.0.6");
for(my $i = 0; $i <= $#clients; $i++){
print "thread $i\n";
$thread_array[$i] = threads->new(\&Test, $clients[$i]);
}
foreach my $thread( @thread_array )
{
$thread -> join( );
}
However, when I run this script, it says “Segment fault” when running more than threads, I do not know why and it stops me moving ahead. Quite disappointed!!!
Then I turn to python…first try the library pyssh:
With following code:
import os
import re
import time
import sys
import pyssh
from threading import Thread
class SSHController(Thread):
"""Connect to remote host with SSH and issue commands.
This is a facade/wrapper that uses PySSH to spawn and control an SSH client.
You must have OpenSSH installed.
@ivar host_name: Host name or IP address
@ivar user_name: User name
@ivar password: Password
@ivar prompt: Command prompt (or partial string matching the end of the prompt)
@ivar ssh: Instance of a pyssh.Ssh object
"""
def __init__(self, host_name, user_name, password, cmd):
"""
@param host_name: Host name or IP address
@param user_name: User name
@param password: Password
@param prompt: Command prompt (or partial string matching the end of the prompt)
"""
Thread.__init__(self)
self.host_name = host_name
self.user_name = user_name
self.password = password
self.port = '22' #default SSH port
self.ssh = None
self.cmd = cmd
def login(self):
"""Connect to a remote host and login.
"""
self.ssh = pyssh.Ssh(self.user_name, self.host_name, self.port)
self.ssh.login(self.password)
def run_command(self, command):
"""Run a command on the remote host.
@param command: Unix command
@return: Command output
@rtype: String
"""
response = self.ssh.sendcmd(command)
return self.__strip_output(command, response)
def logout(self):
"""Close the connection to the remote host.
"""
self.ssh.logout()
def run_atomic_command(self, command):
"""Connect to a remote host, login, run a command, and close the connection.
@param command: Unix command
@return: Command output
@rtype: String
"""
self.login()
command_output = self.run_command(command)
self.logout()
return command_output
def __strip_output(self, command, response):
"""Strip everything from the response except the actual command output.
@param command: Unix command
@param response: Command output
@return: Stripped output
@rtype: String
"""
lines = response.splitlines()
# if our command was echoed back, remove it from the output
if command in lines[0]:
lines.pop(0)
# remove the last element, which is the prompt being displayed again
lines.pop()
# append a newline to each line of output
lines = [item + '\n' for item in lines]
# join the list back into a string and return it
return ''.join(lines)
def run(self):
self.run_atomic_command(self.cmd)
print time.ctime()
pinglist = []
for host in range(1,2):
ip = "10.0.0."+str(host)
print ip
current = SSHController(ip,"tao","123456","ls")
pinglist.append(current)
current.start()
for pingle in pinglist:
pingle.join()
print time.ctime()
But again, this script failed. It says: ValueError: signal only works in main thread．I do not know if it’s pyssh library’s problem.
Then I turn to a library named paramiko, and finally I succeed!
First you should install this library; possibly you may need another library pycrypto-2.0.1.
With all things ready, and the following script:
#!/usr/bin/env python
import os, sys, socket
import paramiko
from threading import Thread
class myThread(Thread):
def __init__ (self, ip, usr, pwd, command):
Thread.__init__(self)
self.ip = ip
self.usr= usr
self.pwd= pwd
self.command = command
def run (self):
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(self.ip, username=self.usr, password=self.pwd)
stdin, stdout, stderr = client.exec_command(self.command)
if self.command.startswith("sudo")
stdin.write(“123456\n”)
stdin.flush()
for line in stdout.read().splitlines():
print 'host: %s: %s' % (self.ip, line)
for line in stderr.read().splitlines():
print 'host: %s: %s' % (self.ip, line)
def test (self):
pass # some code to test this class
mythreads = []
hosts = []
cmd = ""
if __name__ == "__main__":
num = int(sys.argv[1])
print "total %d nodes\n" % num
for i in range(2,2+num):
hosts.append("10.0.0." + str(sys.argv[i]))
for i in range(2+num,len(sys.argv)):
cmd += " " + sys.argv[i]
print "cmd to execute: %s\n" % cmd
for ip in hosts:
t = myThread(ip, "tao","123456",cmd)
mythreads.append(t);
t.start()
for thread in mythreads:
thread.join()
print "execution on %s completes!\n" % thread.ip
print "test done!"
This script accepts arguments like this: 2 1 4 du –h
First argument 2 means I want to execute this command on two hosts, and followed with the last part of ip. ( ip of my machines all have the pattern of 10.0.0.*).
Then the command follows the hosts. If you need to execute a sudo command, remember to modify /etc/sudoers to allow you send ssh commands without a real tty.
You can add a line like this to walk around this problem.
Defaults:tao !requiretty