Gearman是一个分布式的任务调度框架,它包括 a client,a worker ,a job server这三部分组成。
Gearman的执行过程:客户端通过客户端API(PHP,C,Perl等)创建一个任务发送到job server上,Job Server 通过客户端的function name 查找合适的worker,并分到该worker上,worker接收到任务后根据worker的规则执行,并返回数据到job Server,而Job Server则把数据返回给客户端,这样Gearman的执行过程就结束了。
用户可以根据不同的需求制定不同的worker来处理不同的任务,将这些worker存放到不同的服务器上,Job Server会根据不同的客户端发送来的任务的function name寻找worker来执行,从而达到为业务服务器减轻压力;
Gearman的安装:
下载http://launchpad.net/gearmand/trunk/0.12/+download/gearmand-0.12.tar.gz
[falcon@www-001 ~/src/]$ wget http://launchpad.net/gearmand/tr ... earmand-0.12.tar.gz
[falcon@www-001 ~/src/]$ cd gearmand-0.12
[falcon@www-001 ~/src/gearmand-0.12]$ ./configure --prefix=/home/falcon/gearmand
[falcon@www-001 ~/src/gearmand-0.12]$ make && make instal
运行gearman 的job Server
[falcon@www-001 ~/src/gearmand-0.12]$ cd ~/gearmand
[falcon@www-001 ~/gearmand]$ ls
bin include lib sbin share
[falcon@www-001 ~/gearmand]$ sbin/gearmand --help
gearmand 0.12 - https://launchpad.net/gearmand
usage: sbin/gearmand
[OPTIONS]Main Options:
-b, --backlog=BACKLOG Number of backlog connections for listen.
-d, --daemon Daemon, detach and run in the background.
-f, --file-descriptors=FDS Number of file descriptors to allow for the process
(total connections will be slightly less). Default is max allowed for user.
-h, --help Print this help menu.
-j, --job-retries=RETRIES Number of attempts to run the job before the job server removes it. Thisis helpful to ensure a bad job does not crash all available workers. Default is no limit.
-l, --log-file=FILE Log file to write errors and information to. Turning this option on also forces the first verbose level to be enabled.
-L, --listen=ADDRESS Address the server should listen on. Default is INADDR_ANY.
-p, --port=PORT Port the server should listen on.
-P, --pid-file=FILE File to write process ID out to.
-r, --protocol=PROTOCOL Load protocol module.
-R, --round-robin Assign work in round-robin order per workerconnection. The default is to assign work in the order of functions added by the worker.
-q, --queue-type=QUEUE Persistent queue type to use.
-t, --threads=THREADS Number of I/O threads to use. Default=0.
-u, --user=USER Switch to given user after startup.
-v, --verbose Increase verbosity level by one.
-V, --version Display the version of gearmand and exit.
-w, --worker-wakeup=WORKERS Number of workers to wakeup for each job received. The default is to wakeup all available workers.
运行Job Server服务
[falcon@www-001 ~/gearmand]$ sbin/gearmand -d
判断gearmand是否运行
[falcon@www-001 ~/gearmand]$ ps -ef|grep gearmand
falcon 9083 1 0 02:46 ? 00:00:00 sbin/gearmand -d -vv
falcon 9112 28298 0 02:47 pts/1 00:00:00 grep gearmand
[falcon@www-001 ~/gearmand]$ netstat -an -t|grep 4730
tcp 0 0 0.0.0.0:4730 0.0.0.0:* LISTEN
到此Job Server运行正常,下面我们可以简单的在本地上测试Worker和Client是否能够正常接收任务
我们这里用gearman命令来测试
[falcon@www-001 ~/gearmand]$ bin/gearman --help
bin/gearman: invalid option -- -
Client mode: bin/gearman [options] [<data>]
Worker mode: bin/gearman -w [options] [<command> [<args> ...]]
公共参数区
Common options to both client and worker modes.
-f <function> - Function name to use for jobs (can give many)处理任务的函数名
-h <host> - Job server host (Job Server主机,默认是localhost)
-H - Print this help menu
-p <port> - Job server port (Job Server端口,默认是4730)
-t <timeout> - Timeout in milliseconds (执行多长时间超时,微秒)
-i <pidfile> - Create a pidfile for the process (创建进程的pid文件)
Client部分参数
Client options:
-b - Run jobs in the background (后台运行任务)
-I - Run jobs as high priority (高优先级运行任务)
-L - Run jobs as low priority (低优先级运行任务)
-n - Run one job per line (逐行执行任务)
-N - Same as -n, but strip off the newline
-P - Prefix all output lines with functions names (在输入结果前面加处理的函数名)
-s - Send job without reading from standard input 执行任务不返回结果
-u <unique> - Unique key to use for job 任务的唯一标识
Worker部分参数
Worker options:
-c <count> - Number of jobs for worker to run before exiting (统计worker进程处理多少个任务后中止)
-n - Send data packet for each line
-N - Same as -n, but strip off the newline
-w - Run in worker mode 以worker模式运行
示例一、以命令行方式模拟worker 和 client来处理任务
开启一个worker,以function name 为 tongji 来处理输入的数据,统计行数并返回结果
[falcon@www-001 ~/gearmand]$ bin/gearman -w -f tongji -- wc -l
模拟客户端连接到Job Server,以tongji函数名来提交一个文件,并接收结果
[falcon@www-001 ~/gearmand]$ bin/gearman -f tongji < /etc/profile
54
示例二、利用Gearman的php扩展来测试Gearman
安装PHP的Gearman扩展模块
[falcon@www-001 ~/src]$ wget http://pecl.php.net/get/gearman-0.7.0.tgz
[falcon@www-001 ~/src]$ cd gearman-0.7.0
[falcon@www-001 ~/src/gearman-0.7.0]$ /home/falcon/php/bin/phpize
......
[falcon@www-001 ~/src/gearman-0.7.0]$ ./configure \
--with-php-config=/home/falcon/php/bin/php-config --with-gearman=/home/falcon/gearmand
[falcon@www-001 ~/src/gearman-0.7.0]$ make && make install
将gearman.so加入到php.ini配置文件使其生效
测试php是否加载gearman模块
[falcon@www-001 ~/php/bin]$ php -m|grep gearman
官方示例:
将提交的字符串翻转后返回
Worker :worker_reverse.php
<?php
$worker= new GearmanWorker();
$worker->addServer();
$worker->addFunction("reverse", "my_reverse_function");
while ($worker->work());
function my_reverse_function($job)
{
return strrev($job->workload());
}
?>
运行worker
$php work_reverse.php &
Client:client_reverse.php
<?php
$client= new GearmanClient();
$client->addServer();
print $client->do("reverse", "Hello World!");
?>
执行client_reverse.php
$ php client_reverse.php
!dlroW olleH
参考资料:
http://gearman.org/index.php?id=getting_started
http://pecl.php.net/package/gearman
http://www.ibm.com/developerworks/cn/opensource/os-php-gearman/