GearMan Java版试用 03- 简单应用之Client编写和说明

个人认为，Client在gearman中非常重要，肩负着怎么将任务分发到Job Server的责任。

pom.xml文件和worker端的一样，这里就不再赘述了。

代码如下：

package com.baidu.gearman;

import com.google.common.base.Stopwatch;
import org.gearman.*;
import java.util.concurrent.TimeUnit;

/**
 * Hello world!
 */
public class GearClient {
    public static final String ECHO_FUNCTION_NAME = "echo";  //函数名，就是说用哪台机器，哪个端口的哪个函数来执行client的任务，这里的function_name一定要和worker的对应
    public static final String ECHO_HOST = "localhost";  //gearman server 地址
    public static final int ECHO_PORT = 3333;

    public static void main(String[] args) throws InterruptedException {
        Stopwatch stopwatch = new Stopwatch();  //google guava stopwatch计数器，计算任务计算时间
        stopwatch.start();

        Gearman gearman = Gearman.createGearman();  //创建gearman对象，client，worker都有这个对象产生
        GearmanClient client = gearman.createGearmanClient(); 

        GearmanServer server = gearman.createGearmanServer(ECHO_HOST, ECHO_PORT);  //创建server对象
        client.addServer(server);
        System.out.println(">> server count: " + client.getServerCount());

        for (int i = 0; i < 500; i++) {
            GearmanJobReturn jobReturn = client.submitJob(ECHO_FUNCTION_NAME, ("hi,zhaoyang" + i).getBytes());
            while (!jobReturn.isEOF()) {  //根据返回值 是否结束，来判断各种gearman事件状态
                GearmanJobEvent event = jobReturn.poll();  
                switch (event.getEventType()) {
                    case GEARMAN_JOB_SUCCESS:
                        System.out.println(">>>> " + new String(event.getData()));  //获取worker的返回值
                        break;
                    case GEARMAN_SUBMIT_FAIL:
                        System.out.println("### submit fail");
                    case GEARMAN_JOB_FAIL:
                        System.err.println(event.getEventType() + ": " + new String(event.getData()));
                    default:
                }
            }
            Thread.sleep(20);  //如果不休眠，循环提交任务，worker会认为受到攻击，会将任务pending
        }
        gearman.shutdown();  //使用完毕后，一定要将gearman对象进行关闭

        //计算任务执行时间，利用google guava    stopwatch 方法
        stopwatch.stop();
        long nanos = stopwatch.elapsedTime(TimeUnit.SECONDS);
        System.out.println(">>> " + nanos);
    }
}
---------------------------------

Client输出的日志为：

12515 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12516 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
12518 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
>>>> hi,zhaoyang481
12539 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12540 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
>>>> hi,zhaoyang482
12543 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
12563 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12564 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
12569 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
>>>> hi,zhaoyang483
12589 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12591 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
>>>> hi,zhaoyang484
12597 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE

Worker输出的日志为：

3345536 [gearman-87] INFO gearman - [127.0.0.1:3333] : IN : NO_JOB
3345537 [gearman-87] INFO gearman - [127.0.0.1:3333] : OUT : PRE_SLEEP
3345558 [gearman-84] INFO gearman - [127.0.0.1:3333] : IN : NOOP
3345559 [gearman-87] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
3345560 [gearman-84] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
3345561 [gearman-87] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang494
3345561 [gearman-86] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
3345561 [gearman-87] INFO gearman - [127.0.0.1:3333] : IN : NO_JOB
3345562 [gearman-87] INFO gearman - [127.0.0.1:3333] : OUT : PRE_SLEEP
3345583 [gearman-87] INFO gearman - [127.0.0.1:3333] : IN : NOOP
3345583 [gearman-84] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
3345585 [gearman-87] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
3345585 [gearman-84] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang495
3345585 [gearman-86] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
3345586 [gearman-87] INFO gearman - [127.0.0.1:3333] : IN : NO_JOB
3345586 [gearman-87] INFO gearman - [127.0.0.1:3333] : OUT : PRE_SLEEP
3345608 [gearman-84] INFO gearman - [127.0.0.1:3333] : IN : NOOP
3345609 [gearman-87] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
3345611 [gearman-87] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
3345612 [gearman-84] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang496
3345612 [gearman-86] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE

上面准确来说是一种同步的写法，每个任务都要获取worker的返回值后，再进行下一个。500个任务执行完毕后，需要13秒的时间，扣除休眠时间20*500/1000=10秒，500个任务同步执行需要3秒钟

如果通过将任务提交后台执行的方式，就不需要进行thread休眠，速度会快很多，但是没有返回值

只需要将：GearmanJobReturn jobReturn = client.submitJob(ECHO_FUNCTION_NAME, ("hi,zhaoyang" + i).getBytes());

换成：GearmanJobReturn jobReturn = client.submitBackgroundJob(ECHO_FUNCTION_NAME, ("hi,zhaoyang" + i).getBytes());

即可。执行时间为：549毫秒，不到一秒钟。

client 输出的信息为：

99 [gearman-1] INFO gearman - [127.0.0.1:3333] : Connected
103 [gearman-1] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB_BG
106 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
109 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB_BG
110 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED

... ...

Worder端输出的信息为：

@@@@@ hi,zhaoyang496
3054119 [gearman-79] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
3054119 [gearman-75] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
3054119 [gearman-79] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang497
3054120 [gearman-78] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
3054120 [gearman-76] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
3054120 [gearman-79] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang498
3054120 [gearman-78] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
3054120 [gearman-79] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
3054121 [gearman-76] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang499
3054121 [gearman-78] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE

... ...

------------ Client 异步写法 ----------------------

package com.baidu.gearman;

import com.google.common.base.Stopwatch;
import org.gearman.*;
import java.util.concurrent.TimeUnit;

/**
 * Hello world!
 */
public class GearClient implements GearmanJobEventCallback<String> {
    public static final String ECHO_FUNCTION_NAME = "echo";
    public static final String ECHO_HOST = "localhost";
    public static final int ECHO_PORT = 3333;

    public static void main(String[] args) throws InterruptedException {
        Stopwatch stopwatch = new Stopwatch();
        stopwatch.start();
        Gearman gearman = Gearman.createGearman();
        GearmanClient client = gearman.createGearmanClient();

        GearmanServer server = gearman.createGearmanServer(ECHO_HOST, ECHO_PORT);
        client.addServer(server);
        System.out.println(">> server count: " + client.getServerCount());

        for (int i = 0; i < 500; i++) {

            /** ************************************ */
            GearmanJoin<String> join = client.submitJob(ECHO_FUNCTION_NAME, ("hi,zhaoyang" + i).getBytes(), ECHO_FUNCTION_NAME, new GearClient());
            join.join();
            Thread.sleep(20);
        }
        gearman.shutdown();
        stopwatch.stop();
        long nanos = stopwatch.elapsedTime(TimeUnit.SECONDS);
        System.out.println(">>> " + nanos);

    }


    @Override
    public void onEvent(String s, GearmanJobEvent event) {
        switch (event.getEventType()) {
            case GEARMAN_JOB_SUCCESS:
                System.out.println(">>>> " + new String(event.getData()));
                break;
            case GEARMAN_SUBMIT_FAIL:
            case GEARMAN_JOB_FAIL:
                System.out.println(event.getEventType() + ": " + new String(event.getData()));
        }
    }
}
任务执行时间 13s，没有明显变化，扣除10s的线程休眠时间，任务执行时间为3s

client端的日志情况：

12693 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12694 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
>>>> hi,zhaoyang481
12696 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
12716 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12717 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
12720 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
>>>> hi,zhaoyang482
12740 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12741 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
>>>> hi,zhaoyang483
12744 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
12765 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12766 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
12773 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE
>>>> hi,zhaoyang484
12793 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB
12795 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
>>>> hi,zhaoyang485
12812 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : WORK_COMPLETE

Worker端日志输出：

4172152 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : NO_JOB
4172152 [gearman-105] INFO gearman - [127.0.0.1:3333] : OUT : PRE_SLEEP
4172174 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : NOOP
4172174 [gearman-107] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
4172175 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4172175 [gearman-107] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang497
4172176 [gearman-106] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
4172176 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : NO_JOB
4172177 [gearman-105] INFO gearman - [127.0.0.1:3333] : OUT : PRE_SLEEP
4172198 [gearman-106] INFO gearman - [127.0.0.1:3333] : IN : NOOP
4172198 [gearman-105] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
4172199 [gearman-106] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4172199 [gearman-105] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang498
4172199 [gearman-107] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
4172199 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : NO_JOB
4172200 [gearman-105] INFO gearman - [127.0.0.1:3333] : OUT : PRE_SLEEP
4172222 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : NOOP
4172222 [gearman-106] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
4172223 [gearman-105] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
@@@@@ hi,zhaoyang499
4172223 [gearman-107] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE

如果改为 任务后台运行的方式，只需要将

GearmanJoin<String> join = client.submitJob(ECHO_FUNCTION_NAME, ("hi,zhaoyang" + i).getBytes(), ECHO_FUNCTION_NAME, new GearClient()); 改为：

GearmanJoin<String> join = client.submitBackgroundJob(ECHO_FUNCTION_NAME, ("hi,zhaoyang" + i).getBytes(), ECHO_FUNCTION_NAME, new GearClient());

这种实现，没有返回值（注意，这种实现可以移除线程休眠，即Thread.sleep(20)），但是速度很快，半秒钟就完全跑完任务

Client的日志：

587 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
587 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB_BG
588 [gearman-3] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
588 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB_BG
589 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
589 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB_BG
589 [gearman-1] INFO gearman - [127.0.0.1:3333] : IN : JOB_CREATED
590 [main] INFO gearman - [127.0.0.1:3333] : OUT : SUBMIT_JOB_BG
Worker日志：
4633472 [gearman-115] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4633472 [gearman-113] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang493
4633473 [gearman-113] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4633473 [gearman-118] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
@@@@@ hi,zhaoyang4944633473 [gearman-115] INFO gearman - [127.0.0.1:3333] : OUT : GRA

4633474 [gearman-116] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
4633474 [gearman-118] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4633474 [gearman-115] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang495
4633475 [gearman-116] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
4633475 [gearman-115] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4633475 [gearman-113] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang496
4633476 [gearman-117] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE
4633476 [gearman-113] INFO gearman - [127.0.0.1:3333] : IN : JOB_ASSIGN
4633477 [gearman-115] INFO gearman - [127.0.0.1:3333] : OUT : GRAB_JOB
@@@@@ hi,zhaoyang497
4633477 [gearman-117] INFO gearman - [127.0.0.1:3333] : OUT : WORK_COMPLETE

----------- Gearman Server运行截图 ---------------

从图中可以看到，只有执行后端任务执行时，才出现了排队任务（Queued）大于执行任务（Processed），其他情况执行和排队一致，说明客户端如果是同步单线程任务提交，相当于同时只有一个worker节点在工作。如何保证多worker节点不偷懒，需要client在任务提交时，进行并行多线程任务提交方式，不然因为greaman server任务太少，worker总是吃不饱的现象。

转载：http://hi.baidu.com/hivemind/item/17766345fb9aa90cc1161320