GlusterFS ：Addbrick工作流程分析

1、工作流程分析

步骤说明：

1、  Cli接收到用户的输入后，对命令进行解析，判断命令是否合法（命令格式，字符拼写等检查），然后RPC调用本机的glusterd程序的远程函数，glusterd程序开始解析addbrick的命令行，判断里面的hostname所对应机器是否存在；peer内的uuid是否存在；brick所对应的路径是否存在；检查所对应的brick是否已经存在，日志片段如下：

[glusterd-op-sm.c:8544:glusterd_op_set_cli_op] 0-: Returning 0        [glusterd-handler.c:1306:glusterd_handle_add_brick] 0-glusterd: Receiv [glusterd-utils.c:847:glusterd_volinfo_find] 0-: Volume test found    [glusterd-utils.c:855:glusterd_volinfo_find] 0-: Returning 0          [glusterd-utils.c:630:glusterd_brickinfo_new] 0-: Returning 0         [glusterd-utils.c:687:glusterd_brickinfo_from_brick] 0-: Returning 0  [glusterd-utils.c:2755:glusterd_friend_find_by_hostname] 0-glusterd: U [glusterd-utils.c:211:glusterd_is_local_addr] 0-glusterd: sdc0 is loca [glusterd-utils.c:2789:glusterd_hostname_to_uuid] 0-: returning 0     [glusterd-utils.c:642:glusterd_resolve_brick] 0-: Returning 0         [glusterd-utils.c:760:glusterd_volume_brickinfo_get] 0-: Returning -1 [glusterd-utils.c:2927:glusterd_new_brick_validate] 0-: returning 0   [glusterd-utils.c:760:glusterd_volume_brickinfo_get] 0-: Returning -1 [glusterd-utils.c:630:glusterd_brickinfo_new] 0-: Returning 0          [glusterd-utils.c:687:glusterd_brickinfo_from_brick] 0-: Returning 0  [glusterd-utils.c:2707:glusterd_friend_find_by_hostname] 0-glusterd: F [glusterd-utils.c:2789:glusterd_hostname_to_uuid] 0-: returning 0     [glusterd-utils.c:642:glusterd_resolve_brick] 0-: Returning 0         [glusterd-utils.c:2666:glusterd_friend_find_by_uuid] 0-glusterd: Frien [glusterd-utils.c:760:glusterd_volume_brickinfo_get] 0-: Returning -1 [glusterd-utils.c:2927:glusterd_new_brick_validate] 0-: returning 0   [glusterd-utils.c:760:glusterd_volume_brickinfo_get] 0-: Returning -1

2、  对所有机器上的该卷进行加锁，每台机器加锁成功才算加锁成功；每个主机的glusterd进程维护了一个全局变量lock，这是该主机的锁，当加锁的时候，会判断lock.owner变量值，0代表没有加锁，没有加锁的时候，加锁才能够成功。首先是本机加锁，加锁成了，然后再发送信息给其他的主机进行加锁。

glusterd-utils.c:243:glusterd_lock] 0-glusterd: Cluster lock held by f0481376-8cee-4eba-a4c2-4790b5eae856 glusterd-handler.c:420:glusterd_op_txn_begin] 0-glusterd: Acquired local lock glusterd-op-sm.c:8393:glusterd_op_sm_inject_event] 0-glusterd: Enqueuing event: 'GD_OP_EVENT_START_LOCK' glusterd-handler.c:424:glusterd_op_txn_begin] 0-glusterd: Returning 0 glusterd-op-sm.c:8449:glusterd_op_sm] 0-: Dequeued event of type: 'GD_OP_EVENT_START_LOCK' glusterd-rpc-ops.c:1541:glusterd3_1_cluster_lock] 0-glusterd: Returning 0 glusterd-op-sm.c:6462:glusterd_op_ac_send_lock] 0-: Returning with 0 glusterd-utils.c:3182:glusterd_sm_tr_log_transition_add] 0-glusterd: Transitioning from 'Default' to 'Lock sent' due to event 'GD_OP_EVENT_START_LOCK' glusterd-utils.c:3184:glusterd_sm_tr_log_transition_add] 0-: returning 0 glusterd-utils.c:577:glusterd_volume_brickinfos_delete] 0-: Returning 0 glusterd-rpc-ops.c:759:glusterd3_1_cluster_lock_cbk] 0-glusterd: Received ACC from uuid: cb2b0881-831f-4c28-8e4e-e4f1bd01e842

 3、 清除一些brick信息，删除一些时间戳的临时文件；

4、 启动添加的brick进程；

5、  以执行该命令的机器作为第一台机器，然后第一台机器创建卷目录，及其相关的所有的brick文件，并且将内存里面的卷，brick相关信息写入到相应的文件内；第一台机器完成这些操作后，第二台机器重复该操作。。。。。第一台机器收到这些节点信息修改的成功响应后才算成功；

[glusterd-volgen.c:2342:generate_brick_volfiles] 0-: Found a brick - sdc0:/data/test0 [glusterd-volgen.c:1311:server_check_marker_off] 0-: Returning 0 [glusterd-volgen.c:2342:generate_brick_volfiles] 0-: Found a brick - sdc2:/data/test0 [glusterd-volgen.c:1311:server_check_marker_off] 0-: Returning 0 [glusterd-volgen.c:2342:generate_brick_volfiles] 0-: Found a brick - sdc0:/data/test1 [glusterd-volgen.c:1311:server_check_marker_off] 0-: Returning 0 [glusterd-volgen.c:2342:generate_brick_volfiles] 0-: Found a brick - sdc2:/data/test1 [glusterd-volgen.c:1311:server_check_marker_off] 0-: Returning 0 [glusterd-volgen.c:2342:generate_brick_volfiles] 0-: Found a brick - sdc0:/data/test2 [glusterd-volgen.c:1311:server_check_marker_off] 0-: Returning 0 [glusterd-volgen.c:2342:generate_brick_volfiles] 0-: Found a brick - sdc2:/data/test2 [glusterd-volgen.c:1311:server_check_marker_off] 0-: Returning 0 [glusterd-volgen.c:2353:generate_brick_volfiles] 0-: Returning 0 [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: sdc0:/data/test2 [glusterd-utils.c:2755:glusterd_friend_find_by_hostname] 0-glusterd: Unable to find friend: sdc0 [glusterd-utils.c:211:glusterd_is_local_addr] 0-glusterd: sdc0 is local [glusterd-utils.c:2789:glusterd_hostname_to_uuid] 0-: returning 0 [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick [glusterd-utils.c:760:glusterd_volume_brickinfo_get] 0-: Returning 0 [glusterd-utils.c:807:glusterd_volume_brickinfo_get_by_brick] 0-: Returning 0 [glusterd-utils.c:778:glusterd_volume_brickinfo_get_by_brick] 0-: brick: sdc2:/data/test2 [glusterd-utils.c:2707:glusterd_friend_find_by_hostname] 0-glusterd: Friend sdc2 found.. state: 3 [glusterd-utils.c:2789:glusterd_hostname_to_uuid] 0-: returning 0 [glusterd-utils.c:726:glusterd_volume_brickinfo_get] 0-: Found brick [glusterd-utils.c:760:glusterd_volume_brickinfo_get] 0-: Returning 0 [glusterd-utils.c:807:glusterd_volume_brickinfo_get_by_brick] 0-: Returning 0 [glusterd-op-sm.c:1541:glusterd_op_perform_add_bricks] 0-: Returning 0 [glusterd-store.c:608:glusterd_store_create_volume_dir] 0-: Returning with 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:632:glusterd_store_volinfo_write] 0-: Returning 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:1089:glusterd_store_save_value] 0-: returning: 0 [glusterd-store.c:292:glusterd_store_brickinfo_write] 0-: Returning 0

 6、  解锁，每台机器解锁成功才算解锁成功；

[2012-12-14 17:14:43.704757] D [glusterd-op-sm.c:6509:glusterd_op_ac_send_unlock] 0-: Returning with 0 [2012-12-14 17:14:43.704798] D [glusterd-utils.c:3182:glusterd_sm_tr_log_transition_add] 0-glusterd: Transitioning from 'Commit op sent' to 'Unlock sent' due to event 'GD_OP_EVENT_COMMIT_ACC' [2012-12-14 17:14:43.704837] D [glusterd-utils.c:3184:glusterd_sm_tr_log_transition_add] 0-: returning 0 [2012-12-14 17:14:43.705205] I [glusterd-rpc-ops.c:818:glusterd3_1_cluster_unlock_cbk] 0-glusterd: Received ACC from uuid: cb2b0881-831f-4c28-8e4e-e4f1bd01e842