hadoop 2.6 源码 解读之上传文件（-put）内部实现之FsShell init()篇

hdfs 命令行 -put 上传文件操作，通过shell脚本最终调用 FsShell 的main入口方法，实现相应的逻辑功能

为便于debug调试，编写相应的测试程序模拟 -put 操作，示例代码如下

public class TestShell {

    public static void main(String argv[]) throws Exception {
        System.setProperty("hadoop.home.dir", "D:\\aws\\hadoop-2.6.0");
        System.setProperty("HADOOP_USER_NAME", "root");



        FsShell shell = new FsShell();
        Configuration conf = new Configuration();
        conf.setQuietMode(false);
        shell.setConf(conf);

        String[] args = {"-put","sdsd"};


        int res;
        try {
            res = ToolRunner.run(shell, args);
        } finally {
            shell.close();
        }
        System.exit(res);
    }

}

这一段示例代码就是 将FsShell 类的main方法拿过来修改的
跳进ToolRunner.run()

 public static int run(Tool tool, String[] args) 
    throws Exception{
    return run(tool.getConf(), tool, args);
  }

再跳

  public static int run(Configuration conf, Tool tool, String[] args) 
    throws Exception{
    if(conf == null) {
      conf = new Configuration();
    }
    GenericOptionsParser parser = new GenericOptionsParser(conf, args);
    //set the configuration back, so that Tool can configure itself
    tool.setConf(conf);

    //get the args w/o generic hadoop args
    String[] toolArgs = parser.getRemainingArgs();
    return tool.run(toolArgs);
  }

tool.run(toolArgs)，这行代码最后又跳回了 FsShell类的run

@Override
  public int run(String argv[]) throws Exception {
    // initialize FsShell
    init();

    int exitCode = -1;
    if (argv.length < 1) {
      printUsage(System.err);
    } else {
      String cmd = argv[0];
      Command instance = null;
      try {
        instance = commandFactory.getInstance(cmd);
        if (instance == null) {
          throw new UnknownCommandException();
        }
        exitCode = instance.run(Arrays.copyOfRange(argv, 1, argv.length));
      } catch (IllegalArgumentException e) {
        displayError(cmd, e.getLocalizedMessage());
        if (instance != null) {
          printInstanceUsage(System.err, instance);
        }
      } catch (Exception e) {
        // instance.run catches IOE, so something is REALLY wrong if here
        LOG.debug("Error", e);
        displayError(cmd, "Fatal internal error");
        e.printStackTrace(System.err);
      }
    }
    return exitCode;
  }

非常关键的init方法，这里面很啰嗦，初始化操作。主要是建立 相应命令行参数和相应处理类示例的映射关系。读者可以关注下

 protected void init() throws IOException {
    getConf().setQuietMode(true);
    if (commandFactory == null) {
      commandFactory = new CommandFactory(getConf());
      commandFactory.addObject(new Help(), "-help");
      commandFactory.addObject(new Usage(), "-usage");
      registerCommands(commandFactory);
    }
  }

最关键的是registerCommands(commandFactory)

 protected void registerCommands(CommandFactory factory) {
    // TODO: DFSAdmin subclasses FsShell so need to protect the command
    // registration.  This class should morph into a base class for
    // commands, and then this method can be abstract
    if (this.getClass().equals(FsShell.class)) {
      factory.registerCommands(FsCommand.class);
    }
  }

跳进factory.registerCommands()

 public void registerCommands(Class registrarClass) {
    try {
      registrarClass.getMethod(
          "registerCommands", CommandFactory.class
      ).invoke(null, this);
    } catch (Exception e) {
      throw new RuntimeException(StringUtils.stringifyException(e));
    }
  }

分析下CommandFactory 方法registerCommands的设计思路，

1. 获取传参 registrarClass 的 registerCommands 方法，这个方法 和 CommandFactory的 registerCommands() 方法同名，所以源码阅读过程中容易混淆视听，要小心。
2. 通过反射的方式获得参数类示例的registerCommands 方法后，直接使用invoke调用，调用参数继续传入CommandFactory 实例factory。（工程设计上很巧妙，但是阅读起来很累）

接下来方法调用至 FsCommand.registerCommands

  public static void registerCommands(CommandFactory factory) {
    factory.registerCommands(AclCommands.class);
    factory.registerCommands(CopyCommands.class);
    factory.registerCommands(Count.class);
    factory.registerCommands(Delete.class);
    factory.registerCommands(Display.class);
    factory.registerCommands(FsShellPermissions.class);
    factory.registerCommands(FsUsage.class);
    factory.registerCommands(Ls.class);
    factory.registerCommands(Mkdir.class);
    factory.registerCommands(MoveCommands.class);
    factory.registerCommands(SetReplication.class);
    factory.registerCommands(Stat.class);
    factory.registerCommands(Tail.class);
    factory.registerCommands(Test.class);
    factory.registerCommands(Touch.class);
    factory.registerCommands(SnapshotCommands.class);
    factory.registerCommands(XAttrCommands.class);
  }

有点循环的感觉了，factory.registerCommands，继续执行FsCommand子类的registerCommands方法

挑CopyCommands进去看下
这里才触及到 实例和命令行参数的映射关系

  public static void registerCommands(CommandFactory factory) {
    factory.addClass(Merge.class, "-getmerge");
    factory.addClass(Cp.class, "-cp");
    factory.addClass(CopyFromLocal.class, "-copyFromLocal");
    factory.addClass(CopyToLocal.class, "-copyToLocal");
    factory.addClass(Get.class, "-get");
    factory.addClass(Put.class, "-put");
    factory.addClass(AppendToFile.class, "-appendToFile");
  }

总结

通过反射的方式层层注册 实例 和 命令行参数的映射关系，建立一棵实例注册树，便于工程化，以及工程扩展，值得大家积累。