嵌入式 wget 参数解析篇

1 整体概括：

前提说明:

本篇wget分析仅仅是参数解析内容，不包括wget的递归和非递归下载，后面文章会陆续进行分析。本次主要分析参数为tries(t) timeout(T) no-clobber quiet(q) recursive(r) help(h)version(V) append-output(a) execute(e) no(n) clobber, 其中括号里面的为wget短选项，括号前面的为长选项。

 

在wget运行下载文件或页面时，用户可以通过参数来改变wget的行为，比如想查看wget的调试和http数据包可以使用 wget --debugwww.baidu.com/index.html。

 

我们这次分析下载url以baidu 搜索页面(http://www.baidu.com/index.html)为样本，进行分析不同类型的参数，以达到抛砖引玉的目的。wget支持长选项和短选项，比如输出调试信息短选项为-d长选项为—debug

 

wget有全局的struct options opt；保存着wget用户参数设置值，来修改wget行为。本篇主要讲解用户输入参数如何转化为 opt的成员。

 

wget分析的版本为1.13，gcc版本为3.4.5，linux内核版本2.6.9_5-9-0-0

2 详细代码解析:

2.1数据结构

wget 对于配置转化，设置struct options opt 有两张表和长短选项数组

命令行表:

struct cmdline_option option_data

此表保存着wget支持的长短选项和长短选项属性

命令转化设置opt表:

commands

此表用于设置根据参数来设置opt成员。

长选项:

struct option long_options[2*countof(option_data) + 1]

短选项:

struct char short_options[128]

2.2参数解析流程

Main 首先根据不同平台来设置使用时间函数，blog里有monotonic time和wall time讲解，这里就不分析。

2.2.1 defaults();

然后调用defaults函数，该函数主要是给全局opt设置默认值(因为代码太长，给出部分代码)。

 //#######################src/init.c
 /* Reset the variables to default values.  */
   void
   defaults (void)
   {
     char *tmp;

     /* Most of the default values are 0 (and 0.0, NULL, and false).
        Just reset everything, and fill in the non-zero values.  Note
        that initializing pointers to NULL this way is technically
        illegal, but porting Wget to a machine where NULL is not all-zero
        bit pattern will be the least of the implementors' worries.  */
     xzero (opt);

     opt.cookies = true;
     opt.verbose = -1;
     opt.ntry = 20;
     opt.reclevel = 5;
     opt.add_hostdir = true;
     opt.netrc = true;
     opt.ftp_glob = true;

2.2.2 init_switches()

函数很简单，追加一些ch注释

 static void
   init_switches (void)
   {
     //p指向短选项数组
     char *p = short_options;
     size_t i, o = 0;
     //遍历所有选项
     for (i = 0; i < countof (option_data); i++)
       {
         struct cmdline_option *opt = &option_data[i];
         struct option *longopt;

         //如果这个选项数据没有长选项，直接跳过
         if (!opt->long_name)
           /* The option is disabled. */
           continue;

         //longopt指向长选项一个依次节点
         longopt = &long_options[o++];
         //长选项name指向opt的long_name
         longopt->name = opt->long_name;
         //长选项val执行opt的数组索引，用于根据长选项查找opt
         longopt->val = i;
         if (opt->short_name)
          {
             //如果存在短选项，把opt short_name保存在short_options中
             *p++ = opt->short_name;
             //用optmap保存short_name的value 来索引长选项数组
             optmap[opt->short_name - 32] = longopt - long_options;
           }
         switch (opt->type)
           {
           case OPT_VALUE:
            //参数需要值
            longopt->has_arg = required_argument;
           //如果参数需要设置值，并且短选项存在，就需要字符":"
             if (opt->short_name)
               *p++ = ':';
             break;
           case OPT_BOOLEAN:
             /* 如果是bool类型(开关类型参数) 需要支持--option=off and --no-option .look the note of the blow*/
             /* Specify an optional argument for long options, so that
                --option=off works the same as --no-option, for
                compatibility with pre-1.10 Wget.  However, don't specify
                optional arguments short-option booleans because they
                prevent combining of short options.  */
             longopt->has_arg = optional_argument;
             /* For Boolean options, add the "--no-FOO" variant, which is
                identical to "--foo", except it has opposite meaning and
                it doesn't allow an argument.  */
             longopt = &long_options[o++];
             longopt->name = no_prefix (opt->long_name);
             longopt->has_arg = no_argument;
             /* Mask the value so we'll be able to recognize that we're
                dealing with the false value.  */
             //索引加一个负数符号
             longopt->val = i | BOOLEAN_NEG_MARKER;
             break;
           default:
             //others 根据情况设置不同的值
             assert (opt->argtype != -1);
             longopt->has_arg = opt->argtype;
             if (opt->short_name)
               {
                 if (longopt->has_arg == required_argument)
                   *p++ = ':';
                 /* Don't handle optional_argument */
               }
           }
       }
     /* Terminate short_options. */
     *p = '\0';
     /* No need for xzero(long_options[o]) because its storage is static
        and it will be zeroed by default.  */
     assert (o <= countof (long_options));
   }

举例分析(长选项为append-output ，短(a)):

用gdb跟踪下long_options和short_options

截取long_options一部分:

name(append-output) has_arg(1) val(2)

val==2 表示该长选项属性在option_data的索引

其中字符’a’ ascii值为97 那么这个在opt_map中索引为97-32=65

Such 

也就可以通过短选项找个长选项索引，然后这个长选项val就是option_data的数组索引。

2.2.1 main set opt

 while ((ret = getopt_long (argc, argv,
                               short_options, long_options, &longindex)) != -1)
      {
        int val;
        struct cmdline_option *opt;

        /* If LONGINDEX is unchanged, it means RET is referring a short
           option.  */
        if (longindex == -1)
          {
            if (ret == '?')
              {
                print_usage (0);
                printf ("\n");
                printf (_("Try `%s --help' for more options.\n"), exec_name);
                exit (2);
              }
            /* Find the short option character in the mapping.  */
            longindex = optmap[ret - 32];
          }
        val = long_options[longindex].val;

        /* Use the retrieved value to locate the option in the
           option_data array, and to see if we're dealing with the
           negated "--no-FOO" variant of the boolean option "--foo".  */
        opt = &option_data[val & ~BOOLEAN_NEG_MARKER];

我截取了main处理argc argv部分代码。

调用过api getopt_long, 如果longindex==-1那么用户输入的是短选项，通过optmap来确定此短选项在长选项数组索引optmap[ret-32], 然后根据长选项的val找到在opt_data的此选项位置,如果用户输入的是长选项，就直接使用val。

val = long_options[longindex].val;

获取此选项opt_data

opt = &option_data[val &~BOOLEAN_NEG_MARKER];

 

找到了参数在opt_data的位置，然后下面就开始设置全局opt

根据参数类型分析以下参数:

 OPT_VALUE                   tries(t) timeout(T)
 OPT_BOOLEAN             no-clobber quiet(q) recursive(r)
 OPT_FUNCALL                 help(h) version(V)
 OPT__APPEND_OUTPUT          append-output(a)
 OPT_EXECUTE                 execute(e)
 OPT_NO                      no(n)
 OPT__PARENT|OPT__CLOBBER    clobber

代码段：

 switch (opt->type)
    {
    case OPT_VALUE:
      setoptval (opt->data, optarg, opt->long_name);
      break;
    case OPT_BOOLEAN:
      if (optarg)
        /* The user has specified a value -- use it. */
        setoptval (opt->data, optarg, opt->long_name);
      else
        {
          /* NEG is true for `--no-FOO' style boolean options. */
          bool neg = !!(val & BOOLEAN_NEG_MARKER);
          setoptval (opt->data, neg ? "0" : "1", opt->long_name);
        }
      break;
    case OPT_FUNCALL:
      {
        void (*func) (void) = (void (*) (void)) opt->data;
        func ();
      }
      break;
    case OPT__APPEND_OUTPUT:
      setoptval ("logfile", optarg, opt->long_name);
      append_to_log = true;
      break;
    case OPT__EXECUTE:
      run_command (optarg);
      break;
    case OPT__NO:
      {
        /* We support real --no-FOO flags now, but keep these
           short options for convenience and backward
           compatibility.  */
        char *p;
        for (p = optarg; p && *p; p++)
          switch (*p)
            {
            case 'v':
              setoptval ("verbose", "0", opt->long_name);
              break;
            case 'H':
              setoptval ("addhostdir", "0", opt->long_name);
              break;
            case 'd':
              setoptval ("dirstruct", "0", opt->long_name);
              break;
            case 'c':
              setoptval ("noclobber", "1", opt->long_name);
              break;
            case 'p':
              setoptval ("noparent", "1", opt->long_name);
              break;
            default:
              fprintf (stderr, _("%s: illegal option -- `-n%c'\n"),
                       exec_name, *p);
              print_usage (1);
              fprintf (stderr, "\n");
              fprintf (stderr, _("Try `%s --help' for more options.\n"),
                       exec_name);
              exit (1);
            }
        break;
      }
    case OPT__PARENT:
    case OPT__CLOBBER:
    case OPT__CLOBBER:
      {
        /* The wgetrc commands are named noparent and noclobber,
           so we must revert the meaning of the cmdline options
           before passing the value to setoptval.  */
        bool flag = true;
        if (optarg)
          flag = (*optarg == '1' || c_tolower (*optarg) == 'y'
                  || (c_tolower (optarg[0]) == 'o'
                      && c_tolower (optarg[1]) == 'n'));
        setoptval (opt->type == OPT__PARENT ? "noparent" : "noclobber",
                   flag ? "0" : "1", opt->long_name);
        break;
      }
    case OPT__DONT_REMOVE_LISTING:
      setoptval ("removelisting", "0", opt->long_name);
      break;
    }

  longindex = -1;

参数类型OPT_VALUE(t,T)

Setoptval(opt->data, optarg,opt->long_name)

         ->setval_internal(command_by_name(opt->data),“--“+opt->long_name, optarg)

其中command_by_name(opt->data)是通过二分查找，找到data在commands中的索引位置

Code:

 static int
 command_by_name (const char *cmdname)
 {
   /* Use binary search for speed.  Wget has ~100 commands, which
      guarantees a worst case performance of 7 string comparisons.  */
   int lo = 0, hi = countof (commands) - 1;

   while (lo <= hi)
     {
       int mid = (lo + hi) >> 1;
       int cmp = strcasecmp (cmdname, commands[mid].name);
       if (cmp < 0)
         hi = mid - 1;
       else if (cmp > 0)
         lo = mid + 1;
       else
         return mid;
     }
   return -1;
 }

set_internal(comind, “--“+opt->long_name,optarg)

         ->commands[comind].action (“--“+opt->long_name, optarg,commands[comind].place);

比如tries commands信息如下

{ "tries",            &opt.ntry,              cmd_number_inf },

调用cmd_num_inf(“—tries”, optarg, opt.ntry)

函数设置opt.ntry = strtoul(optarg, 10,. NULL)

参数类型OPT_BOOLEAN

和OPT_BOOLEAN大同小异，此处略过。

参数类型OPT_FUNCALL

-h 和 –v

调用opt->data

如果用户输入参数为-h或者-v就会调用print_help or print_version，这里就略过了。

参数类型OPT__APPEND_OUTPUT

setoptval ("logfile", optarg,opt->long_name);//和OPT_VALUE相似，略过。

参数类型OPT__EXECUTE

参数-e

Run_command(optarg)

其中optarg 格式为key=value，此函数解析出key和value，比如append-output=logfile.txt

就会调用set_internal(comind, com, val)来设置opt

参数类型OPT__NO、OPT__PARENT、OPT__CLOBBER、OPT__DONT_REMOVE_LISTING都是大同小异，这里就略过了。