Studying note of GCC-3.4.6 source (71)

4.        Source code parsing

Return from lang_dependent_init, the compiler is almost ready, and enable to parse our source program. At here we can see that, every time to compile source file(s) by command line, the compiler would take above complex initialization. Such compilation, is called translation unit. From above code, can find that the initialization of C++ compiler is much more complicated than C one; and in later, it shows that the processing of C++ also much more difficult than C. For simple source code, C++ compiler no doubt has lower efficiency. But for sophisticated source code, for which C++’s runtime library, the tree form runtime environment, and the rich features are required, C++ compiler may not be inferior.

 

do_compile (continue)

 

4660      compile_file ();

4661

4662      if (flag_unit_at_a_time)

4663      {

4664         rtl_dump_file = cgraph_dump_file;

4665         cgraph_dump_file = NULL;

4666         close_dump_file (DFI_cgraph, NULL, NULL_RTX);

4667      }

4668   }

 

Here compile_file compiles an entire translation unit, and write a file of assembly output and various debugging dumps.

 

1810 static void

1811 compile_file (void)                                                                                   in toplev.c

1812 {

1813   /* Initialize yet another pass.  */

1814

1815   init_final (main_input_filename);

1816   coverage_init (aux_base_name);

1817

1818   timevar_push (TV_PARSE);

1819

1820  /* Call the parser, which parses the entire file (calling

1821     rest_of_compilation for each function).  */

1822   (*lang_hooks.parse_file) (set_yydebug);

 

Routine init_final initializes data used in final pass. Here app_on is set nonzero if have enabled APP processing of our assembler output (this variable used tegother with macro ASM_APP_ON which is a C string constant for text to be output before each asm statement or group of consecutive ones. Normally this is "#APP", which is a comment that has no effect on most assemblers but tells the GNU assembler that it must check the lines that follow for all valid assembler constructs). And variable final_sequence contains the sequence rtx if we are outputting an insn sequence.

 

241  void

242  init_final (const char *filename ATTRIBUTE_UNUSED)                       in final.c

243  {

244    app_on = 0;

245    final_sequence = 0;

246 

247  #ifdef ASSEMBLER_DIALECT

248    dialect_number = ASSEMBLER_DIALECT;

249  #endif

250  }

 

Following, coverage_init does initialization for gcov – the tool of GCC to evaluate code coverage. We just ingore it here. For C++ language, the hook for source file parsing is c_common_parse_file in following.

 

1219 void

1220 c_common_parse_file (int set_yydebug ATTRIBUTE_UNUSED)                   in c-opts.c

1221 {

1222   unsigned file_index;

1223  

1224 #if YYDEBUG != 0

1225   yydebug = set_yydebug;

1226 #else

1227   warning ("YYDEBUG not defined");

1228 #endif

1229

1230   file_index = 0;

1231  

1232   do

1233   {

1234     if (file_index > 0)

1235     {

1236       /* Reset the state of the parser.  */

1237       c_reset_state();

1238

1239       /* Reset cpplib's macros and start a new file.  */

1240       cpp_undef_all (parse_in);

1241       main_input_filename = this_input_filename

1242          = cpp_read_main_file (parse_in, in_fnames[file_index]);

1243       if (this_input_filename == NULL)

1244         break;

1245     }

1246     finish_options ();

1247     if (file_index == 0)

1248       pch_init ();

1249     c_parse_file ();

1250

1251     file_index++;

1252   } while (file_index < num_in_fnames);

1253  

1254   finish_file ();

1255 }

 

Note that in c_common_post_options, the first source file in the command line has been read in and held by main_input_filename. And variable num_in_fnames records the number of source files input via command line. The names of the these files are saved in array in_fnames. Here it can see that source files in the same command line would have their content merged tegother, and forms one big front-end tree.

5.1. Setup macros for the system

Below if cpp_opts->preprocessed is nonzero, means we are handling source files after preprocessing. Otherwise for every input source file, now all command line options have been recorded and but some of them still are unhandled which are kept within deferred_opts. And it is also time to setup macros for the system, which will affect the definitions in system headers.

 

1411 static void

1412 finish_options (void)                                                                                 in c-opts.c

1413 {

1414   if (!cpp_opts->preprocessed)

1415   {

1416     size_t i;

1417

1418     cpp_change_file (parse_in, LC_RENAME, _("<built-in>"));

1419     cpp_init_builtins (parse_in, flag_hosted);

1420     c_cpp_builtins (parse_in);

 

Note that at line 1418, cpp_change_file has following definition. Argument reason of value LC_RENAME means a file name or line number changes for reasons other than new file enter or leave (e.g. a #line directive). Here it means content below forms a included system file “built-in”.

 

973  void

974  cpp_change_file (cpp_reader *pfile, enum lc_reason reason,                          in cppfiles.c

975         const char *new_name)

976  {

977    _cpp_do_file_change (pfile, reason, new_name, 1, 0);

978  }

 

The detail about source file change handling refers to section Do file change.

5.1.1. C++ builtin macros

C++ standard defines some pre-defined macros we can used in programming which are described by builtin_array.

 

289  static const struct builtin builtin_array[] =                                                    in cppinit.c

290  {

291    B("__TIME__",        BT_TIME),

292    B("__DATE__",        BT_DATE),

293    B("__FILE__",         BT_FILE),

294    B("__BASE_FILE__",     BT_BASE_FILE),

295    B("__LINE__",         BT_SPECLINE),

296    B("__INCLUDE_LEVEL__", BT_INCLUDE_LEVEL),

297    /* Keep builtins not used for -traditional-cpp at the end, and

298      update init_builtins() if any more are added.  */

299    B("_Pragma",           BT_PRAGMA),

300    B("__STDC__",        BT_STDC),

301  };

 

Above the rear two are not supported in traditional mode. Below in FOR loop at line 348, function cpp_lookup inserts identifiers of these macros into hashtable ident_hash. As these identifiers stand for builtin objects, the hashnodes should set accordingly. Note that here, we don’t set content for these macros – we don’t know yet.

 

339  void

340  cpp_init_builtins (cpp_reader *pfile, int hosted)                                           in cppinit.c

341  {

342    const struct builtin *b;

343    size_t n = ARRAY_SIZE (builtin_array);

344 

345    if (CPP_OPTION (pfile, traditional))

346      n -= 2;

347 

348    for(b = builtin_array; b < builtin_array + n; b++)

349    {

350      cpp_hashnode *hp = cpp_lookup (pfile, b->name, b->len);

351      hp->type = NT_MACRO;

352      hp->flags |= NODE_BUILTIN | NODE_WARN;

353      hp->value.builtin = b->value;

354    }

355 

356    if (CPP_OPTION (pfile, cplusplus))

357      _cpp_define_builtin (pfile, "__cplusplus 1");

358    else if (CPP_OPTION (pfile, lang) == CLK_ASM)

359      _cpp_define_builtin (pfile, "__ASSEMBLER__ 1");

360    else if (CPP_OPTION (pfile, lang) == CLK_STDC94)

361      _cpp_define_builtin (pfile, "__STDC_VERSION__ 199409L");

362    else if (CPP_OPTION (pfile, c99))

363      _cpp_define_builtin (pfile, "__STDC_VERSION__ 199901L");

364 

365    if (hosted)

366      _cpp_define_builtin (pfile, "__STDC_HOSTED__ 1");

367    else

368      _cpp_define_builtin (pfile, "__STDC_HOSTED__ 0");

369 

370    if (CPP_OPTION (pfile, objc))

371      _cpp_define_builtin (pfile, "__OBJC__ 1");

372  }

 

Then statement at line 357, is just same as declaring: #define __cplusplus 1. For this specific definition, below function is in charge of building the corresponding node of cpp_macro.

 

1832 void

1833 _cpp_define_builtin (cpp_reader *pfile, const char *str)                                in cpplib.c

1834 {

1835   size_t len = strlen (str);

1836   char *buf = alloca (len + 1);

1837   memcpy (buf, str, len);

1838   buf[len] = '/n';

1839   run_directive (pfile, T_DEFINE, buf, len);

1840 }

 

As the name suggests, run_directive runs the directive following the details held in dtable.

 

440  static void

441  run_directive (cpp_reader *pfile, int dir_no, const char *buf, size_t count)     in cpplib.c

442  {

443    cpp_push_buffer (pfile, (const uchar *) buf, count,

444                  /* from_stage3 */ true);

445    /* Disgusting hack.  */

446    if (dir_no == T_PRAGMA)

447      pfile->buffer->file = pfile->buffer->prev->file;

448    start_directive (pfile);

449 

450    /* This is a short-term fix to prevent a leading '#' being

451      interpreted as a directive.  */

452    _cpp_clean_line (pfile);

453 

454    pfile->directive = &dtable[dir_no];

455    if (CPP_OPTION (pfile, traditional))

456      prepare_directive_trad (pfile);

457    pfile->directive->handler (pfile);

458    end_directive (pfile, 1);

459    if (dir_no == T_PRAGMA)

460      pfile->buffer->file = NULL;

461   _cpp_pop_buffer (pfile);

462  }

 

At line 443, cpp_push_buffer links the buf into buffer slot of pfile. Note that at line 447, file field of buffer is of type _cpp_file, which represents the source file. As a result, current buffer is the directive itself, but its file field is 0. As some #pragma directives have effect upon the following content in the source file, the update at line 447 is important.

 

223  static void

224  start_directive (cpp_reader *pfile)                                                               in cpplib.c

225  {

226    /* Setup in-directive state.  */

227    pfile->state.in_directive = 1;

228    pfile->state.save_comments = 0;

229 

230    /* Some handlers need the position of the # for diagnostics.  */

231    pfile->directive_line = pfile->line;

232  }

 

Then start_directive setups related slots of pfile which will affect the behavor of below lex_macro_node. And _cpp_clean_line above, set cur and next_line slots in buffer to the head and end of the definition string. At line 457, handler in fact goes to do_define.

 

502  static void

503  do_define (cpp_reader *pfile)                                                                     in cpplib.c

504  {

505    cpp_hashnode *node = lex_macro_node (pfile);

506 

507    if (node)

508    {

509      /* If we have been requested to expand comments into macros,

510        then re-enable saving of comments.  */

511       pfile->state.save_comments =

512          ! CPP_OPTION (pfile, discard_comments_in_macro_exp);

513 

514      if (_cpp_create_definition (pfile, node))

515        if (pfile->cb.define)

516          pfile->cb.define (pfile, pfile->directive_line, node);

517    }

518  }

 

Function _cpp_lex_token decomposes the content of buffer into tokens (i.e, identifier, string, number etc.), and packs them into cpp_token to return. One token is returned in every invocation.

 

466  static cpp_hashnode *

467  lex_macro_node (cpp_reader *pfile)                                                           in cpplib.c

468  {

469    const cpp_token *token = _cpp_lex_token (pfile);

470 

471    /* The token immediately after #define must be an identifier. That

472      identifier may not be "defined", per C99 6.10.8p4.

473      In C++, it may not be any of the "named operators" either,

474      per C++98 [lex.digraph], [lex.key].

475      Finally, the identifier may not have been poisoned. (In that case

476      the lexer has issued the error message for us.)  */

477 

478    if (token->type == CPP_NAME)

479    {

480      cpp_hashnode *node = token->val.node;

481 

482      if (node == pfile->spec_nodes.n_defined)

483        cpp_error (pfile, CPP_DL_ERROR,

484                 "/"defined/" cannot be used as a macro name");

485      else if (! (node->flags & NODE_POISONED))

486        return node;

487    }

488    else if (token->flags & NAMED_OP)

489      cpp_error (pfile, CPP_DL_ERROR,

490               "/"%s/" cannot be used as a macro name as it is an operator in C++",

491               NODE_NAME (token->val.node));

492    else if (token->type == CPP_EOF)

493      cpp_error (pfile, CPP_DL_ERROR, "no macro name given in #%s directive",

494               pfile->directive->name);

495    else

496      cpp_error (pfile, CPP_DL_ERROR, "macro names must be identifiers");

497 

498    return NULL;

499  }

 

Remember that we can poison an identifier to remove it completely from the program. For an poisoned identifier, it will has flags set with NODE_POISONED. And identifier with flags set by NAMED_OP is named operator in C++. lex_macro_node ensures the identifier is valid for being the name of macro.

The detail of implementing macro definition, can refer to section Create macro definition – ISO mode. The define handle at line 515 is for debugger purpose, we skip it here.

Then function end_directive restores the lexer’s state, and if skip_line is not 0, skips the rest tokens of the line of the directive.

 

235  static void

236  end_directive (cpp_reader *pfile, int skip_line)                                     in cpplib.c

237  {

238    if (CPP_OPTION (pfile, traditional))

239    {

240      /* Revert change of prepare_directive_trad.  */

241      pfile->state.prevent_expansion--;

242 

243      if (pfile->directive != &dtable[T_DEFINE])

244        _cpp_remove_overlay (pfile);

245    }

246    /* We don't skip for an assembler #.  */

247    else if (skip_line)

248   {

249      skip_rest_of_line (pfile);

250      if (!pfile->keep_tokens)

251      {

252        pfile->cur_run = &pfile->base_run;

253        pfile->cur_token = pfile->base_run.base;

254      }

255    }

256 

257    /* Restore state.  */

258    pfile->state.save_comments = ! CPP_OPTION (pfile, discard_comments);

259    pfile->state.in_directive = 0;

260    pfile->state.in_expression = 0;

261    pfile->state.angled_headers = 0;

262    pfile->directive = 0;

263  }