Studying note of GCC-3.4.6 source (81)

5.6.2. Create parser context

After creating the main lexer, it is time to create parser body. First is its context

 

cp_parser_new (continue)

 

2240   parser = ggc_alloc_cleared (sizeof (cp_parser));

2241   parser->lexer = lexer;

2242   parser->context = cp_parser_context_new (NULL);

2243

2244   /* For now, we always accept GNU extensions.  */

2245   parser->allow_gnu_extensions_p = 1;

2246

2247  /* The `>' token is a greater-than operator, not the end of a

2248     template-id.  */

2249   parser->greater_than_is_operator_p = true;

2250

2251   parser->default_arg_ok_p = true;

2252  

2253  /* We are not parsing a constant-expression.  */

2254   parser->integral_constant_expression_p = false;

2255   parser->allow_non_integral_constant_expression_p = false;

2256   parser->non_integral_constant_expression_p = false;

2257

2258  /* We are not parsing offsetof.  */

2259   parser->in_offsetof_p = false;

2260

2261   /* Local variable names are not forbidden.  */

2262   parser->local_variables_forbidden_p = false;

2263

2264   /* We are not processing an `extern "C"' declaration.  */

2265   parser->in_unbraced_linkage_specification_p = false;

2266

2267  /* We are not processing a declarator.  */

2268   parser->in_declarator_p = false;

2269

2270   /* We are not processing a template-argument-list.  */

2271   parser->in_template_argument_list_p = false;

2272

2273   /* We are not in an iteration statement.  */

2274   parser->in_iteration_statement_p = false;

2275

2276  /* We are not in a switch statement.  */

2277   parser->in_switch_statement_p = false;

2278

2279   /* We are not parsing a type-id inside an expression.  */

2280   parser->in_type_id_in_expr_p = false;

2281

2282   /* The unparsed function queue is empty.  */

2283   parser->unparsed_functions_queues = build_tree_list (NULL_TREE, NULL_TREE);

2284

2285  /* There are no classes being defined.  */

2286   parser->num_classes_being_defined = 0;

2287

2288   /* No template parameters apply.  */

2289   parser->num_template_parameter_lists = 0;

2290

2291   return parser;

2292 }

 

This context has type of below.

 

1107 typedef struct cp_parser_context GTY (())                                                    in parser.c

1108 {

1109   /* If this is a tentative parsing context, the status of the

1110     tentative parse.  */

1111   enum cp_parser_status_kind status;

1112  /* If non-NULL, we have just seen a `x->' or `x.' expression. Names

1113     that are looked up in this context must be looked up both in the

1114     scope given by OBJECT_TYPE (the type of `x' or `*x') and also in

1115     the context of the containing expression.  */

1116   tree object_type;

1117   /* The next parsing context in the stack.  */

1118   struct cp_parser_context *next;

1119 } cp_parser_context;

 

All active contexts should form a stack, which is guaranteed by cp_parser_context_new.

 

1137 static cp_parser_context *

1138 cp_parser_context_new (cp_parser_context* next)                                        in parser.c

1139 {

1140   cp_parser_context *context;

1141

1142  /* Allocate the storage.  */

1143   if (cp_parser_context_free_list != NULL)

1144   {

1145     /* Pull the first entry from the free list.  */

1146     context = cp_parser_context_free_list;

1147     cp_parser_context_free_list = context->next;

1148     memset (context, 0, sizeof (*context));

1149   }

1150   else

1151     context = ggc_alloc_cleared (sizeof (cp_parser_context));

1152  /* No errors have occurred yet in this context.  */

1153   context->status = CP_PARSER_STATUS_KIND_NO_ERROR;

1154   /* If this is not the bottomost context, copy information that we

1155     need from the previous context.  */

1156   if (next)

1157   {

1158     /* If, in the NEXT context, we are parsing an `x->' or `x.'

1159       expression, then we are parsing one in this context, too.  */

1160     context->object_type = next->object_type;

1161     /* Thread the stack.  */

1162     context->next = next;

1163   }

1164

1165   return context;

1166 }

5.7. Prepare defer access list

Members of “class” or “struct” or “union” in C++ have access control applied. These access controls should be executed at appropriate point during parsing, even in some code fragments inapplicable to the access control. So the front-end uses stack of deferred_access_stack to controls the access controls. Within this stack, every node is of kind deferred_access, and enumerator deferring_kind indicates the kind of action to be taken for the access control. The new node is added by push_deferring_access_checks, and the removing routine is pop_deferring_access_checks.

 

3077 typedef enum deferring_kind {                                                                    in cp-tree.h

3078   dk_no_deferred = 0, /* Check access immediately */

3079   dk_deferred = 1,    /* Deferred check */

3080   dk_no_check = 2     /* No access check */

3081 } deferring_kind;

 

Hinted by c_parse_file, when begins parsing a new source file, it is also a new scope for access control checking, a new node will be inserted to indicate control at this level. If flag_access_control is nonzero (its default value is 1 for C/C++, and can be overwritten by option –faccess-control), argument deferring will be dk_no_deferred; otherwise is dk_no_check.

 

139  void

140  push_deferring_access_checks (deferring_kind deferring)                       in semantics.c

141  {

142    deferred_access *d;

143 

144    /* For context like template instantiation, access checking

145      disabling applies to all nested context.  */

146    if (deferred_access_stack

147        && deferred_access_stack->deferring_access_checks_kind == dk_no_check)

148      deferring = dk_no_check;

149 

150    /* Recycle previously used free store if available.  */

151    if (deferred_access_free_list)

152    {

153      d = deferred_access_free_list;

154      deferred_access_free_list = d->next;

155    }

156    else

157      d = ggc_alloc (sizeof (deferred_access));

158 

159    d->next = deferred_access_stack;

160    d->deferred_access_checks = NULL_TREE;

161    d->deferring_access_checks_kind = deferring;

162    deferred_access_stack = d;

163  }

 

In struct deferred_access field deferred_access_checks records the deferred access checking at this level (when field deferring_access_checks_kind is dk_deferred), which means deferring the access checking till explicitly required later, while field next chains deferred access checkings at other levels.

 

3119 typedef struct deferred_access GTY(())                                                               in cp-tree.h

3120 {

3121   /* A TREE_LIST representing name-lookups for which we have deferred

3122     checking access controls. We cannot check the accessibility of

3123     names used in a decl-specifier-seq until we know what is being

3124     declared because code like:

3125

3126        class A {

3127          class B {};

3128          B* f();

3129        }

3130

3131        A::B* A::f() { return 0; }

3132

3133     is valid, even though `A::B' is not generally accessible. 

3134

3135     The TREE_PURPOSE of each node is the scope used to qualify the

3136     name being looked up; the TREE_VALUE is the DECL to which the

3137     name was resolved.  */

3138   tree deferred_access_checks;

3139   /* The current mode of access checks.  */

3140   enum deferring_kind deferring_access_checks_kind;

3141   /* The next deferred access data in stack or linked-list.  */

3142   struct deferred_access *next;

3143 } deferred_access;

5.8. Pass token to parser

In previous we have seen that cp_lexer_get_preprocessor_token reads in preprocessed tokens, and converts token from type cpp_token to cp_token. However, the buffer for tokens of cp_token type is maintained by lexer, so in terms of lexer, its token readin function is cp_lexer_read_token.

 

474  static cp_token *

475  cp_lexer_read_token (cp_lexer * lexer)                                                       in parser.c

476  {

477    cp_token *token;

478 

479    /* Make sure there is room in the buffer.  */

480    cp_lexer_maybe_grow_buffer (lexer);

481 

482    /* If there weren't any tokens, then this one will be the first.  */

483    if (!lexer->first_token)

484      lexer->first_token = lexer->last_token;

485    /* Similarly, if there were no available tokens, there is one now.  */

486    if (!lexer->next_token)

487      lexer->next_token = lexer->last_token;

488 

489    /* Figure out where we're going to store the new token.  */

490    token = lexer->last_token;

491 

492    /* Get a new token from the preprocessor.  */

493    cp_lexer_get_preprocessor_token (lexer, token);

494 

495    /* Increment LAST_TOKEN.  */

496    lexer->last_token = cp_lexer_next_token (lexer, token);

497 

498    /* Strings should have type `const char []'. Right now, we will

499      have an ARRAY_TYPE that is constant rather than an array of

500      constant elements.

501      FIXME: Make fix_string_type get this right in the first place.  */

502    if ((token->type == CPP_STRING || token->type == CPP_WSTRING)

503        && flag_const_strings)

504    {

505      tree type;

506 

507      /* Get the current type. It will be an ARRAY_TYPE.  */

508      type = TREE_TYPE (token->value);

509      /* Use build_cplus_array_type to rebuild the array, thereby

510        getting the right type.  */

511       type = build_cplus_array_type (TREE_TYPE (type), TYPE_DOMAIN (type));

512      /* Reset the type of the token.  */

513      TREE_TYPE (token->value) = type;

514    }

515 

516    return token;

517  }

 

To efficiently use the resource, the buffer slot of cpp_lexer is a circular buffer, with last_token refers to the address just following the last available token, first_token refers to the first cached token, and next_token to the first unpeeked token.

Originally, the size of this circular buffer is of size 5 (CP_TOKEN_BUFFER_SIZE), this buffer is very small compared with the complex statement can be written with C++. To prevent running out of buffer, before reading in token from preprocessor, cp_lexer_maybe_grow_buffer is invoked to enlarge buffer if necessary.

 

521  static void

522  cp_lexer_maybe_grow_buffer (cp_lexer * lexer)                                          in parser.c

523  {

524    /* If the buffer is full, enlarge it.  */

525    if (lexer->last_token == lexer->first_token)

526    {

527      cp_token *new_buffer;

528      cp_token *old_buffer;

529      cp_token *new_first_token;

530      ptrdiff_t buffer_length;

531      size_t num_tokens_to_copy;

532 

533      /* Remember the current buffer pointer. It will become invalid,

534        but we will need to do pointer arithmetic involving this

535        value.  */

536      old_buffer = lexer->buffer;

537      /* Compute the current buffer size.  */

538      buffer_length = lexer->buffer_end - lexer->buffer;

539      /* Allocate a buffer twice as big.  */

540      new_buffer = ggc_realloc (lexer->buffer,

541                       2 * buffer_length * sizeof (cp_token));

542       

543      /* Because the buffer is circular, logically consecutive tokens

544        are not necessarily placed consecutively in memory.

545        Therefore, we must keep move the tokens that were before

546        FIRST_TOKEN to the second half of the newly allocated

547        buffer.  */

548      num_tokens_to_copy = (lexer->first_token - old_buffer);

549      memcpy (new_buffer + buffer_length,

550              new_buffer,

551              num_tokens_to_copy * sizeof (cp_token));

552      /* Clear the rest of the buffer. We never look at this storage,

553        but the garbage collector may.  */

554      memset (new_buffer + buffer_length + num_tokens_to_copy, 0,

555             (buffer_length - num_tokens_to_copy) * sizeof (cp_token));

556 

557      /* Now recompute all of the buffer pointers.  */

558      new_first_token

559         = new_buffer + (lexer->first_token - old_buffer);

560      if (lexer->next_token != NULL)

561      {

562        ptrdiff_t next_token_delta;

563 

564        if (lexer->next_token > lexer->first_token)

565          next_token_delta = lexer->next_token - lexer->first_token;

566        else

567          next_token_delta =

568              buffer_length - (lexer->first_token - lexer->next_token);

569        lexer->next_token = new_first_token + next_token_delta;

570      }

571      lexer->last_token = new_first_token + buffer_length;

572      lexer->buffer = new_buffer;

573      lexer->buffer_end = new_buffer + buffer_length * 2;

574      lexer->first_token = new_first_token;

575    }

576  }

 

Following figure demonstrates the details of enlarging buffer for lexer, it needs clear the memory after last token in (b) to tell garbage collector it is in used.

Figure 42: enlarging buffer for lexer

When the buffer contains the resource and new token read in, at line 496, cp_lexer_next_token advances last_token to position next to the token read in.

 

418  static inline cp_token *

419  cp_lexer_next_token (cp_lexer* lexer, cp_token* token)                               in parser.c

420  {

421    token++;

422    if (token == lexer->buffer_end)

423      token = lexer->buffer;

424    return token;

425  }

 

Above at line 503 in cp_lexer_read_token, flag_const_strings if nonzero, means give string constants the type `const char *', as mandated by the standard.