coco/r笔记-扫描器的字符规则定义

扫描器的字符规则定义

Scanner Specification
A scanner has to read source text, skip meaningless characters, recognize tokens and
pass them to the parser. This is described in a scanner specification, which consists of
five optional parts:
ScannerSpecification =
["IGNORECASE"]
["CHARACTERS" {SetDecl}]
["TOKENS" {TokenDecl}]
["PRAGMAS" {PragmaDecl}]
{CommentDecl}
{WhiteSpaceDecl}.
2.3.1 Character sets
This section allows the user to declare character sets such as letters or digits. Their
names can then be used in the other sections of the scanner specification. Coco/R
supports the Unicode character set (UTF-8-encoded).
SetDecl = ident '=' Set '.'.
Set = BasicSet {('+'|'-') BasicSet}.
BasicSet = string | ident | char [".." char] | "ANY".
SetDecl associates a name with a character set. Basic character sets are denoted as:
string a set consisting of all the characters in the string
ident a previously declared character set with this name
char a set containing the character char
char1..char2 the set of all characters from char1 to char2
ANY the set of all characters in the range 0 .. 65535
Character sets may be formed from basic sets using the operators
+ set union
- set difference
Examples
digit = "0123456789". /* the set of all digits */
hexDigit = digit + "ABCDEF". /* the set of all hexadecimal digits */
letter = 'A' .. 'Z'. /* the set of all upper case letters */
eol = '\r'. /* the end-of-line character */
noDigit = ANY - digit. /* any character that is not a digit */