分割字符串：strtok()函数

函数strtok将字符串分解为一系列标记(token)标记就是一系列用分隔符(delimiting chracter，通常是空格或标点符号)分开的字符。例如，在一行文本中，每个单词可以作为标记，空格是分隔符。
    需要多次调用strtok才能将字符串分解为标记(假设字符串中包含多个标记)。第一次调用strtok包含两个参数，即要标记化的字符串和包含用来分隔标记的字符的字符串(即分隔符)：在图5．33的例子中，下列语句：
    tokenPtr = Strtok(string, " ")；
将tokenPtr赋给string中第一个标记的指针。strtok的第二个参数””表示string中的标记用空格分开。
函数strtok搜索string中不是分隔符(空格)的第一个字符，这是第一个标记的开头。然后函数寻找字符串中的下一个分隔符，将其换成null(， w，)字符,这是当前标记的终点。函数strtok保存string中标记后面的下一个字符的指针，并返回当前标记的指针。
    后面再调用strtok时，第一个参数为NULL，继续将string标记化。NULL参数表示调用strtok继续从string中上次调用 strtok时保存的位置开始标记化。如果调用strtok时已经没有标记，则strtok返回NULL。图5．33的程序用strtok将字符串” This is sentence with 7 tokens”标记化。分别打印每个标记。注意strtok修改输入字符串，因此，如果调用strtok之后还要在程序中使用这个字符串，则应复制这个字 符串。

    常见编程错误5．23
    没有认识到strtok修改正在标记化的字符串，调用sstrtok后还在程序中使用这个字符串(以为还是原字符串) 函数strlen取一个字符串作为参数，并返回字符串中的字符个数，长度中不包括null终止符。
    图5.34的程序演示了函数strlen。

1 // Fig. 5.33:fig05 33.cpp
2 // Using strtok
#include #include int main() { char string[] = "This is a sentence with 7 tokens"; char *tokenPtr; cout << "The string to be tokenized is:/n/n" << string << "/n/nThe tokens are:/n"; tokenPtr = strtok( string, " " ); while ( tokenPtr != NULL ) { cout << tokenPtr << '/n'; tokenPtr = strtok( NULL, " " ); } return 0; }

输出结果：
The string to be tokenized is:
This is a sentence with 7 tokens

The tokens are:
This
is
a
sentence
7
tokens

strtok函数源码

/*** *strtok.c - tokenize a string with given delimiters * * Copyright (c) 1989-1997, Microsoft Corporation. All rights reserved. * *Purpose: * defines strtok() - breaks string into series of token * via repeated calls. * *******************************************************************************/ #include #include #ifdef _MT #include #endif /* _MT */ /*** *char *strtok(string, control) - tokenize string with delimiter in control * *Purpose: * strtok considers the string to consist of a sequence of zero or more * text tokens separated by spans of one or more control chars. the first * call, with string specified, returns a pointer to the first char of the * first token, and will write a null char into string immediately * following the returned token. subsequent calls with zero for the first * argument (string) will work thru the string until no tokens remain. the * control string may be different from call to call. when no tokens remain * in string a NULL pointer is returned. remember the control chars with a * bit map, one bit per ascii char. the null char is always a control char. * *Entry: * char *string - string to tokenize, or NULL to get next token * char *control - string of characters to use as delimiters * *Exit: * returns pointer to first token in string, or if string * was NULL, to next token * returns NULL when no more tokens remain. * *Uses: * *Exceptions: * *******************************************************************************/ char * __cdecl strtok ( char * string, const char * control ) { unsigned char *str; const unsigned char *ctrl = control; unsigned char map[32]; int count; #ifdef _MT _ptiddata ptd = _getptd(); #else /* _MT */ static char *nextoken; #endif /* _MT */ /* Clear control map */ for (count = 0; count < 32; count++) map[count] = 0; /* Set bits in delimiter table */ do { map[*ctrl >> 3] |= (1 << (*ctrl & 7)); } while (*ctrl++); /* Initialize str. If string is NULL, set str to the saved * pointer (i.e., continue breaking tokens out of the string * from the last strtok call) */ if (string) str = string; else #ifdef _MT str = ptd->_token; #else /* _MT */ str = nextoken; #endif /* _MT */ /* Find beginning of token (skip over leading delimiters). Note that * there is no token iff this loop sets str to point to the terminal * null (*str == '/0') */ while ( (map[*str >> 3] & (1 << (*str & 7))) && *str ) str++; string = str; /* Find the end of the token. If it is not the end of the string, * put a null there. */ for ( ; *str ; str++ ) if ( map[*str >> 3] & (1 << (*str & 7)) ) { *str++ = '/0'; break; } /* Update nextoken (or the corresponding field in the per-thread data * structure */ #ifdef _MT ptd->_token = str; #else /* _MT */ nextoken = str; #endif /* _MT */ /* Determine if a token has been found. */ if ( string == str ) return NULL; else return string; }