python正则表达式指南

Abstract（摘要）

This document is an introductory tutorial to using regular expressions in Python with the re module. It provides a gentler introduction than the corresponding section in the Library Reference.
本文是通过Python的 re 模块来使用正则表达式的一个入门教程，和库参考手册的对应章节相比，更为浅显易懂、循序渐进。

This document is available from http://www.amk.ca/python/howto .
本文可以从 http://www.amk.ca/python/howto 捕获

Contents（目录）

Contents

1. Introduction（简介）
2. Simple Patterns（简单模式）
   1. Matching Characters（字符匹配）
   2. Repeating Things（重复）
3. Using Regular Expressions（使用正则表达式）
   1. Compiling Regular Expressions（编译正则表达式）
   2. The Backslash Plague（反斜杠的麻烦）
   3. Performing Matches（执行匹配）
   4. Module-Level Functions（模块级函数）
   5. Compilation Flags（编译标志）
4. More Pattern Power（更多模式功能）
   1. More Metacharacters（更多的元字符）
   2. Grouping（分组）
   3. Non-capturing and Named Groups（无捕获组和命名组）
   4. Lookahead Assertions（前向界定符）
5. Modifying Strings（修改字符串）
   1. Splitting Strings（将字符串分片）
   2. Search and Replace（搜索和替换）
6. Common Problems（常见问题）
   1. Use String Methods（使用字符串方式）
   2. match() versus search()（match() vs search()）
   3. Greedy versus Non-Greedy（贪婪 vs 不贪婪）
   4. Not Using re.VERBOSE（不用 re.VERBOSE）
7. Feedback（反馈）
8. About this document（关于本文档）

 

 

Introduction（简介）

 

The re module was added in Python 1.5, and provides Perl-style regular expression patterns. Earlier versions of Python came with the regex module, which provides Emacs-style patterns. Emacs-style patterns are slightly less readable and don't provide as many features, so there's not much reason to use the regex module when writing new code, though you might encounter old code that uses it.
Python 自1.5版本起增加了re 模块，它提供 Perl 风格的正则表达式模式。Python 1.5之前版本则是通过 regex 模块提供 Emecs 风格的模式。Emacs 风格模式可读性稍差些，而且功能也不强，因此编写新代码时尽量不要再使用 regex 模块，当然偶尔你还是可能在老代码里发现其踪影。

Regular expressions (or REs) are essentially a tiny, highly specialized programming language embedded inside Python and made available through the re module. Using this little language, you specify the rules for the set of possible strings that you want to match; this set might contain English sentences, or e-mail addresses, or TeX commands, or anything you like. You can then ask questions such as "Does this string match the pattern?", or "Is there a match for the pattern anywhere in this string?". You can also use REs to modify a string or to split it apart in various ways.
就其本质而言，正则表达式（或 RE）是一种小型的、高度专业化的编程语言，（在Python中）它内嵌在Python中，并通过 re 模块实现。使用这个小型语言，你可以为想要匹配的相应字符串集指定规则；该字符串集可能包含英文语句、e-mail地址、TeX命令或任何你想搞定的东西。然后你可以问诸如“这个字符串匹配该模式吗？”或“在这个字符串中是否有部分匹配该模式呢？”。你也可以使用 RE 以各种方式来修改或分割字符串。

Regular expression patterns are compiled into a series of bytecodes which are then executed by a matching engine written in C. For advanced use, it may be necessary to pay careful attention to how the engine will execute a given RE, and write the RE in a certain way in order to produce bytecode that runs faster. Optimization isn't covered in this document, because it requires that you have a good understanding of the matching engine's internals.
正则表达式模式被编译成一系列的字节码，然后由用 C 编写的匹配引擎执行。在高级用法中，也许还要仔细留意引擎是如何执行给定 RE ，如何以特定方式编写 RE 以令生产的字节码运行速度更快。本文并不涉及优化，因为那要求你已充分掌握了匹配引擎的内部机制。

The regular expression language is relatively small and restricted, so not all possible string processing tasks can be done using regular expressions. There are also tasks that can be done with regular expressions, but the expressions turn out to be very complicated. In these cases, you may be better off writing Python code to do the processing; while Python code will be slower than an elaborate regular expression, it will also probably be more understandable.
正则表达式语言相对小型和受限（功能有限），因此并非所有字符串处理都能用正则表达式完成。当然也有些任务可以用正则表达式完成，不过最终表达式会变得异常复杂。碰到这些情形时，编写 Python 代码进行处理可能反而更好；尽管 Python 代码比一个精巧的正则表达式要慢些，但它更易理解。

 

Simple Patterns（简单模式）

 

We'll start by learning about the simplest possible regular expressions. Since regular expressions are used to operate on strings, we'll begin with the most common task: matching characters.
我们将从最简单的正则表达式学习开始。由于正则表达式常用于字符串操作，那我们就从最常见的任务：字符匹配 下手。

For a detailed explanation of the computer science underlying regular expressions (deterministic and non-deterministic finite automata), you can refer to almost any textbook on writing compilers.
有关正则表达式底层的计算机科学上的详细解释（确定性和非确定性有限自动机），你可以查阅编写编译器相关的任何教科书。

 

1. Matching Characters（字符匹配）

 

Most letters and characters will simply match themselves. For example, the regular expression test will match the string "test" exactly. (You can enable a case-insensitive mode that would let this RE match "Test" or "TEST" as well; more about this later.)
大多数字母和字符一般都会和自身匹配。例如，正则表达式 test 会和字符串“test”完全匹配。（你也可以使用大小写不敏感模式，它还能让这个 RE 匹配“Test”或“TEST”；稍后会有更多解释。）

There are exceptions to this rule; some characters are special, and don't match themselves. Instead, they signal that some out-of-the-ordinary thing should be matched, or they affect other portions of the RE by repeating them. Much of this document is devoted to discussing various metacharacters and what they do.
这个规则当然会有例外；有些字符比较特殊，它们和自身并不匹配，而是会表明应和一些特殊的东西匹配，或者它们会影响到 RE 其它部分的重复次数。本文很大篇幅专门讨论了各种元字符及其作用。

Here's a complete list of the metacharacters; their meanings will be discussed in the rest of this HOWTO.
这里有一个元字符的完整列表；其含义会在本指南余下部分进行讨论。

. ^ $ * + ? { [ ] / | ( )

 

The first metacharacters we'll look at are "[" and "]". They're used for specifying a character class, which is a set of characters that you wish to match. Characters can be listed individually, or a range of characters can be indicated by giving two characters and separating them by a "-". For example, [abc] will match any of the characters "a", "b", or "c"; this is the same as [a-c], which uses a range to express the same set of characters. If you wanted to match only lowercase letters, your RE would be [a-z].
我们首先考察的元字符是 "[" 和 "]"。它们常用来指定一个字符类别，所谓字符类别就是你想匹配的一个字符集。字符可以单个列出，也可以用“-”号分隔的两个给定字符来表示一个字符区间。例如， [abc] 将匹配"a", "b", 或 "c"中的任意一个字符；也可以用区间[a-c]来表示同一字符集，和前者效果一致。如果你只想匹配小写字母，那么 RE 应写成 [a-z]。

Metacharacters are not active inside classes. For example, [akm$] will match any of the characters "a", "k", "m", or "$"; "$" is usually a metacharacter, but inside a character class it's stripped of its special nature.
元字符在类别里并不起作用。例如，[akm$]将匹配字符"a", "k", "m", 或 "$" 中的任意一个；"$"通常用作元字符，但在字符类别里，其特性被除去，恢复成普通字符。

You can match the characters not within a range by complementing the set. This is indicated by including a "^{" as the first character of the class; `"</sup>" elsewhere will simply match the "<sup>"` character. For example, [}5] will match any character except "5".
你可以用补集来匹配不在区间范围内的字符。其做法是把"^"作为类别的首个字符；其它地方的"^"只会简单匹配 "^" 字符本身。例如，[^5] 将匹配除 "5" 之外的任意字符。

Perhaps the most important metacharacter is the backslash, "/". As in Python string literals, the backslash can be followed by various characters to signal various special sequences. It's also used to escape all the metacharacters so you can still match them in patterns; for example, if you need to match a "[" or "/", you can precede them with a backslash to remove their special meaning: /[ or //.
也许最重要的元字符是反斜杠"/"。 做为 Python 中的字符串字母，反斜杠后面可以加不同的字符以表示不同特殊意义。它也可以用于取消所有的元字符，这样你就可以在模式中匹配它们了。举个例子，如果你需要匹配字符 "[" 或 "/"，你可以在它们之前用反斜杠来取消它们的特殊意义： /[ 或 //。

Some of the special sequences beginning with "/" represent predefined sets of characters that are often useful, such as the set of digits, the set of letters, or the set of anything that isn't whitespace. The following predefined special sequences are available:
一些用 "/" 开始的特殊字符所表示的预定义字符集通常是很有用的，象数字集，字母集，或其它非空字符集。下列是可用的预设特殊字符：

/d

• Matches any decimal digit; this is equivalent to the class [0-9].
  匹配任何十进制数；它相当于类 [0-9]。

/D

• Matches any non-digit character; this is equivalent to the class [^0-9].
  匹配任何非数字字符；它相当于类 [^0-9]。

/s

• Matches any whitespace character; this is equivalent to the class [ /t/n/r/f/v].
  匹配任何空白字符；它相当于类 [ /t/n/r/f/v]。

/S

• Matches any non-whitespace character; this is equivalent to the class [^ /t/n/r/f/v].
  匹配任何非空白字符；它相当于类 [^ /t/n/r/f/v]。

/w

• Matches any alphanumeric character; this is equivalent to the class [a-zA-Z0-9_].
  匹配任何字母数字字符；它相当于类 [a-zA-Z0-9_]。

/W

• Matches any non-alphanumeric character; this is equivalent to the class [^a-zA-Z0-9_].
  匹配任何非字母数字字符；它相当于类 [^a-zA-Z0-9_]。

These sequences can be included inside a character class. For example, [/s,.] is a character class that will match any whitespace character, or "," or ".".
这样特殊字符都可以包含在一个字符类中。如，[/s,.]字符类将匹配任何空白字符或","或"."。

The final metacharacter in this section is .. It matches anything except a newline character, and there's an alternate mode (re.DOTALL) where it will match even a newline. "." is often used where you want to match any character.
本节最后一个元字符是 . 。它匹配除了换行字符外的任何字符，在 alternate 模式（re.DOTALL）下它甚至可以匹配换行。"." 通常被用于你想匹配“任何字符”的地方。

 

2. Repeating Things（重复）

 

Being able to match varying sets of characters is the first thing regular expressions can do that isn't already possible with the methods available on strings. However, if that was the only additional capability of regexes, they wouldn't be much of an advance. Another capability is that you can specify that portions of the RE must be repeated a certain number of times.
正则表达式第一件能做的事是能够匹配不定长的字符集，而这是其它能作用在字符串上的方法所不能做到的。 不过，如果那是正则表达式唯一的附加功能的话，那么它们也就不那么优秀了。它们的另一个功能就是你可以指定正则表达式的一部分的重复次数。

The first metacharacter for repeating things that we'll look at is *. * doesn't match the literal character "*"; instead, it specifies that the previous character can be matched zero or more times, instead of exactly once.
我们讨论的第一个重复功能的元字符是 *。* 并不匹配字母字符 "*"；相反，它指定前一个字符可以被匹配零次或更多次，而不是只有一次。

For example, ca*t will match "ct" (0 "a"characters), "cat" (1 "a"), "caaat" (3 "a"characters), and so forth. The RE engine has various internal limitations stemming from the size of C's int type, that will prevent it from matching over 2 billion "a" characters; you probably don't have enough memory to construct a string that large, so you shouldn't run into that limit.
举个例子， ca*t 将匹配 "ct" (0 个 "a" 字符), "cat" (1 个 "a"), "caaat" (3 个 "a" 字符)等等。RE 引擎有各种来自 C 的整数类型大小的内部限制，以防止它匹配超过2亿个 "a" 字符；你也许没有足够的内存去建造那么大的字符串，所以将不会累计到那个限制。

Repetitions such as * are greedy; when repeating a RE, the matching engine will try to repeat it as many times as possible. If later portions of the pattern don't match, the matching engine will then back up and try again with few repetitions.
象 * 这样地重复是“贪婪的”；当重复一个 RE 时，匹配引擎会试着重复尽可能多的次数。如果模式的后面部分没有被匹配，匹配引擎将退回并再次尝试更小的重复。

A step-by-step example will make this more obvious. Let's consider the expression a[bcd]*b. This matches the letter "a", zero or more letters from the class [bcd], and finally ends with a "b". Now imagine matching this RE against the string "abcbd".
一步步的示例可以使它更加清晰。让我们考虑表达式 a[bcd]*b。它匹配字母 "a"，零个或更多个来自类 [bcd]中的字母，最后以 "b" 结尾。现在想一想该 RE 对字符串 "abcbd" 的匹配。

Step	Matched	Explanation
1	a	The a in the RE matches. a 匹配模式
2	abcbd	The engine matches [bcd]*, going as far as it can, which is to the end of the string. 引擎匹配 [bcd]*，并尽其所能匹配到字符串的结尾
3	Failure	The engine tries to match b, but the current position is at the end of the string, so it fails. 引擎尝试匹配 b，但当前位置已经是字符的最后了，所以失败
4	abcb	Back up, so that [bcd]* matches one less character. 退回，[bcd]*尝试少匹配一个字符。
5	Failure	Try b again, but the current position is at the last character, which is a "d". 再次尝次b，但在当前最后一位字符是"d"。
6	abc	Back up again, so that [bcd]* is only matching "bc". 再次退回，[bcd]*只匹配 "bc"。
7	abcb	Try b again. This time but the character at the current position is "b", so it succeeds. 再次尝试 b ，这次当前位上的字符正好是 "b"

 

The end of the RE has now been reached, and it has matched "abcb". This demonstrates how the matching engine goes as far as it can at first, and if no match is found it will then progressively back up and retry the rest of the RE again and again. It will back up until it has tried zero matches for [bcd]*, and if that subsequently fails, the engine will conclude that the string doesn't match the RE at all.
RE 的结尾部分现在可以到达了，它匹配 "abcb"。这证明了匹配引擎一开始会尽其所能进行匹配，如果没有匹配然后就逐步退回并反复尝试 RE 剩下来的部分。直到它退回尝试匹配 [bcd] 到零次为止，如果随后还是失败，那么引擎就会认为该字符串根本无法匹配 RE 。

Another repeating metacharacter is +, which matches one or more times. Pay careful attention to the difference between * and +; * matches zero or more times, so whatever's being repeated may not be present at all, while + requires at least one occurrence. To use a similar example, ca+t will match "cat" (1 "a"), "caaat" (3 "a"'s), but won't match "ct".
另一个重复元字符是 +，表示匹配一或更多次。请注意 * 和 + 之间的不同；＊ 匹配零或更多次，所以根本就可以不出现，而 + 则要求至少出现一次。用同一个例子，ca+t 就可以匹配 "cat" (1 个 "a")， "caaat" (3 个 "a")， 但不能匹配 "ct"。

There are two more repeating qualifiers. The question mark character, ?, matches either once or zero times; you can think of it as marking something as being optional. For example, home-?brew matches either "homebrew" or "home-brew".
还有更多的限定符。问号 ? 匹配一次或零次；你可以认为它用于标识某事物是可选的。例如：home-?brew 匹配 "homebrew" 或 "home-brew"。

The most complicated repeated qualifier is {m,n}, where m and n are decimal integers. This qualifier means there must be at least m repetitions, and at most n. For example, a/{1,3}b will match "a/b", "a//b", and "a///b". It won't match "ab", which has no slashes, or "a////b", which has four.
最复杂的重复限定符是 {m,n}，其中 m 和 n 是十进制整数。该限定符的意思是至少有 m 个重复，至多到 n 个重复。举个例子，a/{1,3}b 将匹配 "a/b"，"a//b" 和 "a///b"。它不能匹配 "ab" 因为没有斜杠，也不能匹配 "a////b" ，因为有四个。

You can omit either m or n; in that case, a reasonable value is assumed for the missing value. Omitting m is interpreted as a lower limit of 0, while omitting n results in an upper bound of infinity -- actually, the 2 billion limit mentioned earlier, but that might as well be infinity.
你可以忽略 m 或 n；因为会为缺失的值假设一个合理的值。忽略 m 会认为下边界是 0，而忽略 n 的结果将是上边界为无穷大 -- 实际上是先前我们提到的 2 兆，但这也许同无穷大一样。

Readers of a reductionist bent may notice that the three other qualifiers can all be expressed using this notation. {0,} is the same as *, {1,} is equivalent to +, and {0,1} is the same as ?. It's better to use *, +, or ? when you can, simply because they're shorter and easier to read.
细心的读者也许注意到其他三个限定符都可以用这样方式来表示。 {0,} 等同于 *，{1,} 等同于 +，而{0,1}则与 ? 相同。如果可以的话，最好使用 *，+，或?。很简单因为它们更短也再容易懂。

 

Using Regular Expressions（使用正则表达式）

 

Now that we've looked at some simple regular expressions, how do we actually use them in Python? The re module provides an interface to the regular expression engine, allowing you to compile REs into objects and then perform matches with them.
现在我们已经看了一些简单的正则表达式，那么我们实际在 Python 中是如何使用它们的呢？ re 模块提供了一个正则表达式引擎的接口，可以让你将 REs 编译成对象并用它们来进行匹配。

 

1. Compiling Regular Expressions（编译正则表达式）

 

Regular expressions are compiled into RegexObject instances, which have methods for various operations such as searching for pattern matches or performing string substitutions.
正则表达式被编译成 RegexObject 实例，可以为不同的操作提供方法，如模式匹配搜索或字符串替换。

Toggle line numbers Toggle line numbers Toggle line numbers

 1 pat = re.compile(r"/s*(?P<header>[^:]+)/s*:(?P<value>.*?)/s*$")