大量字符串正则哪家快

使用pd.Series.str更快

本文对比了pd.Series.apply和pd.Series.str在大量字符串正则上的效率，实验如下

import numpy as np
import string

# 随机生成1-100长度的字符串
def random_str():
    return ''.join(np.random.choice(list(string.ascii_letters) + list(string.digits), size=np.random.randint(1, 100), replace=True))

print(random_str())

ao3iemnPTYirQW5HYutAj

# 生成一千万条字符串
str_ls = [random_str() for i in range(int(1e7))]
len(str_ls)

10000000

# 分配到一个DataFrame里
import pandas as pd
df = pd.DataFrame(str_ls, columns=['string'])

# 定义正则表达式
import re
str_regex = re.compile('(IOU)')

# 对比两种方法的效率
import time

start = time.time()
df['apply_res'] = df['string'].apply(lambda x: re.match(str_regex, x))
end = time.time()

print('apply use time:')
print(end - start)

start = time.time()
df['pandas_res'] = df['string'].str.match(str_regex)
end = time.time()

print('pandas str regex use time:')
print(end - start)

apply use time:
10.389425277709961
pandas str regex use time:
4.18695330619812

可见pd.Series.str快了1倍多

完