英文词干提取有多种方式，在实践中，可能涉及到机器学习数据挖掘等多方面的内容。
这里主要介绍的是易于实现的几种原始算法：
Lovins (1968)
Porter (1980)
Porter2 (2000)
1. Lovins
Lovins是最早的实现
1.1. 简介
算法涉及如下部件：
ending, 词后缀，共有294个，详细列表见最后
condition, 词后缀去除条件，每个ending对应一个condition，共有29个，详细列表见最后
transformation, 转换ending的方式，共有35个，详细列表见最后
算法分为两部：
对英文词，根据ending列表，按照ending从长到短扫描，找到第一个符合condition的ending
根据剩下的stem应用transformation，将ending转为恰当的形式
1.2. 例子
第一步
英文词为nationally，按照endling列表，从长到短扫描，首先找到 .09. ationally B，
对应的规则是B Minimum stem length = 3，要求去除ending后，剩余的部分长度大于等于3
nationally 去除 ationally 后只剩下 n, 不符合condition
继续扫描ending，找到 .07. ionally A，对应的规则是 A No restrictions on stem,没有任何限制。
于是最终选定 ionally作为ending
第二步
英文词nationally的stem是nat, 查找transformation，发现没有符合的transformation，不进行变换直接输出。
比如又一个词sitting，第一步得到stem是sitt, 第二步这里会应用第一条transformation，最终输出sit
1.Appendix.A endings 列表
.11.
alistically B arizability A izationally B
.10.
antialness A arisations A arizations A entialness A
.09.
allically C antaneous A antiality A arisation A
arization A ationally B ativeness A eableness E
entations A entiality A entialize A entiation A
ionalness A istically A itousness A izability A
izational A
.08.
ableness A arizable A entation A entially A
eousness A ibleness A icalness A ionalism A
ionality A ionalize A iousness A izations A
lessness A
.07.
ability A aically A alistic B alities A
ariness E aristic A arizing A ateness A
atingly A ational B atively A ativism A
elihood E encible A entally A entials A
entiate A entness A fulness A ibility A
icalism A icalist A icality A icalize A
ication G icianry A ination A ingness A
ionally A isation A ishness A istical A
iteness A iveness A ivistic A ivities A
ization F izement A oidally A ousness A
.06.
aceous A acious B action G alness A
ancial A ancies A ancing B ariser A
arized A arizer A atable A ations B
atives A eature Z efully A encies A
encing A ential A enting C entist A
eously A ialist A iality A ialize A
ically A icance A icians A icists A
ifully A ionals A ionate D ioning A
ionist A iously A istics A izable E
lessly A nesses A oidism A
.05.
acies A acity A aging B aical A
alist A alism B ality A alize A
allic BB anced B ances B antic C
arial A aries A arily A arity B
arize A aroid A ately A ating I
ation B ative A ators A atory A
ature E early Y ehood A eless A
elity A ement A enced A ences A
eness E ening E ental A ented C
ently A fully A ially A icant A
ician A icide A icism A icist A
icity A idine I iedly A ihood A
inate A iness A ingly B inism J
inity CC ional A ioned A ished A
istic A ities A itous A ively A
ivity A izers F izing F oidal A
oides A otide A ously A
.04.
able A ably A ages B ally B
ance B ancy B ants B aric A
arly K ated I ates A atic B
ator A ealy Y edly E eful A
eity A ence A ency A ened E
enly E eous A hood A ials A
ians A ible A ibly A ical A
ides L iers A iful A ines M
ings N ions B ious A isms B
ists A itic H ized F izer F
less A lily A ness A ogen A
ward A wise A ying B yish A
.03.
acy A age B aic A als BB
ant B ars O ary F ata A
ate A eal Y ear Y ely E
ene E ent C ery E ese A
ful A ial A ian A ics A
ide L ied A ier A ies P
ily A ine M ing N ion Q
ish C ism B ist A ite AA
ity A ium A ive A ize F
oid A one R ous A
.02.
ae A al BB ar X as B
ed E en F es E ia A
ic A is A ly B on S
or T um U us V yl R
s' A 's A
.01.
a A e A i A o A
s W y B 1.Appendix.B conditions 列表
A No restrictions on stem
B Minimum stem length = 3
C Minimum stem length = 4
D Minimum stem length = 5
E Do not remove ending after e
F Minimum stem length = 3 and do not remove ending after e
G Minimum stem length = 3 and remove ending only after f
H Remove ending only after t or ll
I Do not remove ending after o or e
J Do not remove ending after a or e
K Minimum stem length = 3 and remove ending only after l, i or u*e
L Do not remove ending after u, x or s, unless s follows o
M Do not remove ending after a, c, e or m
N Minimum stem length = 4 after s**, elsewhere = 3
O Remove ending only after l or i
P Do not remove ending after c
Q Minimum stem length = 3 and do not remove ending after l or n
R Remove ending only after n or r
S Remove ending only after dr or t, unless t follows t
T Remove ending only after s or t, unless t follows o
U Remove ending only after l, m, n or r
V Remove ending only after c
W Do not remove ending after s or u
X Remove ending only after l, i or u*e
Y Remove ending only after in
Z Do not remove ending after f
AA Remove ending only after d, f, ph, th, l, er, or, es or t
BB Minimum stem length = 3 and do not remove ending after met or ryst
CC Remove ending only after l1.Appendix.C transformations 列表
1 remove one of double b, d, g, l, m, n, p, r, s, t
2 iev -> ief
3 uct -> uc
4 umpt -> um
5 rpt -> rb
6 urs -> ur
7 istr -> ister
7a metr -> meter
8 olv -> olut
9 ul -> l except following a, o, i
10 bex -> bic
11 dex -> dic
12 pex -> pic
13 tex -> tic
14 ax -> ac
15 ex -> ec
16 ix -> ic
17 lux -> luc
18 uad -> uas
19 vad -> vas
20 cid -> cis
21 lid -> lis
22 erid -> eris
23 pand -> pans
24 end -> ens except following s
25 ond -> ons
26 lud -> lus
27 rud -> rus
28 her -> hes except following p, t
29 mit -> mis
30 ent -> ens except following m
31 ert -> ers
32 et -> es except following n
33 yt -> ys
34 yz -> ys 2. Porter
2.1. 简介
元音与辅音
元音辅音与常见的定义略有不同：
元音(Vowel) - A E I O U, 以及辅音后边的Y
辅音(Consonant) - 除了 A E I O U，以及元音后边的Y
单词的分组
连续的元音看作元音组V，连续的辅音看作辅音组C，于是任意一个单词都可以表示成VC交错的形式，例如：
segmentfault -> s/e/gm/e/ntf/au/lt -> CVCVCVC
porter -> p/o/rt/e/r -> CVCVC
application -> a/ppl/i/c/a/t/io/n -> VCVCVCVC
apple -> a/ppl/e -> V/C/V综合起来，可以表示为 VC 组的形式：$$ C^m[V] $$
其中参数m类似于Lovin中condition的stem长度，用于后续的判断
规则
Porter算法以rule为主，rule的形式为：
(condition) S1 -> S2condition作用于去除了S1的stem，除了m还有其他特征：
m - 表示VC组的数目
* - 表示任意字符, 和子串，v,d,o配合使用
大写字母 - 表示子串
v - 表示一个元音字符
d - 表示两个一样的辅音
o - 表示cvc, 其中第二个c不能是W,X,Y
S1是词的后缀，S2的变化后的后缀
和Lovin不同，一个词语经过多个规则的串联处理，输出目标词(Lovin是一次性输出)
例如 hopping, 首先应用规则(*v*) ING ->, 变为hopp
然后应用规则(*d and not (*L or *S or *Z)) -> single letter，从hopp变为hop
流程
整个算法是从上往下应用规则，有些规则比较特殊，如果触发了要处理额外的规则
规则很多，于是对规则进行分组(step)，这里的分组是为了逻辑上做区分(实际上算法也可以根据分组优化)，整个算法就是从头到位执行的，流程如下：
do Step_1a
do Step_1b (如果命中step 2b.2 or step 2b.3, 则做一些额外工作)
do Step_1c
do Step_2
do Step_3
do Step_4
do Step_5a
do Step_5b
每个Step的详细内容见附录
2.2. 例子
2.Appendix Step 1a
SSES -> SS
IES -> I
SS -> SS
S ->2.Appendix Step 1b
(m>0) EED -> EE
(*v*) ED ->
(*v*) ING ->
If the second or third of the rules in Step 1b is successful, the following is done:
AT -> ATE
BL -> BLE
IZ -> IZE
(*d and not (*L or *S or *Z)) -> single letter
(m=1 and *o) -> E2.Appendix Step 1c
(*v*) Y -> I2.Appendix Step 2
(m>0) ATIONAL -> ATE
(m>0) TIONAL -> TION
(m>0) ENCI -> ENCE
(m>0) ANCI -> ANCE
(m>0) IZER -> IZE
(m>0) ABLI -> ABLE
(m>0) ALLI -> AL
(m>0) ENTLI -> ENT
(m>0) ELI -> E
(m>0) OUSLI -> OUS
(m>0) IZATION -> IZE
(m>0) ATION -> ATE
(m>0) ATOR -> ATE
(m>0) ALISM -> AL
(m>0) IVENESS -> IVE
(m>0) FULNESS -> FUL
(m>0) OUSNESS -> OUS
(m>0) ALITI -> AL
(m>0) IVITI -> IVE
(m>0) BILITI -> BLE2.Appendix Step 3
(m>0) ICATE -> IC
(m>0) ATIVE ->
(m>0) ALIZE -> AL
(m>0) ICITI -> IC
(m>0) ICAL -> IC
(m>0) FUL ->
(m>0) NESS ->2.Appendix Step 4
(m>1) AL ->
(m>1) ANCE ->
(m>1) ENCE ->
(m>1) ER ->
(m>1) IC ->
(m>1) ABLE ->
(m>1) IBLE ->
(m>1) ANT ->
(m>1) EMENT ->
(m>1) MENT ->
(m>1) ENT ->
(m>1 and (*S or *T)) ION ->
(m>1) OU ->
(m>1) ISM ->
(m>1) ATE ->
(m>1) ITI ->
(m>1) OUS ->
(m>1) IVE ->
(m>1) IZE ->2.Appendix Step 5a
(m>1) E ->
(m=1 and not *o) E ->2.Appendix Step 5b
(m > 1 and *d and *L) -> single letter