elasticsearch ik分词器的安装和使用
elasticsearch几种常用分词器如下：
| 分词器 | 分词方式 |
|---|---|
| StandardAnalyzer | 单字分词 |
| CJKAnalyzer | 二分法 |
| IKAnalyzer | 词库分词 |
其中常用的是IKAnalyzer，但IK是第三方插件，需要安装。
安装分词器
下载
IK软件包下载地址：
https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v5.6.16
在tags栏选择与自己es相同版本的包，如下图对应方式：
可以选择源码自己编译安装也可以选择编译好的版本安装。由于我是内网环境没有办法自己编译所以选择了已经编译好的版本。
安装
1、解压 elasticsearch-analysis-ik-5.6.16.zip
unzip elasticsearch-analysis-ik-5.6.16.zip
解压后得到一个名为elasticsearch的文件夹，内容如下：
2、安装ik
在每个ES节点的，plugins目录下创建ik文件夹，将解压出来的文件全部拷贝到ik文件夹下。
mkdir -p ${your_es_dir}/plugins/ik
cp -a elasticsearch/* ${your_es_dir}/plugins/ik
3、重启ES。
由于做了修改，最好先前台重启一下，看下ES启动是否成功。
通常可能会遇到以下几种报错。
1） 5.3版本信息不全报错。
[2019-05-07T18:39:13,452][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hdzx_elk_02] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Could not load plugin descriptor for existing plugin [ik]. Was the plugin built before 2.0?
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.IllegalStateException: Could not load plugin descriptor for existing plugin [ik]. Was the plugin built before 2.0?
at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:295) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:131) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:302) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap$6.(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
... 6 more
Caused by: java.nio.file.NoSuchFileException: /home/admin/data/esdata/elasticsearch-5.3.0/plugins/ik/plugin-descriptor.properties
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?]
at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_111]
at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_111]
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_111]
at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_111]
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:86) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:292) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:131) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:302) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap$6.(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
解决办法
以上报错是由于我的es是5.3版本的所以我最初下载的是5.3版本的ik包，根据关键词java.nio.file.NoSuchFileException: /home/admin/data/esdata/elasticsearch-5.3.0/plugins/ik/plugin-descriptor.properties可以判断出是因为plugins/ik路径下缺少必要组件，但5.3版本的包并没有提供，于是便下载了5.6.16的ik 将jar包换成5.3的jar包继续启动。
- 使用5.6.16的版本启动报错如下。
[2019-05-07T18:53:04,881][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hdzx_elk_02] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [analysis-ik] is incompatible with Elasticsearch [5.3.0]. Was designed for version [5.6.16]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.IllegalArgumentException: Plugin [analysis-ik] is incompatible with Elasticsearch [5.3.0]. Was designed for version [5.6.16]
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:108) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.plugins.PluginsService.getPluginBundles(PluginsService.java:292) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:131) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:302) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap$6.(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
解决办法
根据关键词org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [analysis-ik] is incompatible with Elasticsearch [5.3.0]. Was designed for version [5.6.16]判断是插件的版本与ES 版本不符，但是ik插件官方文档说明5.* 版本的ik插件可以支持5.*版本的ES，所以修改ik插件的配置文件即可。
在${your_es_dir}/plugins/ik/plugin-descriptor.properties中修改如下配置，将5.6.16修改成5.3.0：
如下，将：
plugin-descriptor.properties:version=5.6.16
plugin-descriptor.properties:elasticsearch.version=5.6.16
改成
plugin-descriptor.properties:version=5.3.0
plugin-descriptor.properties:elasticsearch.version=5.3.0
重新启动ES即可。
配置IK分词器生效
1、ES5.*之前 可以直接在配置文件增加一行配置如下即可(我选择的是ik_smart,这个根据业务自行选择)
index.analysis.analyzer.default.type: "ik_smart"
但5.*之后，不支持在配置文件中做索引级别的配置，会报错如下：
*************************************************************************************
Found index level settings on node level configuration.
Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.
Please ensure all required values are updated on all indices by executing:
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
"index.analysis.analyzer.default.type" : "ik_smart"
}'
*************************************************************************************
[2019-05-07T21:18:27,927][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [hdzx_elk_02] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: node settings must not contain any index level settings
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:58) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.IllegalArgumentException: node settings must not contain any index level settings
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:132) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:335) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.node.Node.(Node.java:238) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap$6.(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
... 6 more
2、按照提示执行
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
"index.analysis.analyzer.default.type" : "ik_smart"
}'
会报错，返回json字符串，贴到json.cn里格式化一下，会发现主要报错如下：
"Can't update non dynamic settings [[index.analysis.analyzer.default.type]] for open indices
解决办法如下
curl -XPOST 'localhost:9200/_all/_close'
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{"index.analysis.analyzer.default.type" : "ik_smart"}'
curl -XPOST 'localhost:9200/_all/_open'
配置扩展词典
在${your_es_dir}/plugins/ik/config/IKAnalyzer.cfg.xml中如图框出来的位置增加字典。
所添加的字典文件中，添加你要扩展的字典。
