docker容器中 搭建 elasticsearch 坑 ik head
文章目录
- docker容器中 搭建 elasticsearch 坑 ik head
- 1 安装
- 2 坑
- 2.1 报错
- 2.2 解决第一个坑
- 2.3 服务启动不了
- 2.4 分析第二个坑
- 2.5 宿主机的优化配置
- 3、安装分词器
- 3.1 上传分词器
- 3.2 测试
- 4 head安装
- 4.1 安装head
- 4.2 测试
docker容器中 搭建 elasticsearch 坑 ik head
1 安装
[root@localhost ~]# docker run -di --name=liubijun_el -p 9200:9200 -p 9300:9300 elasticsearch:5.6.8
13525c38b193943a01546aab8f1e232258ea9eee8a71016350730a1be2197f41
[root@localhost ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
13525c38b193 elasticsearch:5.6.8 "/docker-entrypoin..." 12 seconds ago Up 10 seconds 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp liubijun_el
0a816c1be4f2 mysql "docker-entrypoint..." 4 months ago Up About an hour 0.0.0.0:3306->3306/tcp, 33060/tcp liubijun_mysql
2 坑
2.1 报错
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{4JJen49tQAOw6iFRGxXuTQ}{192.168.85.198}{192.168.85.198:9300}]
通过报错信息，我们可以知道需要修改访问控制。elasticsearch.yml
2.2 解决第一个坑
[root@localhost ~]# docker exec -it liubijun_el /bin/bash
root@13525c38b193:/usr/share/elasticsearch# ls
NOTICE.txt README.textile bin config data lib logs modules plugins
root@13525c38b193:/usr/share/elasticsearch# cd config/
root@13525c38b193:/usr/share/elasticsearch/config# ls
elasticsearch.yml log4j2.properties scripts
[root@localhost ~]# docker cp liubijun_el:/usr/share/elasticsearch/config/elasticsearch.yml elasticsearch.yml
[root@localhost ~]# vim el
elasticsearch.yml el.tar
[root@localhost ~]# vim el
elasticsearch.yml el.tar
[root@localhost ~]# vim elasticsearch.yml
[root@localhost ~]# docker cp elasticsearch.yml liubijun_el:/usr/share/elasticsearch/config/elasticsearch.yml
[root@localhost ~]# docker restart liubijun_el
[root@localhost ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
13525c38b193 elasticsearch:5.6.8 "/docker-entrypoin..." 36 minutes ago Up 7 seconds 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp liubijun_el
0a816c1be4f2 mysql "docker-entrypoint..." 4 months ago Up About an hour 0.0.0.0:3306->3306/tcp, 33060/tcp liubijun_mysql
[root@localhost ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a816c1be4f2 mysql "docker-entrypoint..." 4 months ago Up About an hour 0.0.0.0:3306->3306/tcp, 33060/tcp liubijun_mysql
[root@localhost ~]# docker logs -f -t --tail 100 liubijun_el
2.3 服务启动不了
使用docker 命令查看日志。查找报错的原因
[root@localhost ~]# docker logs -f -t --tail 100 liubijun_el
2019-08-20T03:42:41.844980000Z [2019-08-20T03:42:41,838][INFO ][o.e.n.Node ] [] initializing ...
2019-08-20T03:42:42.236464000Z [2019-08-20T03:42:42,232][INFO ][o.e.e.NodeEnvironment ] [2_TNU5z] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/centos-root)]], net usable_space [22.1gb], net total_space [26.9gb], spins? [possibly], types [xfs]
2019-08-20T03:42:42.236717000Z [2019-08-20T03:42:42,233][INFO ][o.e.e.NodeEnvironment ] [2_TNU5z] heap size [1.9gb], compressed ordinary object pointers [true]
2019-08-20T03:42:42.236869000Z [2019-08-20T03:42:42,234][INFO ][o.e.n.Node ] node name [2_TNU5z] derived from node ID [2_TNU5zUSaSFeVR36M-rIQ]; set [node.name] to override
2019-08-20T03:42:42.237016000Z [2019-08-20T03:42:42,234][INFO ][o.e.n.Node ] version[5.6.8], pid[1], build[688ecce/2018-02-16T16:46:30.010Z], OS[Linux/3.10.0-693.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_162/25.162-b12]
2019-08-20T03:42:42.237292000Z [2019-08-20T03:42:42,235][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/share/elasticsearch]
2019-08-20T03:42:46.910967000Z [2019-08-20T03:42:46,909][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [aggs-matrix-stats]
2019-08-20T03:42:46.911238000Z [2019-08-20T03:42:46,909][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [ingest-common]
2019-08-20T03:42:46.911400000Z [2019-08-20T03:42:46,909][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-expression]
2019-08-20T03:42:46.911593000Z [2019-08-20T03:42:46,909][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-groovy]
2019-08-20T03:42:46.923269000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-mustache]
2019-08-20T03:42:46.923454000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-painless]
2019-08-20T03:42:46.923682000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [parent-join]
2019-08-20T03:42:46.923829000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [percolator]
2019-08-20T03:42:46.923974000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [reindex]
2019-08-20T03:42:46.924255000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [transport-netty3]
2019-08-20T03:42:46.924426000Z [2019-08-20T03:42:46,919][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [transport-netty4]
2019-08-20T03:42:46.924633000Z [2019-08-20T03:42:46,920][INFO ][o.e.p.PluginsService ] [2_TNU5z] no plugins loaded
2019-08-20T03:42:52.320751000Z [2019-08-20T03:42:52,319][INFO ][o.e.d.DiscoveryModule ] [2_TNU5z] using discovery type [zen]
2019-08-20T03:42:53.782424000Z [2019-08-20T03:42:53,774][INFO ][o.e.n.Node ] initialized
2019-08-20T03:42:53.782695000Z [2019-08-20T03:42:53,775][INFO ][o.e.n.Node ] [2_TNU5z] starting ...
2019-08-20T03:42:54.332980000Z [2019-08-20T03:42:54,331][INFO ][o.e.t.TransportService ] [2_TNU5z] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
2019-08-20T03:42:54.347009000Z [2019-08-20T03:42:54,345][WARN ][o.e.b.BootstrapChecks ] [2_TNU5z] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
2019-08-20T03:42:57.547282000Z [2019-08-20T03:42:57,537][INFO ][o.e.c.s.ClusterService ] [2_TNU5z] new_master {2_TNU5z}{2_TNU5zUSaSFeVR36M-rIQ}{n4KVzgyQRMuyu7V62fs0qQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
2019-08-20T03:42:57.622008000Z [2019-08-20T03:42:57,617][INFO ][o.e.h.n.Netty4HttpServerTransport] [2_TNU5z] publish_address {172.17.0.3:9200}, bound_addresses {[::]:9200}
2019-08-20T03:42:57.622297000Z [2019-08-20T03:42:57,617][INFO ][o.e.n.Node ] [2_TNU5z] started
2019-08-20T03:42:57.692486000Z [2019-08-20T03:42:57,686][INFO ][o.e.g.GatewayService ] [2_TNU5z] recovered [0] indices into cluster_state
2019-08-20T04:17:53.213570000Z [2019-08-20T04:17:53,207][INFO ][o.e.n.Node ] [2_TNU5z] stopping ...
2019-08-20T04:17:53.242121000Z [2019-08-20T04:17:53,239][INFO ][o.e.n.Node ] [2_TNU5z] stopped
2019-08-20T04:17:53.242525000Z [2019-08-20T04:17:53,239][INFO ][o.e.n.Node ] [2_TNU5z] closing ...
2019-08-20T04:17:53.272189000Z [2019-08-20T04:17:53,267][INFO ][o.e.n.Node ] [2_TNU5z] closed
2019-08-20T04:17:56.418863000Z [2019-08-20T04:17:56,415][INFO ][o.e.n.Node ] [] initializing ...
2019-08-20T04:17:56.614094000Z [2019-08-20T04:17:56,612][INFO ][o.e.e.NodeEnvironment ] [2_TNU5z] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/centos-root)]], net usable_space [22.1gb], net total_space [26.9gb], spins? [possibly], types [xfs]
2019-08-20T04:17:56.614333000Z [2019-08-20T04:17:56,613][INFO ][o.e.e.NodeEnvironment ] [2_TNU5z] heap size [1.9gb], compressed ordinary object pointers [true]
2019-08-20T04:17:56.620219000Z [2019-08-20T04:17:56,617][INFO ][o.e.n.Node ] node name [2_TNU5z] derived from node ID [2_TNU5zUSaSFeVR36M-rIQ]; set [node.name] to override
2019-08-20T04:17:56.620500000Z [2019-08-20T04:17:56,617][INFO ][o.e.n.Node ] version[5.6.8], pid[1], build[688ecce/2018-02-16T16:46:30.010Z], OS[Linux/3.10.0-693.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_162/25.162-b12]
2019-08-20T04:17:56.620708000Z [2019-08-20T04:17:56,617][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/share/elasticsearch]
2019-08-20T04:17:58.371146000Z [2019-08-20T04:17:58,370][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [aggs-matrix-stats]
2019-08-20T04:17:58.371393000Z [2019-08-20T04:17:58,370][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [ingest-common]
2019-08-20T04:17:58.371549000Z [2019-08-20T04:17:58,370][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-expression]
2019-08-20T04:17:58.384214000Z [2019-08-20T04:17:58,370][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-groovy]
2019-08-20T04:17:58.384507000Z [2019-08-20T04:17:58,380][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-mustache]
2019-08-20T04:17:58.384659000Z [2019-08-20T04:17:58,380][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [lang-painless]
2019-08-20T04:17:58.384834000Z [2019-08-20T04:17:58,381][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [parent-join]
2019-08-20T04:17:58.385091000Z [2019-08-20T04:17:58,381][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [percolator]
2019-08-20T04:17:58.385234000Z [2019-08-20T04:17:58,381][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [reindex]
2019-08-20T04:17:58.385559000Z [2019-08-20T04:17:58,381][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [transport-netty3]
2019-08-20T04:17:58.385704000Z [2019-08-20T04:17:58,381][INFO ][o.e.p.PluginsService ] [2_TNU5z] loaded module [transport-netty4]
2019-08-20T04:17:58.386022000Z [2019-08-20T04:17:58,381][INFO ][o.e.p.PluginsService ] [2_TNU5z] no plugins loaded
2019-08-20T04:18:01.883418000Z [2019-08-20T04:18:01,877][INFO ][o.e.d.DiscoveryModule ] [2_TNU5z] using discovery type [zen]
2019-08-20T04:18:02.975165000Z [2019-08-20T04:18:02,968][INFO ][o.e.n.Node ] initialized
2019-08-20T04:18:02.975526000Z [2019-08-20T04:18:02,968][INFO ][o.e.n.Node ] [2_TNU5z] starting ...
2019-08-20T04:18:03.275150000Z [2019-08-20T04:18:03,274][INFO ][o.e.t.TransportService ] [2_TNU5z] publish_address {172.17.0.3:9300}, bound_addresses {[::]:9300}
2019-08-20T04:18:03.302163000Z [2019-08-20T04:18:03,300][INFO ][o.e.b.BootstrapChecks ] [2_TNU5z] bound or publishing to a non-loopback address, enforcing bootstrap checks
2019-08-20T04:18:03.315973000Z ERROR: [1] bootstrap checks failed
2019-08-20T04:18:03.316159000Z [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
2019-08-20T04:18:03.335091000Z [2019-08-20T04:18:03,329][INFO ][o.e.n.Node ] [2_TNU5z] stopping ...
2019-08-20T04:18:03.431190000Z [2019-08-20T04:18:03,429][INFO ][o.e.n.Node ] [2_TNU5z] stopped
2019-08-20T04:18:03.431655000Z [2019-08-20T04:18:03,429][INFO ][o.e.n.Node ] [2_TNU5z] closing ...
2019-08-20T04:18:03.457176000Z [2019-08-20T04:18:03,455][INFO ][o.e.n.Node ] [2_TNU5z] closed
2.4 分析第二个坑
2019-08-20T04:18:03.315973000Z ERROR: [1] bootstrap checks failed
2019-08-20T04:18:03.316159000Z [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
通过日志分析需要对宿主机进行优化：
2.5 宿主机的优化配置
我们一共需要修改两处
修改/etc/security/limits.conf ，追加内容
nofile是单个进程允许打开的最大文件个数 soft nofile 是软限制 hard nofile是硬限制
修改/etc/sysctl.conf，追加内容
[root@localhost ~]# vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
[root@localhost ~]# vim /etc/sysctl.conf
vm.max_map_count=655360
[root@localhost ~]# reboot
限制一个进程可以拥有的VMA(虚拟内存区域)的数量
执行下面命令 修改内核参数马上生效
sysctl ‐P
3、安装分词器
3.1 上传分词器
>#yum install lrzsz unzip -y // 上传命令和解压缩命令
[root@localhost ~]# unzip ik.zip -d ik
Archive: ik.zip
creating: ik/ik/
inflating: ik/ik/commons-codec-1.9.jar
inflating: ik/ik/commons-logging-1.2.jar
creating: ik/ik/config/
inflating: ik/ik/config/custom.dic
inflating: ik/ik/config/extra_main.dic
inflating: ik/ik/config/extra_single_word.dic
inflating: ik/ik/config/extra_single_word_full.dic
inflating: ik/ik/config/extra_single_word_low_freq.dic
inflating: ik/ik/config/extra_stopword.dic
inflating: ik/ik/config/IKAnalyzer.cfg.xml
inflating: ik/ik/config/main.dic
inflating: ik/ik/config/preposition.dic
inflating: ik/ik/config/quantifier.dic
inflating: ik/ik/config/stopword.dic
inflating: ik/ik/config/suffix.dic
inflating: ik/ik/config/surname.dic
inflating: ik/ik/elasticsearch-analysis-ik-5.6.8.jar
inflating: ik/ik/httpclient-4.5.2.jar
inflating: ik/ik/httpcore-4.4.4.jar
inflating: ik/ik/plugin-descriptor.properties
[root@localhost ~]# ll
[root@localhost ik]# docker rm liubijun_el
liubijun_el
[root@localhost ik]# docker run -di --name=liubijun_el -p 9200:9200 -p 9300:9300 elasticsearch:5.6.8
c4de8f237ce06311b0f533778f2d0e21cd00648bab494178be57cea37df63a5c
[root@localhost ik]# docker cp ik liubijun_el:/usr/share/elasticsearch/plugins
3.2 测试
http://192.168.85.198:9200/_analyze?analyzer=ik_max_word&pretty=true&text=刘必君
{
tokens: [
{
token: "刘必君",
start_offset: 0,
end_offset: 3,
type: "CN_WORD",
position: 0
}
]
}
4 head安装
4.1 安装head
[root@localhost ~]# docker load < head.tar
b6ca02dfe5e6: Loading layer [==================================================>] 128.9 MB/128.9 MB
60a0858edcd5: Loading layer [==================================================>] 45.19 MB/45.19 MB
53c779688d06: Loading layer [==================================================>] 126.6 MB/126.6 MB
0a5e2b2ddeaa: Loading layer [==================================================>] 330.5 MB/330.5 MB
fa18e5ffd316: Loading layer [==================================================>] 352.3 kB/352.3 kB
604c78617f34: Loading layer [==================================================>] 113.7 kB/113.7 kB
cf2eea3d6e04: Loading layer [==================================================>] 50.16 MB/50.16 MB
d556e03b8284: Loading layer [==================================================>] 2.56 kB/2.56 kB
95ea76455b84: Loading layer [==================================================>] 12.72 MB/12.72 MB
ce5705289a91: Loading layer [==================================================>] 4.096 kB/4.096 kB
d09533ddfc0d: Loading layer [==================================================>] 143.3 MB/143.3 MB
eb415bbb4658: Loading layer [==================================================>] 23.71 MB/23.71 MB
Loaded image: mobz/elasticsearch-head:5
[root@localhost ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
mynginx_1.0 latest 242f3c6e0f81 4 months ago 109 MB
docker.io/nginx latest 27a188018e18 4 months ago 109 MB
docker.io/tomcat latest 5a069ba3df4d 4 months ago 465 MB
docker.io/mongo latest b1c8142b3efc 4 months ago 410 MB
docker.io/zookeeper latest 2a7f6fc5c8a1 4 months ago 150 MB
docker.io/redis latest a55fbf438dfd 4 months ago 95 MB
docker.io/mysql latest 7bb2586065cd 4 months ago 477 MB
docker.io/centos latest 9f38484d220f 5 months ago 202 MB
rabbitmq management df80af9ca0c9 12 months ago 149 MB
elasticsearch 5.6.8 6c0bdf761f3b 17 months ago 570 MB
mobz/elasticsearch-head 5 b19a5c98e43b 2 years ago 824 MB
[root@localhost ~]# docker run -di --name=liubijun_es_head -p 9100:9100 mobz/elasticsearch-head:5
89227b3809e84c4e1895fe06514685582ab5c6781e49900b26277271d34bd114
[root@localhost ~]#
4.2 测试
