k8s无脑系列(十一)-安装ElasticSearch到集群并设置中文分词

安装ElasticSearch到集群并设置中文分词

1. 先讲讲过程

用helm安装

helm install \
apphub/elasticsearch \
--name es-smokelee \
--set data.persistence.storageClass=common-svc-nfs-storage-class \
--set data.persistence.size=20Gi

安装过程很顺利，20来分钟全部搞定。再设置Ingress来外放接口出去

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: es-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: es.smokelee.com
    http:
      paths:
      - path: /
        backend:
          serviceName: es-smokelee-elasticsearch-coordinating-only
          servicePort: 9200

后来发现中文分词没有。必须的ICU也不在。

$curl http://es.smokelee.com/_cat/plugins
么有插件在

2. 解决思路

1. 到$HOME/.helm/cache/archive/找到char包解压，查看Chart
2. 通过找到Chart作者站点Github发现可以通过plugins参数添加ICU，并且可以通过image.repository image.tag image.registry来重新指定镜像仓库与地址
3. 但IK、Pinyin没有说明
4. 找到Image作者bitnami的Github(同一个人) 选择Tag 7.6.0-debian-10-r1 查看镜像Dockerfile
5. 查看Image说明
   1. icu可以通过设置plugins参数进行添加
   2. ik、pinyin需要手动添加（github.relase国内下载不了）
6. 下载ik、pinyin的zip包，重新编写Dockerfile
7. 重新打包并放入私有仓库

3. 重新制作镜像(可以抄了)

下载ES插件

打开ss，下载ik、pinyin。并解压到Docker目录下分别取名ik,pinyin

$wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.6.0/elasticsearch-analysis-ik-7.6.0.zip
$wget https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v7.6.0/elasticsearch-analysis-pinyin-7.6.0.zip

编写Dockerfile

$cat Dockerfile<<EOF
FROM docker.io/bitnami/elasticsearch:7.6.0-debian-10-r18
COPY ik /opt/bitnami/elasticsearch/plugins/ik
COPY pinyin /opt/bitnami/elasticsearch/plugins/pinyin
EOF
$docker build -t es.smokelee.com/base/elastic:7.6.0 .
$docker push es.smokelee.com/base/elastic:7.6.0

Helm开始安装

$helm install \
helm install \
apphub/elasticsearch \
--name hj-es \
--set image.repository=base/elastic-hj \
--set image.tag=7.6.0 \
--set image.registry=registry.smokelee.com \
--set master.replicas=2 \
--set master.heapSize=256m \
--set coordinating.heapSize=256m \
--set data.persistence.storageClass=common-svc-nfs-storage-class \
--set data.persistence.size=20Gi \
--set data.heapSize=1024m \
--set plugins=analysis-icu \
--set sysctlImage.enabled=false

测试

$curl http://es.smokelee.com
{
  "name" : "xxxxxxxxx",
  "cluster_name" : "xxxxxx",
  "cluster_uuid" : "xxxxxxx",
  "version" : {
    "number" : "7.6.0",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "7f634e9f44834fbc12724506cc1da681b0c3b1e3",
    "build_date" : "2020-02-06T00:09:00.449973Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

查看插件，已经全部安装。

$curl http://es.smokelee.com/_cat/plugins
es-smokelee-elasticsearch-coordinating-only-6ccbb4fb78-z9btt analysis-icu    7.6.0
es-smokelee-elasticsearch-coordinating-only-6ccbb4fb78-z9btt analysis-ik     7.6.0
es-smokelee-elasticsearch-coordinating-only-6ccbb4fb78-z9btt analysis-pinyin 7.6.0
es-smokelee-elasticsearch-coordinating-only-6ccbb4fb78-9lm5h analysis-icu    7.6.0
es-smokelee-elasticsearch-coordinating-only-6ccbb4fb78-9lm5h analysis-ik     7.6.0
es-smokelee-elasticsearch-coordinating-only-6ccbb4fb78-9lm5h analysis-pinyin 7.6.0
es-smokelee-elasticsearch-master-1                           analysis-icu    7.6.0
es-smokelee-elasticsearch-master-1                           analysis-ik     7.6.0
es-smokelee-elasticsearch-master-1                           analysis-pinyin 7.6.0
es-smokelee-elasticsearch-master-0                           analysis-icu    7.6.0
es-smokelee-elasticsearch-master-0                           analysis-ik     7.6.0
es-smokelee-elasticsearch-master-0                           analysis-pinyin 7.6.0

中文测试，成功！

$curl --header "Content-Type: application/json" 'http://es.smokelee.com/_analyze?pretty=true' -XPOST -d '{"text":"这里是好记性不如烂笔头感叹号的博客园","analyzer":"ik_smart"}'
{
  "tokens" : [
    {
      "token" : "清华大学",
      "start_offset" : 0,
      "end_offset" : 4,
      "type" : "CN_WORD",
      "position" : 0
    },
    {
      "token" : "是",
      "start_offset" : 4,
      "end_offset" : 5,
      "type" : "CN_CHAR",
      "position" : 1
    },
    {
      "token" : "一",
      "start_offset" : 5,
      "end_offset" : 6,
      "type" : "TYPE_CNUM",
      "position" : 2
    },
    {
      "token" : "所好",
      "start_offset" : 6,
      "end_offset" : 8,
      "type" : "CN_WORD",
      "position" : 3
    },
    {
      "token" : "大学",
      "start_offset" : 8,
      "end_offset" : 10,
      "type" : "CN_WORD",
      "position" : 4
    }
  ]
}