docker-compose 安装ES系列

注明：新手程序员笔记，没配置集群，先从单点开始熟悉功能
环境：windows，选docker是为了之后适应线上环境部署

先去官网上翻了一下最新版本：8.2.2
那么整个系列都确定用一个版本
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

image.png

1. 启动elasticsearch

说明：8.2的一些配置和7不同，用7的配置可能会启动不起来哦

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.2.2
    container_name: es
    privileged: true
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml 
      - ./es/data:/usr/share/elasticsearch/data #数据文件挂载
      - ./es/plugins:/usr/share/elasticsearch/plugins #插件文件挂载
    ports:
      - 9200:9200
    networks:
      - esnet

networks:
  esnet:
    driver: bridge

elasticsearch.yml文件：

cluster.name: "elasticsearch" #设置集群名称为elasticsearch
network.host: 0.0.0.0
discovery.type: single-node #以单一节点模式启动

# 是否支持跨域
http.cors.enabled: true
# 表示支持所有域名
http.cors.allow-origin: "*"
# 内存交换的选项，官网建议为true
bootstrap.memory_lock: true
# resetful端口
http.port: 9200

# xpack安全
xpack.security.enabled: false

执行docker-compose up -d
在本地访问：http://127.0.0.1:9200/ 确定es已经正常运行

2. 启动kibana

kibana可以理解为，类似mysql的图形化client，给开发用的
准备配置文件：./es/kibana/config/kibana.yml

server.host: "0.0.0.0"
server.name: kibana
xpack.monitoring.ui.container.elasticsearch.enabled: true
i18n.locale: zh-CN #中文

说明：kibana-8.2.2配置文件kibana.yml中的elasticsearch.hosts字段不需要填写

kibana:
    image: docker.elastic.co/kibana/kibana:8.2.2
    container_name: kibana
    volumes:
      - ./kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch #kibana在elasticsearch启动之后再启动      
    networks:
      - esnet

执行docker-compose up -d
在本地访问：http://127.0.0.1:5601/ 确定kibana已经正常运行
首次启动ikbana需要配置它对应的ES，选择手动配置：http://elasticsearch:9200
说明：这个地方的访问地址，要根据kibana容器内访问es的地址来配置。因为用了docker-compose所以千万不能写成：127.0.0.1:9200，还有注意是http不是https。
配置后提示需要输入一个安全码，进入kibana的容器，运行获得

$ docker exec -it kibana sh
$ cd /usr/share/kibana/bin
$ kibana-verification-code
Your verification code is:  273 883

3. Logstash

只是先跟着教程安装了一个，具体怎么用不清楚

logstash:
    image: docker.elastic.co/logstash/logstash:8.2.2
    container_name: logstash
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" #限制内存
    ports:
      - 5044:5044
    volumes:
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./logstash/pipeline:/usr/share/logstash/pipeline
    depends_on:
      - elasticsearch #logstash在elasticsearch启动之后再启动
    networks:
      - esnet

logstash.yml

http.host: "0.0.0.0"

pipleline/logstash.conf

input {
  file {
    path => "/usr/share/logstash/pipeline/data/movies.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {

    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }

}
output {
   elasticsearch {
     hosts => "http://elasticsearch:9200" # 要配置成docker内访问的
     index => "movies"
     document_id => "%{id}"
   }
  stdout {}
}

4. Cerebro安装

可以简单理解为，给运维看的

 cerebro:
    image: lmenezes/cerebro:0.8.3
    container_name: cerebro
    ports:
      - "9100:9000"
    depends_on:
      - elasticsearch #logstash在elasticsearch启动之后再启动
    networks:
      - esnet