半江瑟瑟

ELK+Filebeat日志收集集群搭建

ELK安装教程

作者:半江瑟瑟

ELK安装教程
Elasticsearch 6.3.1 安装

1.1创建elk域

groupadd elk

useradd -g elk -m elk

usermod -a -G elk elk

groups elk

passwd elk

密码：elk123

1.2. 修改配置文件，防止安装时出现问题(root 目录操作)

1. 2.1修改limit信息

vi /etc/security/limits.conf

* soft nofile 65536

* hard nofile 131072

* soft nproc 4096

* hard nproc 4096

* soft memlock unlimited

* hard memlock unlimited

执行：source /etc/ security/limits.conf

1.2.2 修改集群配置信息

 $ vim /etc/hosts
11.181.14.181 node-1
11.181.14.182 node-2
11.181.14.183 node-3

1.2.3修改sysctl 文件

vi /etc/sysctl.conf

添加下面配置：

vm.max_map_count=655360

并执行命令：

sysctl -p

1.3 安装elasticsearch

1.3.1 上传elasticsearch 文件到elk组下

在/home/elk/ap/ifsp/下上传 elasticsearch-6.3.1.zip/ elasticsearch-6.3.1.tz 文件

su elk 进入elk用户

Zip:

Unzip elasticsearch-6.3.1.zip 命令解压

tar -zxvf elasticsearch-6.3.1.tz 命令解压

1.3.2修改配置文件

[elk@gaapospapp31l-14181 config]$ cd /home/elk/ap/ifsp/elasticsearch-6.3.1/config

cd /home/elk/ap/ifsp/elasticsearch-6.3.1/conf/

vi elasticsearch.yml

# ======================== Elasticsearch Configuration =========================

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

# Before you set out to tweak and tune the configuration, make sure you

# understand what are you trying to accomplish and the consequences.

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

# Please consult the documentation for further information on configuration options:

# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

# ---------------------------------- Cluster -----------------------------------

# Use a descriptive name for your cluster:

#集群名称

cluster.name: application

# ------------------------------------ Node ------------------------------------

# Use a descriptive name for the node:

#节点名同一集群下节点名称不同

node.name: node-1

#主节点（父节点）

node.master: true

node.data: false

# Add custom attributes to the node:

#node.attr.rack: r1

# ----------------------------------- Paths ------------------------------------

# Path to directory where to store the data (separate multiple locations by comma):

#路径修改为自己设置的路径

path.data: /home/elk/ap/ifsp/data

# Path to log files:

#路径修改为自己设置的路径

path.logs: /home/elk/ap/ifsp/logs

# ----------------------------------- Memory -----------------------------------

# Lock the memory on startup:

#设置内存锁定，防止数据swap

bootstrap.memory_lock: true

bootstrap.system_call_filter: false

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

# Elasticsearch performs poorly when the system is swapping the memory.

# ---------------------------------- Network -----------------------------------

# Set the bind address to a specific IP (IPv4 or IPv6):

#设置绑定地址，这里取本机地址

network.host: 11.18.14.181

# Set a custom port for HTTP:

#本机端口

http.port: 9200

# For more information, consult the network module documentation.

# --------------------------------- Discovery ----------------------------------

# Pass an initial list of hosts to perform discovery when new node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

#集群中其他节点ip

discovery.zen.ping.unicast.hosts: ["11.18.14.181", "11.18.14.182","11.18.14.183"]

# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):

#discovery.zen.minimum_master_nodes:

# For more information, consult the zen discovery module documentation.

# ---------------------------------- Gateway -----------------------------------

# Block initial recovery after a full cluster restart until N nodes are started:

#gateway.recover_after_nodes: 3

# For more information, consult the gateway module documentation.

# ---------------------------------- Various -----------------------------------

# Require explicit names when deleting indices:

#action.destructive_requires_name: true

这里三台机器进行集群，每台的配置和这个类似只是稍微有些去别，配置的集群名在所有参加集群的机器中都是一样的，只是节点的地方有改动例如下面时配置：