Graylog日志采集系统
安装环境:Linux CentOS 7,jdk1.8 ,pwgen命令
安装组件:mongodb-4.4.1 ,elasticsearch-7.10.2 ,graylog-4.0
(1)、规划安装目录
cd /usr/local
(2)、下载安装包
访问elasticSearch官网地址 https://www.elastic.co/
下载指定版本的安装包:elasticsearch-7.10.2-linux-x86_64.tar.gz
(3)、上传安装包到指定目录
通过FTP工具上传安装包到指定目录,或者在服务器目录下直接下载:
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-linux-x86_64.tar.gz
(4)、解压安装包
tar -zxvf elasticsearch-7.10.2-linux-x86_64.tar.gz
(5)、重命名安装目录
mv elasticsearch-7.10.2 elasticsearch
(6)、修改配置文件
进入到es安装目录下的config文件夹中,修改elasticsearch.yml 文件。修改内容:
#集群名称设置为graylog
cluster.name: graylog
#禁止自动创建索引
action.auto_create_index: false
#设置外网访问
network.host: 0.0.0.0
#解决跨域问题
http.cors.enabled: true
http.cors.allow-origin: "*"
#设置为单机模式可快速启动
discovery.type: single-node
(7)、启动 elasticsearch, 并访问 ip:9200
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-6Rs1hjHd-1636944428084)(./pic/image-20210701172322791.png)]
(1)、规划安装目录
cd /usr/local
(2)、下载安装包
访问MongoDB官网地址 https://www.mongodb.com/
下载指定版本的安装包:mongodb-linux-x86_64-rhel70-4.4.1.tgz
(3)、上传安装包到指定目录
通过FTP工具上传安装包到指定目录,或者在服务器目录下直接下载。
(4)、解压安装包到对应目录
tar -zxvf mongodb-linux-x86_64-rhel70-4.4.1.tgz
mv mongodb-linux-x86_64-rhel70-4.4.1 /usr/local/momgodb
(5)、配置系统文件profile
sudo vi /etc/profile
添加以下内容:
export MONGODB_HOME=/usr/local/mongodb
export PATH=$PATH:$MONGODB_HOME/bin
注意保存后要重启系统配置:
source /etc/profile
(6)、创建用于存放数据和日志文件的文件夹,并修改其权限增加读写权限
cd /usr/local/mongodb
sudo mkdir -p data/db
sudo chmod -r 777 data/db
touch mongodb.log
(7)、mongodb启动配置
进入到bin目录,增加一个配置文件:
cd /usr/local/mongodb/bin
sudo vi mongodb.conf
插入下列内容:
dbpath = /usr/local/mongodb/data/db #数据文件存放目录
logpath = /usr/local/mongodb/logs/mongodb.log #日志文件存放目录
port = 27017 #端口
fork = true #以守护程序的方式启用,即在后台运行
(8)、启动mongod数据库服务,以配置文件的方式启动
cd /usr/local/mongodb/bin
./mongod -f mongodb.conf
#启动
./mongo
(9)、将mongodb 设置为系统服务
vim /etc/rc.d/init.d/mongod
#插入以下内容
#!/bin/sh
#
#mongod - Startup script for mongod
#chkconfig: - 85 15
#description: Mongodb database.
#processname: mongod
#Source function library
. /etc/rc.d/init.d/functions
#things from mongod.conf get there by mongod reading it
#OPTIONS
OPTIONS=" --dbpath=/home/data/mongodb/ --logpath=/home/data/mongodb/mongodb.log --logappend &"
#mongod
mongod="/usr/local/mongodb/bin/mongod"
lockfile=/var/lock/subsys/mongod
start()
{
echo -n $"Starting mongod: "
daemon $mongod $OPTIONS
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch $lockfile
}
stop()
{
echo -n $"Stopping mongod: "
killproc $mongod -QUIT
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f $lockfile
}
restart () {
stop
start
}
ulimit -n 12000
RETVAL=0
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload|force-reload)
restart
;;
condrestart)
[ -f $lockfile ] && restart || :
;;
status)
status $mongod
RETVAL=$?
;;
*)
echo "Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
RETVAL=1
esac
exit $RETVAL
(9)、创建管理员用户,以及garylog用户。
use admin
db.createUser({ user: "root", pwd: "123456", roles: [{ role:
"userAdminAnyDatabase", db: "admin" }] })
db.auth("root", "123456")
use graylog
db.createUser({ user: "grayloguser", pwd: "123456", roles: [{ role: "readWrite",
db: "graylog" }] })
(1)、安装graylog-server 以及相关插件。也可以选择从官网下载安装包进行安装
$ sudo rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-4.0-repository_latest.rpm
$ sudo yum install graylog-server graylog-enterprise-plugins graylog-integrations-plugins graylog-enterprise-integrations-plugins
(2)、修改graylog配置文件,可以直接将以下内容复制粘贴进去。
vi /etc/graylog/server/server.conf
is_master = true
node_id_file = /etc/graylog/server/node-id
password_secret = OMJHpCNySCGv5pmYHcUXh4LXqvky82uY7Cbe1VeXQxf9pMIhKhZd6FI4xcIlUhY975gf8tpo7S3tCbcKCGkX7qUUIW8PmzmG
root_username = admin
root_password_sha2 = 673ae3e8aafd8471c713ebcb76bf39e0c3d2db79980f27970aceb952c8b50715
root_timezone = Asia/Shanghai
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 10.4.230.122:9000
web_enable = true
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 1
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = true
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 20
outputbuffer_processors = 40
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
lb_recognition_period_seconds = 3
mongodb_uri = mongodb://grayloguser:[email protected]:27017/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5
content_packs_dir = /usr/share/graylog-server/contentpacks
content_packs_auto_load = grok-patterns.json
proxied_requests_thread_pool_size = 32
elasticsearch_hosts = http://10.4.230.122:9200
elasticsearch_discovery_enabled = false
#其中 root_password_sha2 通过过命令输入密码生成
echo -n "Enter Password: " && head -1
(3)、启动graylog并设置开机自启
$ sudo systemctl daemon-reload
$ sudo systemctl enable graylog-server.service
$ sudo systemctl start graylog-server.service
$ sudo systemctl --type=service --state=active | grep graylog
(4)、访问graylog页面 ip:9000
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-roVe14aO-1636944428087)(./pic/image-20210701172454255.png)]
elasticsearch_hosts 根据安装elasticsearch的安装进行进行修改
(3)、启动graylog并设置开机自启
$ sudo systemctl daemon-reload
$ sudo systemctl enable graylog-server.service
$ sudo systemctl start graylog-server.service
$ sudo systemctl --type=service --state=active | grep graylog
(4)、访问graylog页面 ip:9000
[外链图片转存中...(img-roVe14aO-1636944428087)]