安装graylog2日志服务器

系统架构:1.最小安装 2.生产部署 3.高可用部署

wKioL1OirfaA8Ix1AAOYo8NCeTE684.jpg

wKioL1OirjfweZkKAAIiXsND3oU778.jpg

Highly available setup with Graylog2 Radio

wKiom1Oirq2wjS3TAAN6YWcLjhU491.jpg

安装前要求:

  1. Elasticsearch v0.90.10

  2. MongoDB (as recent stable version as possible, at least v2.0)

  3. Java 7


添加系统以外源:

http://kernal.blog.51cto.com/8136890/1426095

Graylog2 is an excellent centralized logging application created by the excellent guys at torch.sh which utilizes elasticsearch to store logs. It’s scaleable, robust, can deal with a huge number of logs (if coupled with Graylog2-Radio) and best of all, is open source.

There are two components required before we actually install the server and web component of the app. First, we need to have a mongo db database.

1.[install&configure Mongo]

vim /etc/yum.repos.d/mongodb.repo
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
enabled=1
gpgcheck=0
yum --enablerepo=mongodb -y install mongo-10gen mongo-10gen-server
# service mongod start
Starting mongod:                                           [  OK  ]
# chkconfig mongod on

# 开启服务并加入开机启动

我们需要给admin创建一个密码,使用mongo命令

# mongo
MongoDB shell version: 2.6.1
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
    http://docs.mongodb.org/
Questions? Try the support group
    http://groups.google.com/group/mongodb-user
> use admin                            # 切换到adin数据库
switched to db admin
> db.createUser(
... {
... user : "admin",
... pwd : "logadmin",
... roles : [ "readWrite", "dbAdmin" ]
... }
... )
Successfully added user: { "user" : "admin", "roles" : [ "readWrite", "dbAdmin" ] }
> db.auth('admin','logadmin')
1                                      # 返回1为认证成功

我们需要为graylog2创建一个数据库:

> use graylog2                         # Mongodb使用use就是创建数据库
switched to db graylog2
> db.createUser(
... {
... user : "graylog",
... pwd : "graylog",
... roles : [ "readWrite", "dbAdmin" ]
... }
... )
Successfully added user: { "user" : "graylog", "roles" : [ "readWrite", "dbAdmin" ] }
> db.auth('graylog', 'graylog')        # 认证测试
1

2.[install&configure Elasticsearch]

运行一个elasticsearch集群不是一个容易的事情,我这里假设只需要一台elasticsearch,其它文档请看http://www.elasticsearch.org/guide/

Elasticsearch已经有rpm包所以安装很简单,graylog2仅适用于特定的elasticsearc版本,安装时请注意,elasticsearch需要有java环境。

# java -version
java version "1.7.0_55"
OpenJDK Runtime Environment (rhel-2.4.7.1.el6_5-x86_64 u55-b13)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)

如果没有请安装:

# yum -y install java7
# wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.noarch.rpm
# yum -y install elasticsearch-0.90.10.noarch.rpm
vim /etc/elasticsearch/elasticsearch.yml
32 cluster.name: graylog2.carson.cn
182 bootstrap.mlockall: true
319 discovery.zen.ping.multicast.enabled: false
324 discovery.zen.ping.unicast.hosts: ["graylog2.carson.cn"]
# service elasticsearch start
# service elasticsearch restart
Stopping elasticsearch:                                    [  OK  ]
Starting elasticsearch:                                    [  OK  ]
# cat /var/log/elasticsearch/graylog2.carson.cn.log
[root@graylog2 elasticsearch]# tail /var/log/elasticsearch/graylog2.carson.log 
[2014-06-16 11:06:35,079][INFO ][node                     ] [Spectral] initializing ...
[2014-06-16 11:06:35,088][INFO ][plugins                  ] [Spectral] loaded [], sites []
[2014-06-16 11:06:38,316][INFO ][node                     ] [Spectral] initialized
[2014-06-16 11:06:38,316][INFO ][node                     ] [Spectral] starting ...
[2014-06-16 11:06:38,430][INFO ][transport                ] [Spectral] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.1.186:9300]}
[2014-06-16 11:06:41,488][INFO ][cluster.service          ] [Spectral] new_master [Spectral][6Ry1_DIETdiEtNXloG3K-Q][inet[/192.168.1.186:9300]], reason: zen-disco-join (elected_as_master)
[2014-06-16 11:06:41,605][INFO ][discovery                ] [Spectral] graylog2.carson.cn/6Ry1_DIETdiEtNXloG3K-Q
[2014-06-16 11:06:41,663][INFO ][http                     ] [Spectral] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.186:9200]}
[2014-06-16 11:06:41,664][INFO ][node                     ] [Spectral] started
[2014-06-16 11:06:41,695][INFO ][gateway                  ] [Spectral] recovered [0] indices into cluster_state

能看到这信息,以确保以上配置正确;

3.[install&configure graylog2-server]

#wget https://github.com/jaxxstorm/graylog2-server-rpm/releases/download/0.20.0-rc1-1/graylog2-server-0.20.0-rc1.1.el6.noarch.rpm -O graylog2-server-0.20.0-rc1.1.el6.noarch.rpm
# yum -y install graylog2-server-0.20.0-rc1.1.el6.noarch.rpm
# yum install perl-Digest-SHA

创建个脚本随机生成字符串:64位

# cat string.sh
#!/bin/bash
 
randstr() {
  index=0
  str=""
  for i in {a..z}; do arr[index]=$i; index=`expr ${index} + 1`; done
  for i in {A..Z}; do arr[index]=$i; index=`expr ${index} + 1`; done
  for i in {0..9}; do arr[index]=$i; index=`expr ${index} + 1`; done
  for i in {1..64}; do str="$str${arr[$RANDOM%$index]}"; done
  echo $str
}
echo `randstr`

The binaries live in /opt/graylog2 and the config files live in /etc/graylog2. In this case we need to set a few config options in /etc/graylog2/server.conf

  • is_master = true �C you need at least one

  • password_secret �C set a 64 character string here. You’ll need to reuse this for any additional server nodes and the web interface portion

  • root_password_sha2 = enter your root password’s hash here

  • elasticsearch_shards = 1 �C you only have one elasticsearch host at the moment, so ensure this is set to 1 (change it if you have more than one shard, obviously)

  • elasticsearch_replicas = 0 �C see above

  • elasticsearch_cluster_name = graylog2 �C set this to the same as your elasticsearch cluster name

  • elasticsearch_transport_tcp_port = 9350 �C make sure this is not the same as your elasticsearch node you configured previously

  • elasticsearch_discovery_zen_ping_multicast_enabled = false
    elasticsearch_discovery_zen_ping_unicast_hosts = localhost:9300 �C remember what we said about multicast previously? This allows you to discover the cluster

  • Mongodb info �C make sure you set useauth to true, and add your database, username and password here


cat /etc/graylog2/server.conf
is_master = true
node_id_file = /etc/graylog2-server-node-id
password_secret = JtMBS4TbbjtPALosVZUk50sUYnsc0pVOkkpKzrD40r6nsoSl5fnSZ6z3PWflFWRy #随机生成
root_password_sha2 = 76cd2c0d...7c1b28bee   # 通过 echo -n yourpassword | shasum -a 256 获得
plugin_dir = plugin 
rest_listen_uri = http://127.0.0.1:12900/
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 1                    # 集群数量
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog2
allow_leading_wildcard_searches = false
elasticsearch_cluster_name = graylog2       # 这个名称和elasticsearch.yml中的配置一样
elasticsearch_node_name = graylog2-server
elasticsearch_transport_tcp_port = 9350           # 注意下这个端口号
elasticsearch_discovery_zen_ping_multicast_enabled = false         # 建议使用单播方式
elasticsearch_discovery_zen_ping_unicast_hosts = localhost:9300    # 改成localhost(可以写好多个)
elasticsearch_analyzer = standard
output_batch_size = 5000
processbuffer_processors = 5
outputbuffer_processors = 5
processor_wait_strategy = blocking
ring_size = 1024
mongodb_useauth = true
mongodb_user = grayloguser                # 在Mongodb数据库中创建用户
mongodb_password = gl2-password           # 在Mongodb数据库中创建的密码
mongodb_host = 127.0.0.1                  # Mongodb服务主机ip或者hostname
mongodb_database = graylog2               # 创建的数据库
mongodb_port = 27017                      # 链接数据库的端口号
mongodb_max_connections = 100
mongodb_threads_allowed_to_block_multiplier = 5
transport_email_enabled = false
transport_email_hostname = mail.example.com
transport_email_port = 587
transport_email_use_auth = true
transport_email_use_tls = true
transport_email_use_ssl = true
transport_email_auth_username = [email protected]
transport_email_auth_password = secret
transport_email_subject_prefix = [graylog2]
transport_email_from_email = [email protected]

验证以上配置是否正确

# java -jar /opt/graylog2/server/graylog2-server.jar -f /etc/graylog2/server.conf
2014-06-16 12:24:43,890 INFO : org.graylog2.outputs.OutputRegistry - Initialized output <org.graylog2.outputs.ElasticSearchOutput>.
2014-06-16 12:24:44,023 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Index [graylog2_0] is empty. Not calculating ranges.
2014-06-16 12:24:44,025 INFO : org.graylog2.indexer.ranges.RebuildIndexRangesJob - Done calculating index ranges for 1 indices. Took 161ms.
2014-06-16 12:24:44,027 INFO : org.graylog2.system.jobs.SystemJobManager - SystemJob <24451020-f50e-11e3-863a-1ab442c6715b> [org.graylog2.indexer.ranges.RebuildIndexRangesJob] finished in 200ms.
2014-06-16 12:24:49,223 INFO : org.glassfish.jersey.server.ApplicationHandler - Initiating Jersey application, version Jersey: 2.5 2013-12-18 14:27:29...
2014-06-16 12:24:51,970 INFO : org.graylog2.Core - Started REST API at <http://127.0.0.1:12900/>
# service graylog2-server start
Starting graylog2-server: 
# chkconfig graylog2-server on

4.[install&configure graylog2-web-interface]

# wget https://github.com/jaxxstorm/graylog2-web-rpm/releases/download/0.20.0-rc1-1/graylog2-web-0.20.0-rc1.1.el6.noarch.rpm -O graylog2-web-0.20.0-rc1.1.el6.noarch.rpm
# yum install graylog2-web-0.20.0-rc1.1.el6.noarch.rpm

The config file for the web interface is much simpler than the server interface. Take a look in /etc/graylog2/web.conf. You need two fields

  • graylog2-server.uris �C set this to the server address, usually local host unless you made them seperate

  • application.secret=”" �C set this to the same key you have in server.conf password_secret

# vim /etc/graylog2/web.conf
graylog2-server.uris="
application.secret="JtMBS4TbbjtPALosVZUk50sUYnsc0pVOkkpKzrD40r6nsoSl5fnSZ6z3PWflFWRy"
field_list_limit=100
# /opt/graylog2/web/bin/graylog2-web-interface -Dconfig.file=/etc/graylog2/web.conf 
Play server process ID is 3153
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

确保以上配置文件正确!

# service graylog2-web start
# chkconfig graylog2-web on

先写到这....

你可能感兴趣的:(elasticsearch,日志服务器,graylog2)