Flume 数据流监控 Ganglia 的安装与部署

Ganglia 的安装与部署

1.安装httpd 服务与 php

sudo yum -y install httpd php

2.安装其它依赖

sudo yum -y install rrdtool perl-rrdtool rrdtool-devel

sudo yum -y install apr-devel

3.安装 ganglia

sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm #推荐使用

sudo rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm #已经过时了（无法使用）

sudo yum -y install ganglia-gmetad 

sudo yum -y install ganglia-web

sudo yum -y install ganglia-gmond

Ganglia 由 gmond、gmetad 和 gweb 三部分组成。

gmond（Ganglia Monitoring Daemon）是一种轻量级服务，安装在每台需要收集指标

数据的节点主机上。使用 gmond，你可以很容易收集很多系统指标数据，如 CPU、内存、

磁盘、网络和活跃进程的数据等。

gmetad（Ganglia Meta Daemon）整合所有信息，并将其以 RRD 格式存储至磁盘的服

务。

gweb（Ganglia Web）Ganglia 可视化工具，gweb 是一种利用浏览器显示 gmetad 所存

储数据的 PHP 前端。在 Web 界面中以图表方式展现集群的运行状态下收集的多种不同指标

数据。

4.修改配置文件/etc/httpd/conf.d/ganglia.conf

sudo vim /etc/httpd/conf.d/ganglia.conf

4.1 修改为红色的配置

# Ganglia monitoring system php web frontend

Alias /ganglia /usr/share/ganglia

Order deny,allow  #这个也可以注释掉

#Deny from all

# Allow from all    #这是老版本的修改(不推荐)

# Allow from 127.0.0.1

# Allow from ::1

# Allow from .example.com

  Require all granted  #新推荐的配置

 

5.修改配置文件/etc/ganglia/gmetad.conf

sudo vim /etc/ganglia/gmetad.conf

5.1 修改内容为:

data_source "hadoop102" 192.168.10.102 #注意修改正确的，这data_source会有多个 注意看示例的注释部分

#data_source "my cluster" localhost
 data_source "hadoop102" 192.168.10.102

6.修改配置文件/etc/ganglia/gmond.conf

 sudo vim /etc/ganglia/gmond.conf

6.1修改内容为:

修改为：

cluster {

name = "hadoop102"

owner = "unspecified"

latlong = "unspecified"

url = "unspecified"

}

udp_send_channel {

#bind_hostname = yes # Highly recommended, soon to be default.

# This option tells gmond to use a source

address

# that resolves to the machine's hostname.

Without

# this, the metrics may appear to come from any

# interface and the DNS names associated with

# those IPs will be used to create the RRDs.

# mcast_join = 239.2.11.71

host = 192.168.9.102

port = 8649

ttl = 1

}

udp_recv_channel {

# mcast_join = 239.2.11.71

port = 8649

bind = 192.168.9.102

retry_bind = true

# Size of the UDP buffer. If you are handling lots of metrics you

really

# should bump it up to e.g. 10MB or even higher.

# buffer = 10485760

}

7.修改配置文件/etc/selinux/config

sudo vim /etc/selinux/config

修改为：

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - No SELinux policy is loaded.

SELINUX=disabled

# SELINUXTYPE= can take one of these two values:

# targeted - Targeted processes are protected,

# mls - Multi Level Security protection.

SELINUXTYPE=targeted

提示 ：selinux 本次生效关闭必须重启，如果此时不想重启，可以临时生效之：

 

[atguigu@hadoop102 flume]$ sudo setenforce 0

8.启动 ganglia 或停止

[atguigu@hadoop102 flume]$ sudo service httpd start
[atguigu@hadoop102 flume]$ sudo service gmetad start
[atguigu@hadoop102 flume]$ sudo service gmond start

=======================================================
[atguigu@hadoop102 flume]$ sudo service httpd stop
[atguigu@hadoop102 flume]$ sudo service gmetad stop
[atguigu@hadoop102 flume]$ sudo service gmond stop

9.打开网页浏览 ganglia 页面

http:// 192.168.10.102 /ganglia

 

提示： 如果完成以上操作依然出现权限不足错误，请修改/var/lib/ganglia 目录的权限：

 

[atguigu@hadoop102 flume]$ sudo chmod -R 777 /var/lib/ganglia

 

10.操作和配置 Flume 测试监控

10.1.修改/opt/module/flume/conf 目录下的 flume-env.sh 配置：

JAVA_OPTS="-Dflume.monitoring.type=ganglia -Dflume.monitoring.hosts= 192.168.10.102 :8649 -Xms100m -Xmx200m"

 

10.2. 启动 Flume 任务

[atguigu@hadoop102 flume]$ bin/flume-ng agent \

--conf conf/ \

--name a1 \

--conf-file job/flume-netcat-logger.conf \

-Dflume.root.logger==INFO,console \

-Dflume.monitoring.type=ganglia \

-Dflume.monitoring.hosts= 192.168.10.102 :8649

 

10.3.发送数据观察 ganglia 监测图

[atguigu@hadoop102 flume]$ nc localhost 44444 #根据自己设置的端口号