scribe安装记录

折腾了快2天,终于装上了,内流满面

scribe安装较复杂,官网无详细安装手册,所以自己装成功了还是记录下吧~

1.系统版本:基于CentOS-6.3-x86_64

2.gcc
   安装版本为4.4.6 官方要求>=3.3.5,但低于4会有各种问题
    g++ 如果g++的结果是no input files表示已安装,不是则安装:
   
yum -y install gcc+  gcc-c++

   

3.flex
   确认是否安装,无则:
  
yum -y install flex


4.pkgconfig
   确认是否安装,无则:
  
yum -y install pkgconfig


5.m4
   确认是否安装,无则:
  
yum -y install m4


6.python
   安装版本为2.6.6
  
   yum -y install python.x86_64
   yum -y install python-devel.x86_64
   


7.ruby
   安装版本为1.8.7
  
   yum -y install ruby.x86_64
   yum -y install ruby-devel.x86_64
   


8.libevent
   安装版本为1.4.13
  
   yum -y install libevent.x86_64 
   yum -y install libevent-devel.x86_64
   


9.openssl-devel
   安装版本为1.0.0
  
   yum -y install openssl.x86_64 
   yum -y install openssl-devel.x86_64
   


10.bz2-devel
   安装版本为1.0.5
  
   yum -y install openssl.x86_64 
   yum -y install openssl-devel.x86_64
   


11.zlib-devel
   安装版本为1.2.3
  
   yum -y install zlib.x86_64 
   yum -y install zlib-devel.x86_64
   


12.bison
   安装版本为2.4.1
  
   yum -y install bison.x86_64 
   yum -y install bison-devel.x86_64
   


13.automake
   安装版本为2.69   thrift要求>=2.65
  
   wget ftp://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz  
   tar zxvf autoconf-2.69.tar.gz -C /usr/local/
   cd /usr/local/autoconf-2.69/
   ./configure --prefix=/usr
   make
   make install
   


14.autoconf
   安装版本为1.10   thrift安装警告,要求>=1.9
  
   wget http://ftp.gnu.org/gnu/automake/automake-1.10.tar.gz 
   tar zxvf automake-1.10.tar.gz -C /usr/local/
   cd /usr/local/automake-1.10
   ./configure --prefix=/usr
   make
   make install
   


15.libtool
   安装版本为2.24
  
   wget http://ftp.gnu.org/gnu/libtool/libtool-2.2.4.tar.gz 
   tar zxvf automake-1.10.tar.gz -C /usr/local/
   cd /usr/local/automake-1.10
   ./configure --prefix=/usr
   make
   make install
   


16.安装boost
  
   安装版本为1.45.0 官方版本要求>=1.36
   确认系统中目前没有别的boost版本,有必须清楚干净,否则多boost会有各种问题。 
  
   
    wget http://nchc.dl.sourceforge.net/project/boost/boost/1.45.0/boost_1_45_0.tar.gz
   tar zxvf boost_1_45_0.tar.gz -C /usr/local/ 
   cd /usr/local/boost_1_45_0
   ./bootstrap.sh
   ./bjam install
   

   在/etc/profile中加入:
    
     export BOOST_ROOT=/usr/local
    export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib
    source /etc/profile
    


17.thrift + fb303
   安装版本为0.7.0  官网要求>=0.5.0
  
 
   wget http://archive.apache.org/dist/thrift/0.7.0/thrift-0.7.0.tar.gz 
   tar zxvf thrift-0.7.0.tar.gz -C /usr/local/
   cd /usr/local/thrift-0.7.0
   chmod +x configure
   ./configure
   make
   make install
   

  
   开始安装fb303 cd contrib/fb303
  
 
   chmod +x bootstrap.sh
   ./bootstrap.sh
   ./configure
   make
   make install
   


18.scribe
   下载地址https://github.com/facebook/scribe/downloads,下载当前版本63e4824,(版本2.2安装失败)
  
 
   tar zxvf facebook-scribe-63e4824.tar.gz -C /usr/local/ 
   cd facebook-scribe-63e4824
   ./bootstrap.sh
   ./configure --prefix=/usr/local/scribe
   make
   make install
   



检验安装是否成功(纯转):


1、配置scribe

首先我的scribe装在/usr/local/scribe中,在该目录/usr/local/scribe下,

mkdir conf

然后将facebook-scribe下的examples/example1.conf拷到conf中

2、启动scribe

cd /usr/local/scribe

bin/scribed -c conf/example1.conf

出现如下提示,启动成功:

[Wed Nov  9 10:35:52 2011] "STATUS: STARTING"
[Wed Nov  9 10:35:52 2011] "STATUS: configuring"
[Wed Nov  9 10:35:52 2011] "got configuration data from file <conf/example1.conf>"
[Wed Nov  9 10:35:52 2011] "CATEGORY : default"
[Wed Nov  9 10:35:52 2011] "Creating default store"
[Wed Nov  9 10:35:52 2011] "configured <1> stores"
[Wed Nov  9 10:35:52 2011] "STATUS: "
[Wed Nov  9 10:35:52 2011] "STATUS: ALIVE"
[Wed Nov  9 10:35:52 2011] "Starting scribe server on port 1463"
Thrift: Wed Nov  9 10:35:52 2011 libevent 1.4.13-stable method epoll

3、验证scribe能正常工作

examples文件夹下有Readme,按照这个操作就行,简单来说

先启动scribe:

scribe安装路径/bin/scribed -c scribe解压路径/examples/example1.conf

出现提示信息:

[Wed Dec 21 17:12:01 2011] "STATUS: STARTING"

[Wed Dec 21 17:12:01 2011] "STATUS: configuring"

[Wed Dec 21 17:12:01 2011] "got configuration data from file <conf/example1.conf>"

[Wed Dec 21 17:12:01 2011] "CATEGORY : default"

[Wed Dec 21 17:12:01 2011] "Creating default store"

[Wed Dec 21 17:12:01 2011] "configured <1> stores"

[Wed Dec 21 17:12:01 2011] "STATUS: "

[Wed Dec 21 17:12:01 2011] "STATUS: ALIVE"

[Wed Dec 21 17:12:01 2011] "Starting scribe server on port 1463"

然后在examples目录下,发个数据给scribe

echo "hello world" | ./scribe_cat test

发现scribe出现提示信息:


[Wed Dec 21 17:14:30 2011] "[test] Creating new category store from model default"

[Wed Dec 21 17:14:30 2011] "store thread starting"

[Wed Dec 21 17:14:30 2011] "[test] Opened file </tmp/scribetest/test/test_00000> for writing"

[Wed Dec 21 17:14:30 2011] "[test] Opened file </tmp/test/test_00000> for writing"

[Wed Dec 21 17:14:30 2011] "[test] Changing state from <DISCONNECTED> to <SENDING_BUFFER>"

[Wed Dec 21 17:14:30 2011] "[test] read <0> entries of <0> bytes from file </tmp/test/test_00000>"

[Wed Dec 21 17:14:30 2011] "[test] No more buffer files to send, switching to streaming mode"

[Wed Dec 21 17:14:30 2011] "[test] Changing state from <SENDING_BUFFER> to <STREAMING>"


查看log

cat /tmp/scribetest/test/test_00000

出现:

hello world

表示安装成功。


4、用rb验证scribe能正常工作(这一步并不是必须的。3验证成功后scribe就已经正常工作了。)

cd /usr/local/scribe

mkdir testapp

cd testapp

/usr/local/bin/thrift -o . -I /usr/local/share/ --gen rb /opt/apps_install/scribe/facebook-scribe-6600084/if/scribe.thrift
/usr/local/bin/thrift -o . -I /usr/local/share/ --gen rb /usr/local/share/fb303/if/fb303.thrift

mv get_rb scribe

此时/usr/local/scribe/testapp/scribe下有6个文件

-rw-r--r-- 1 root root 15879 Nov  9 10:17 facebook_service.rb
-rw-r--r-- 1 root root   140 Nov  9 10:17 fb303_constants.rb
-rw-r--r-- 1 root root   414 Nov  9 10:17 fb303_types.rb
-rw-r--r-- 1 root root   141 Nov  9 10:17 scribe_constants.rb
-rw-r--r-- 1 root root  1943 Nov  9 10:17 scribe.rb
-rw-r--r-- 1 root root   643 Nov  9 10:17 scribe_types.rb


然后再testapp中新建一个test.rb文件,内容是

#!/usr/bin/env ruby

$LOAD_PATH.unshift(File.dirname(__FILE__) + '/scribe')
require 'scribe'

begin
   socket = Thrift::Socket.new('localhost', 1463)
   transport = Thrift::FramedTransport.new(socket)
   protocol = Thrift::BinaryProtocol.new(transport, false)
   client = Scribe::Client.new(protocol)
   transport.open()
   log_entry = LogEntry.new(:category => 'test', :message => 'This is a test message')
   client.Log([log_entry])
   transport.close()
rescue Thrift::Exception => tx
   print 'Thrift::Exception: ', tx.message, "\n"
end
修改test.rb的执行权限,chmod +x test.rb
执行test.rb,./test.rb
scribe出现提示信息:
[Wed Nov  9 10:35:58 2011] "[test] Creating new category store from model default"
[Wed Nov  9 10:35:58 2011] "store thread starting"
[Wed Nov  9 10:35:58 2011] "[test] Opened file </tmp/scribetest/test/test_00000> for writing"
[Wed Nov  9 10:35:58 2011] "[test] Opened file </tmp/test/test_00000> for writing"
[Wed Nov  9 10:35:58 2011] "[test] Changing state from <DISCONNECTED> to <SENDING_BUFFER>"
[Wed Nov  9 10:35:58 2011] "[test] read <0> entries of <0> bytes from file </tmp/test/test_00000>"
[Wed Nov  9 10:35:58 2011] "[test] No more buffer files to send, switching to streaming mode"
[Wed Nov  9 10:35:58 2011] "[test] Changing state from <SENDING_BUFFER> to <STREAMING>"

查看/tmp/scribetest/test/test_00000,它的内容是:
This is a test message
表示scribe工作正常啦~~


对这位博主表示无与伦比的感谢:
http://abentotoro.blog.sohu.com/190515962.html


你可能感兴趣的:(hadoop,Facebook,scribe,日志收集)