安装HBase
In order to use OpenTSDB, you need to have HBase up and running. This page will help you get started with a simple, single-node HBase setup, which is good enough to evaluate OpenTSDB or monitor small installations. If you need scalability and reliability, you will need to setup a full HBase cluster.
要使用OpenTSDB,你需要去让HBase跑起来。这个页面将帮助你以一种简单的,单结点的方式安装HBase,这是足够好的去支撑OpenTSDB或监控小装置。如果你需要可扩展性和可靠性,你将需要去安装一个完整的HBase群。
You can copy-paste all the following instructions directly into a terminal.
以下所有指令均可直接复制粘贴到终端下。
安装一个单结点的HBase例子
If you already have an HBase cluster, skip this step. If you're gonna be using less than 5-10 nodes, stick to a single node. Deploying HBase on a single node is easy and can help get you started with OpenTSDB quickly. You can always scale to a real cluster and migrate your data later.
如果你想直接安装一个HBase群,跳过这一步。如果你要使用低于5至10个节点,那么坚持使用单结点。在单结点上部署HBase很简单,并且能帮你快速开启OpenTSDB。你总能扩展到一个真正的集群,然后迁移你的数据。
wget http://www.apache.org/dist/hbase/hbase-0.92.1/hbase-0.92.1.tar.gz
tar xfz hbase-0.92.1.tar.gz
cd hbase-0.92.1
At this point, you are ready to start HBase (without HDFS) on a single node. But before starting it, I recommend using the following configuration:
此时,你将准备在单个结点上开启HBase(没有HDFS) 。不过在开始之前,我推荐先用以下配置:
hbase_rootdir=${TMPDIR-'/tmp'}/tsdhbase
iface=lo`uname | sed -n s/Darwin/0/p`
cat >conf/hbase-site.xml < hbase.rootdir file:///$hbase_rootdir/hbase-\${user.name}/hbase hbase.zookeeper.dns.interface $iface hbase.regionserver.dns.interface $iface hbase.master.dns.interface $iface EOF Make sure to adjust the value of 如果你希望HBase将其数据存储在一个比临时目录更耐用的地方,那么认真设置hbase_rootdir的值。默认使用/tmp,意味着无论何时你重启你的系统,你将丢失你所有的数据。随后的设置不是很重要,只是简单的使HBase持续监控端口( Mac OS X是lo0,Linux是lo),当你只是想在单个节点上测试HBase时简化了操作。 Now start HBase: 现在开启HBase: ./bin/start-hbase.sh 使用LZO There is no reason to not use LZO with HBase. Except in rare cases, the CPU cycles spent on doing LZO compression / decompression pay for themselves by saving you time wasted doing more I/O. This is certainly true for OpenTSDB where LZO can easily compress OpenTSDB's binary data by 3 to 4x. Installing LZO is simple and is done as follows. 使用HBase,没有理由不使用LZO。除了极少数情况下,CPU周期为它们自己花时间做LZO压缩/解压缩,以节省您浪费在做更多的I / O的时间。对于OpenTSDB,可以肯定的是LZO可以很容易地压缩OpenTSDB的二进制数据3至4倍。安装LZO是简单的,如下: 先决条件 In order to build 要连接hadoop-lzo,你需要安装Ant和libzo2的开发头文件: apt-get install ant liblzo2-dev # Debian/Ubuntu yum install ant ant-nodeps lzo-devel.x86_64 # RedHat/CentOS/Fedora brew install lzo # Mac OS X 编译和配置 Thanks to our friends at Cloudera for maintaining the Hadoop-LZO package: git clone git://github.com/cloudera/hadoop-lzo.git cd hadoop-lzo CLASSPATH=path/to/hadoop-core-1.0.3.jar CFLAGS=-m64 CXXFLAGS=-m64 ant compile-native tar hbasedir=path/to/hbase mkdir -p $hbasedir/lib/native cp build/hadoop-lzo-0.4.14/hadoop-lzo-0.4.14.jar $hbasedir/lib cp -a build/hadoop-lzo-0.4.14/lib/native/* $hbasedir/lib/native Restart HBase and make sure you create your tables with 重启HBase,确认你用 Common gotchas: 常见的错误: Where to find 去哪儿找到 On Mac OS X, you may get 在 Mac OS X,你可能会得到 On RedHat/CentOS/Fedora you may have to specify where Java is, by adding 在 RedHat/CentOS/Fedora你可能需要去确定java在哪,可以在ant命令行,CLASSPATH之 前加入: On RedHat/CentOS/Fedora, if you get the weird error message that "Cause: the class org.apache.tools.ant.taskdefs.optional.Javah was not found." then you need to install the 在 RedHat/CentOS/Fedora,如果你得到了奇怪的错误信息:Cause: the class org.apache.tools.ant.taskdefs.optional.Javah was not found,那你需要去安装 迁移至一个真正的HBase群 TBD. In short: 简要TBD: Shut down all your TSDs. 关闭所有的TSD Shut down your single-node HBase cluster. 关闭你单结点的HBase群 Copy the directories named 从你的本地主机文件系统复制tsdb和tsdb-uid目录至HDFS群,以备份你的真实HBase群。 Run Restart your real HBase cluster (sorry). 重启你的真实HBase群。 Restart your TSDs after making sure they now use your real HBase cluster. 确认TSD正在使用你的真实HBase群后,重启TSD。 将HBase付诸生产 TBD. In short: 简要TBD: Stay on a single node unless you can deploy HBase on at least 5 machines, preferably at least 10. 保持一个单结点除非你能在至少5台机器,最好至少10台机器上部署HBase。 Make sure you have LZO installed and make sure it's enabled for the tables used by OpenTSDB. 确认你已经安装LZO,并且确认它已经能够用于被OpenTSDB使用的表。 TBD... TBD。。。hbase_rootdir
if you want HBase to store its data in somewhere more durable than a temporary directory. The default is to use /tmp
, which means you'll lose all your data whenever your server reboots. The remaining settings are less important and simply force HBase tostick to the loopback interface (lo0
on Mac OS X, or just lo
on Linux), which simplifies things when you're just testing HBase on a single node.Using LZO
Pre-requisites
hadoop-lzo
, you need to have Ant installed as well as liblzo2 with development headers:Compile & Deploy
COMPRESSION => 'LZO'
COMPRESSION => 'LZO'
创建了表。
hadoop-core-1.0.3.jar
? On a normal, production HBase install, it will be under HBase's lib/
directory. In your development environment it may be stashed under HBase's target/
directory, use find
to locate it.hadoop-core-1.0.3.jar
?通常,HBase安装后,它会放在HBase的lib/目录下。在你的开发环境下,它可能被放置在HBase的target/目录下,用find命令去定位它吧。error: Native java headers not found. Is $JAVA_HOME set correctly?
when configure
is looking for jni.h
, in which case you need to insert CPPFLAGS=-I/System/Library/Frameworks/JavaVM.framework/Versions/Current/Headers
before CLASSPATH
on the 3rd command above (the one that invokes ant
).error:Native java headers not found. Is $JAVA_HOME set correctly?
当
configure
正在查找
jni.h
的情况下,你需要在
CLASSPAT
之前第三个命令(调用
ant
的命令)插入
correctly? when configure is looking for jni.h, in which case you need to insert CPPFLAGS=-I/System/Library/Frameworks/JavaVM.framework/Versions/Current/Headers
。
JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
(or similar) to the ant
command-line, before the CLASSPATH
.JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64
(或相似
的)。
ant-nodeps
package.ant-nodeps
包。
Migrating to a real HBase cluster
tsdb
and tsdb-uid
from your local filesystem to the HDFS cluster backing up your real HBase cluster../bin/hbase org.jruby.Main ./bin/add_table.rb
/hdfs/path/to/hbase/
tsdb
and again for the tsdb-uid
directory.跑命令:
./bin/hbase org.jruby.Main ./bin/add_table.rb
/hdfs/path/to/hbase/
tsdb
然后相同的方式跑tsdb-uid。Putting HBase in production