OpenTSDB单机部署
| 环境 | 版本 |
|---|---|
| java | OracleJDK （jdk1.8.0_271） |
| ZooKeeper | zookeeper-3.4.11 |
| hbase | hbase-2.3.3 |
| opentsdb | opentsdb-2.4.0 |
Java环境部署
Oracle与OpenJDK之间的区别，本次安装Oracle JDK。
登录Oracle官网，手动下载jdk-8u271-linux-x64.tar.gz（不登录无法下载）
mkdir -p /usr/java
tar zvxf jdk-8u271-linux-x64.tar.gz -C /usr/java
# 设置JAVA_HOME
vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_271
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
#使环境变量生效
source /etc/profile
#查看配置结果
echo $JAVA_HOME
Zookeeper环境部署
wget http://archive.apache.org/dist/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz
tar -zvxf zookeeper-3.4.11.tar.gz
cd zookeeper-3.4.11/
#创建数据目录
mkdir data
mkdir logs
cd conf/
#新建配置文件
vi zoo.cfg
zoo.cfg的内容如下：
tickTime=2000
dataDir=/usr/single_hbase/zookeeper-3.4.11/data
dataLogDir=/usr/single_hbase/zookeeper-3.4.11/logs
clientPort=3181
cd ../bin/
#启动
./zkServer.sh start
Hbase环境部署
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/stable/hbase-2.3.3-bin.tar.gz
tar -zxf hbase-2.3.3-bin.tar.gz
cd hbase-2.3.3/
#创建数据目录
mkdir data
查看设置的JAVA_HOME
echo $JAVA_HOME
# ====== output info ======
/usr/java/jdk1.8.0_271
修改hbase-env.sh
vi conf/hbase-env.sh
# 修改JAVA_HOME选项为输入路径
export JAVA_HOME=/usr/java/jdk1.8.0_271
注意：如果使用自己安装的zookeeper，再修改
hbase-env.sh的以下选项，如果不是直接略过export HBASE_MANAGES_ZK=false修改
hbase-env.cmd最后一行vi hbase-env.cmd @rem set HBASE_MANAGES_ZK=false
修改hbase-site.xml
vi conf/hbase-site.xml
# 参考内容如下
hbase.rootdir
file:///usr/single_hbase/hbase-2.3.3/data
hbase.cluster.distributed
true
hbase.unsafe.stream.capability.enforce
false
hbase.tmp.dir
usr/single_hbase/hbase-2.3.3/tmp
hbase.zookeeper.property.clientPort
3181
hbase.zookeeper.quorum
localhost
hbase.zookeeper.property.dataDir
/usr/single_hbase/zookeeper-3.4.11/data
hbase.procedure.store.wal.use.hsync
true
解决ERROR [main] master.HMasterCommandLine: Master exiting错误：
配置文件要有，其实刚下载完hbase是有这个选项的，后来我根据网上的教程把这个选项删除了，所以最好不要动原始配置，需要什么就直接添加
hbase.unsafe.stream.capability.enforce false
# 启动hbase
./bin/start-hbase.sh
输入jps，可以看到HMaster，HRegionServer等信息。QuorumPeerMain是zookeeper的集群启动的入口类，我有两个zookeeper集群，所以会显示两个
1140 QuorumPeerMain
9157 QuorumPeerMain
7286 HMaster
10057 Jps
13294 Kafka
7519 HRegionServer
OpenTSDB环境部署
# 安装Gnuplot
apt install gnuplot
wget https://github.com/OpenTSDB/opentsdb/releases/download/v2.4.0/opentsdb-2.4.0.tar.gz
tar -xvzf opentsdb-2.4.0.tar.gz
# 此步随意
mv opentsdb-2.4.0/ ./hbase-2.3.3/
# 此步必须
cp -r third_party ./build
./build.sh
# 编辑配置文件
vi src/opentsdb.conf
#============================== 参考配置如下 =============================
# --------- NETWORK ----------
# # The TCP port TSD should use for communications
# # *** REQUIRED ***
tsd.network.port = 4242
#
# # The IPv4 network address to bind to, defaults to all addresses
# # tsd.network.bind = 0.0.0.0
#
# # Disable Nagel's algorithm, default is True
# #tsd.network.tcp_no_delay = true
#
# # Determines whether or not to send keepalive packets to peers, default
# # is True
# #tsd.network.keep_alive = true
#
# # Determines if the same socket should be used for new connections, default
# # is True
# #tsd.network.reuse_address = true
#
# # Number of worker threads dedicated to Netty, defaults to # of CPUs * 2
# #tsd.network.worker_threads = 8
#
# # Whether or not to use NIO or tradditional blocking IO, defaults to True
# #tsd.network.async_io = true
# #
#
# # ----------- HTTP -----------
# # The location of static files for the HTTP GUI interface.
# # *** REQUIRED ***
tsd.http.staticroot = /usr/single_hbase/hbase-2.3.3/opentsdb-2.4.0/build/staticroot
#
tsd.http.request.enable_chunked = true
tsd.http.request.max_chunk = 102400000
# # Where TSD should write it's cache files to
# # *** REQUIRED ***
tsd.http.cachedir = /tmp/opentsdb
# #sd.core.plugin_path = /usr/share/opentsdb/plugins
# #tsd.storage.hbase.data_table = tsdb
# # --------- CORE ----------
# # Whether or not to automatically create UIDs for new metric types, default
# # is False
tsd.core.auto_create_metrics = true
tsd.storage.fix_duplicates = true
#
# # Whether or not to enable the built-in UI Rpc Plugins, default
# # is True
# #tsd.core.enable_ui = true
tsd.query.filter.expansion_limit = 65535
# # Whether or not to enable the built-in API Rpc Plugins, default
# # is True
# #tsd.core.enable_api = true
#
# # --------- STORAGE ----------
# # Whether or not to enable data compaction in HBase, default is True
tsd.storage.enable_compaction = false
#
# # How often, in milliseconds, to flush the data point queue to storage,
# # default is 1,000
# # tsd.storage.flush_interval = 1000
#
# # Max number of rows to be returned per Scanner round trip
# # tsd.storage.hbase.scanner.maxNumRows = 128
#
# # Name of the HBase table where data points are stored, default is "tsdb"
# tsd.storage.hbase.data_table = tsdb
#
# # Name of the HBase table where UID information is stored, default is "tsdb-uid"
# #tsd.storage.hbase.uid_table = tsdb-uid
#
# # Path under which the znode for the -ROOT- region is located, default is "/hbase"
# #tsd.storage.hbase.zk_basedir = /hbase
#
# # A comma separated list of Zookeeper hosts to connect to, with or without
# # port specifiers, default is "localhost"
tsd.storage.hbase.zk_quorum = 127.0.0.1:3181
#
# # --------- COMPACTIONS ---------------------------------
# # Frequency at which compaction thread wakes up to flush stuff in seconds, default 10
# # tsd.storage.compaction.flush_interval = 10
#
# # Minimum rows attempted to compact at once, default 100
# # tsd.storage.compaction.min_flush_threshold = 100
#
# # Maximum number of rows, compacted concirrently, default 10000
# # tsd.storage.compaction.max_concurrent_flushes = 10000
#
# # Compaction flush speed multiplier, default 2
# # tsd.storage.compaction.flush_speed = 2
#
解决opentsdb执行./build.sh报错：
cp -r third_party ./build
# 执行脚本,在hbase中创建opentsdb相关表，根目录下执行
env COMPRESSION=NONE HBASE_HOME=/usr/single_hbase/hbase-2.3.3/ ./src/create_table.sh
进入hbase根目录，执行bin/hbase shell ，再执行list，可以看到有四张表
hbase(main):001:0> list
TABLE
tsdb
tsdb-meta
tsdb-tree
tsdb-uid
4 row(s)
Took 0.4887 seconds
=> ["tsdb", "tsdb-meta", "tsdb-tree", "tsdb-uid"]
hbase(main):002:0>
进入opentsdb根目录，运行以下命令启动TSD
./build/tsdb tsd --config=./src/opentsdb.conf