大数据入门--HBase（一）环境搭建&常见命令
环境搭建&常见命令
- 概念
-
- 架构角色
- 集群环境搭建
-
- 准备工作
- 安装
-
- 1. 解压安装包
- 2.修改配置文件
- 停起服务
- 常见命令
-
- 相关概念
- DDL
-
- Namespace
- Table
- DML
概念
架构角色
| 角色 | 作用 |
|---|---|
| Master | 是 HBase 集群中的主服务器，负责监控集群中的所有 RegionServer，并且是所有元数据更改的接口。通过zookeeper实现高可用。 Table : 创建（create）、删除（delete）、更新（alter） RegionServer : 分配regions到每个RegionServer，监控每个RegionServer的状态 |
| Region Server | RegionServer 主要负责响应用户的请求，向 HDFS 读写数据。 Data : 获取（get）、添加（put）、更新（put）、删除（delete） Region : 切分Region（splitRegion），合并Region（compactRegion） |
| zookeeper | HBase 通过 Zookeeper 来做 Master 的高可用、RegionServer 的监控、元数据的入口以及集群配置的维护等工作。 |
| HDFS | 为 HBase 提供最终的底层数据存储服务，同时为 HBase 提供高可用的支持。 |
集群环境搭建
准备工作
- Hbase依赖zookeeper，所以需要先搭建zookeeper，参考此文
- Hbase对时间同步要求较高，建议使用ntp做时间同步，参考此文：ntp服务安装
安装
当前安装版本为：hbase-1.3.1
1. 解压安装包
tar -zxvf hbase-1.3.1-bin.tar.gz -C /opt/module/
cd /opt/module/hbase-1.3.1
2.修改配置文件
hbase-env.sh
添加
export JAVA_HOME=/opt/module/jdk1.8.0_144
# 不使用自带zk
export HBASE_MANAGES_ZK=false
hbase-site.xml
<property>
<name>hbase.rootdirname>
<value>hdfs://hadoop101:8020/HBasevalue>
property>
<property>
<name>hbase.cluster.distributedname>
<value>truevalue>
property>
<property>
<name>hbase.master.portname>
<value>60000value>
property>
<property>
<name>hbase.zookeeper.quorumname>
<value>hadoop101:2181,hadoop102:2181,hadoop103:2181value>
property>
<property>
<name>hbase.zookeeper.property.dataDirname>
<value>/opt/module/zookeeper-3.4.5/zkDatavalue>
property>
regionserver
hadoop101
hadoop102
hadoop103
软连接 hadoop 配置文件到 HBase
ln -s /opt/module/hadoop-3.1.3/etc/hadoop/core-site.xml conf/core-site.xml
ln -s /opt/module/hadoop-3.1.3/etc/hadoop/hdfs-site.xml conf/hdfs-site.x
分布Hbase安装文件
xsync /opt/module/hbase-1.3.1
停起服务
单节点停起master
bin/hbase-daemon.sh start master
bin/hbase-daemon.sh stop master
检查是否启动成功,可以使用jps,出现如下进程证明master/regionserver是启动成功的
jps
---
3588 HMaster
3855 HRegionServer
单节点停起regionserver
bin/hbase-daemon.sh start regionserver
bin/hbase-daemon.sh stop regionserver
查看Hbase管理页面，网页地址：http://hadoop101:16010/
集群停起服务
# 执行start-hbase.sh的主机为master节点，同时根据conf/regionservers配置启动regionserver
bin/start-hbase.sh
bin/stop-hbase.sh
在浏览一下管理页面，三个节点都展示就OK了
常见命令
温馨提示
hbase命令行删除时backspace向后删除，delete向前删除，与平时习惯相反。
帮助
help
help ‘COMMAND’
hbase(main):001:0> help
HBase Shell, version 1.3.1, r930b9a55528fe45d8edce7af42fef2d35e77677a, Thu Apr 6 19:36:54 PDT 2017
Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command.
Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.
COMMAND GROUPS:
Group name: general
Commands: status, table_help, version, whoami
Group name: ddl
Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, get_table, is_disabled, is_enabled, list, locate_region, show_filters
Group name: namespace
Commands: alter_namespace, create_namespace, describe_namespace, drop_namespace, list_namespace, list_namespace_tables
Group name: dml
Commands: append, count, delete, deleteall, get, get_counter, get_splits, incr, put, scan, truncate, truncate_preserve
Group name: tools
Commands: assign, balance_switch, balancer, balancer_enabled, catalogjanitor_enabled, catalogjanitor_run, catalogjanitor_switch, close_region, compact, compact_rs, flush, major_compact, merge_region, move, normalize, normalizer_enabled, normalizer_switch, split, splitormerge_enabled, splitormerge_switch, trace, unassign, wal_roll, zk_dump
Group name: replication
Commands: add_peer, append_peer_tableCFs, disable_peer, disable_table_replication, enable_peer, enable_table_replication, get_peer_config, list_peer_configs, list_peers, list_replicated_tables, remove_peer, remove_peer_tableCFs, set_peer_tableCFs, show_peer_tableCFs
Group name: snapshots
Commands: clone_snapshot, delete_all_snapshot, delete_snapshot, delete_table_snapshots, list_snapshots, list_table_snapshots, restore_snapshot, snapshot
Group name: configuration
Commands: update_all_config, update_config
Group name: quotas
Commands: list_quotas, set_quota
Group name: security
Commands: grant, list_security_capabilities, revoke, user_permission
Group name: procedures
Commands: abort_procedure, list_procedures
Group name: visibility labels
Commands: add_labels, clear_auths, get_auths, list_labels, set_auths, set_visibility
SHELL USAGE:
Quote all names in HBase Shell such as table and column names. Commas delimit
command parameters. Type <RETURN> after entering a command to run it.
Dictionaries of configuration used in the creation and alteration of tables are
Ruby Hashes. They look like this:
{'key1' => 'value1', 'key2' => 'value2', ...}
and are opened and closed with curley-braces. Key/values are delimited by the
'=>' character combination. Usually keys are predefined constants such as
NAME, VERSIONS, COMPRESSION, etc. Constants do not need to be quoted. Type
'Object.constants' to see a (messy) list of all constants in the environment.
If you are using binary keys or values and need to enter them in the shell, use
double-quote'd hexadecimal representation. For example:
hbase> get 't1', "key\x03\x3f\xcd"
hbase> get 't1', "key\003\023\011"
hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"
The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added.
For more on the HBase Shell, see http://hbase.apache.org/book.html
相关概念
- namespace：名称空间
- table：表
- Column Family ：列族
- Column：列
连接hbase
bin/hbase shell
DDL
Namespace
查看NS
hbase(main):001:0> list_namespace
NAMESPACE
default
hbase
2 row(s) in 0.1660 seconds
创建NS
create_namespace 'study01'
删除NS
名称空间内不能有表，才允许删除NS
drop_namespace 'study01'
Table
建表
create 'study:test1', 'info1','info2'
查看表
list_namespace_tables 'study'
删除表：先禁用后删除
disable 'study01:tab01'
drop 'study01:tab01'
DML
插入/更新数据
hbase(main):012:0> put 'study:test1','0001','info1:name','zhangsan'