hbase开发，hbase表操作及其java api实现
开发环境：
hadoop: hadoop-1.1.2
hbase: hbase-0.94.11-security
eclipse:Juno Service Release 2
配置Eclipse
通过 Eclipse 创建一个新 Java 工程，右击项目根目录，选择“Properties> Java Build Path> Library> Add External JARs”，将 HBase 安装文件解压后根目录下的hbase-0.94.11-security.jar、hbase-0.94.11-security-tests.jar 和 lib 子目录下所有的 jar 包添加到本工程的 Build Path下。
示例使用表users及表的基本操作：
创建表
>create 'users','user_id','address','info'
添加记录
put 'users','xiaoming','info:age','24'; put 'users','xiaoming','info:birthday','1987-06-17'; put 'users','xiaoming','info:company','alibaba'; put 'users','xiaoming','address:contry','china'; put 'users','xiaoming','address:province','zhejiang'; put 'users','xiaoming','address:city','hangzhou'; put 'users','zhangyifei','info:birthday','1987-4-17'; put 'users','zhangyifei','info:favorite','movie'; put 'users','zhangyifei','info:company','alibaba'; put 'users','zhangyifei','address:contry','china'; put 'users','zhangyifei','address:province','guangdong'; put 'users','zhangyifei','address:city','jieyang'; put 'users','zhangyifei','address:town','xianqiao'注意：最后一个语句没有分号，此时，按回车后将添加数据到表'users'中。
获取一条记录
1.取得一个id的所有数据>get 'users','xiaoming'2.获取一个id，一个列族的所有数据
>get 'users','xiaoming','info'3.获取一个id，一个列族中一个列的所有数据
>get 'users','xiaoming','info:age'
更新记录
>put 'users','xiaoming','info:age' ,'29' >get 'users','xiaoming','info:age' >put 'users','xiaoming','info:age' ,'30' >get 'users','xiaoming','info:age'
获取单元格数据的版本数据
>get 'users','xiaoming',{COLUMN=>'info:age',VERSIONS=>1}
>get 'users','xiaoming',{COLUMN=>'info:age',VERSIONS=>2}
>get 'users','xiaoming',{COLUMN=>'info:age',VERSIONS=>3}
获取单元格数据的某个版本数据
〉get 'users','xiaoming',{COLUMN=>'info:age',TIMESTAMP=>1364874937056}
删除xiaoming值的'info:age'字段
>delete 'users','xiaoming','info:age' >get 'users','xiaoming'
删除整行
>deleteall 'users','xiaoming'
统计表的行数
>count 'users'
清空表
>truncate 'users'
全表扫描
>scan 'users'
输出结果：
hbase(main):022:0> scan 'users' ROW COLUMN+CELL xiaoming column=address:city, timestamp=1378733106132, value=hangzhou xiaoming column=address:contry, timestamp=1378733106058, value=china xiaoming column=address:province, timestamp=1378733106120, value=zhejiang xiaoming column=info:age, timestamp=1378733105943, value=24 xiaoming column=info:birthday, timestamp=1378733105961, value=1987-06-17 xiaoming column=info:company, timestamp=1378733106006, value=alibaba zhangyifei column=address:city, timestamp=1378733106184, value=jieyang zhangyifei column=address:contry, timestamp=1378733106176, value=china zhangyifei column=address:province, timestamp=1378733106180, value=guangdong zhangyifei column=address:town, timestamp=1378733106189, value=xianqiao zhangyifei column=info:birthday, timestamp=1378733106161, value=1987-4-17 zhangyifei column=info:company, timestamp=1378733106171, value=alibaba zhangyifei column=info:favorite, timestamp=1378733106167, value=movie 2 row(s) in 0.1900 seconds
示例一：输出表“users”的列族名称
代码：
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.util.Bytes;
public class ExampleClient {
public static void main(String[] args) throws IOException {
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "master");//使用eclipse时必须添加这个，否则无法定位
conf.set("hbase.zookeeper.property.clientPort", "2181");
HBaseAdmin admin = new HBaseAdmin(conf);// 新建一个数据库管理员
HTableDescriptor tableDescriptor = admin.getTableDescriptor(Bytes.toBytes("users"));
byte[] name = tableDescriptor.getName();
System.out.println("result:");
System.out.println("table name: "+ new String(name));
HColumnDescriptor[] columnFamilies = tableDescriptor
.getColumnFamilies();
for(HColumnDescriptor d : columnFamilies){
System.out.println("column Families: "+ d.getNameAsString());
}
}
}
编译结果：
2013-09-09 15:58:51,890 WARN conf.Configuration (Configuration.java:<clinit>(195)) - DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:host.name=w253245.ppp.asahi-net.or.jp 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.version=1.6.0_10-rc2 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.vendor=Sun Microsystems Inc. 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.home=C:\Java\jre6 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.class.path=...............//太占篇幅此处省略 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.library.path=D:\workspace\Eclipse-jee\hadoop-1.1.21\lib\native;D:\workspace\Eclipse-jee\hadoop-1.1.21\lib\native 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.io.tmpdir=C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\ 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.compiler=<NA> 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.name=Windows XP 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.arch=x86 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.version=5.1 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.name=hadoop 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.home=C:\Documents and Settings\Administrator 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.dir=D:\workspace\Eclipse-jee\Hadoop_APPs_U_tht 2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (ZooKeeper.java:<init>(438)) - Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=hconnection 2013-09-09 15:58:56,171 INFO zookeeper.RecoverableZooKeeper (RecoverableZooKeeper.java:<init>(104)) - The identifier of this process is 6032@tht 2013-09-09 15:58:56,234 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server master/121.1.253.251:2181. Will not attempt to authenticate using SASL (无法定位登录配置) 2013-09-09 15:58:56,296 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(849)) - Socket connection established to master/121.1.253.251:2181, initiating session 2013-09-09 15:58:56,484 INFO zookeeper.ClientCnxn (ClientCnxn.java:onConnected(1207)) - Session establishment complete on server master/121.1.253.251:2181, sessionid = 0x141011ad7db000e, negotiated timeout = 180000 result: table name: users column Families: address column Families: info column Families: user_id
示例二：使用HBase Java API对表users2进行操作：
代码：
import java.util.ArrayList;
import java.util.List;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
public class OperateTable {
// 声明静态配置
private static Configuration conf = null;
static {
conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum",
"master");
conf.set("hbase.zookeeper.property.clientPort", "2181");
}
// 创建数据库表
public static void createTable(String tableName, String[] columnFamilys)
throws Exception {
// 新建一个数据库管理员
HBaseAdmin hAdmin = new HBaseAdmin(conf);
if (hAdmin.tableExists(tableName)) {
System.out.println("表已经存在");
System.exit(0);
} else {
// 新建一个 scores 表的描述
HTableDescriptor tableDesc = new HTableDescriptor(tableName);
// 在描述里添加列族
for (String columnFamily : columnFamilys) {
tableDesc.addFamily(new HColumnDescriptor(columnFamily));
}
// 根据配置好的描述建表
hAdmin.createTable(tableDesc);
System.out.println("创建表成功");
}
}
// 删除数据库表
public static void deleteTable(String tableName) throws Exception {
// 新建一个数据库管理员
HBaseAdmin hAdmin = new HBaseAdmin(conf);
if (hAdmin.tableExists(tableName)) {
// 关闭一个表
hAdmin.disableTable(tableName);
// 删除一个表
hAdmin.deleteTable(tableName);
System.out.println("删除表成功");
} else {
System.out.println("删除的表不存在");
System.exit(0);
}
}
// 添加一条数据
public static void addRow(String tableName, String row,
String columnFamily, String column, String value) throws Exception {
HTable table = new HTable(conf, tableName);
Put put = new Put(Bytes.toBytes(row));
// 参数出分别：列族、列、值
put.add(Bytes.toBytes(columnFamily), Bytes.toBytes(column),
Bytes.toBytes(value));
table.put(put);
}
// 删除一条数据
public static void delRow(String tableName, String row) throws Exception {
HTable table = new HTable(conf, tableName);
Delete del = new Delete(Bytes.toBytes(row));
table.delete(del);
}
// 删除多条数据
public static void delMultiRows(String tableName, String[] rows)
throws Exception {
HTable table = new HTable(conf, tableName);
List<Delete> list = new ArrayList<Delete>();
for (String row : rows) {
Delete del = new Delete(Bytes.toBytes(row));
list.add(del);
}
table.delete(list);
}
// get row
public static void getRow(String tableName, String row) throws Exception {
HTable table = new HTable(conf, tableName);
Get get = new Get(Bytes.toBytes(row));
Result result = table.get(get);
// 输出结果
for (KeyValue rowKV : result.raw()) {
System.out.print("Row Name: " + new String(rowKV.getRow()) + " ");
System.out.print("Timestamp: " + rowKV.getTimestamp() + " ");
System.out.print("column Family: " + new String(rowKV.getFamily()) + " ");
System.out.print("Row Name: " + new String(rowKV.getQualifier()) + " ");
System.out.println("Value: " + new String(rowKV.getValue()) + " ");
}
}
// get all records
public static void getAllRows(String tableName) throws Exception {
HTable table = new HTable(conf, tableName);
Scan scan = new Scan();
ResultScanner results = table.getScanner(scan);
// 输出结果
for (Result result : results) {
for (KeyValue rowKV : result.raw()) {
System.out.print("Row Name: " + new String(rowKV.getRow()) + " ");
System.out.print("Timestamp: " + rowKV.getTimestamp() + " ");
System.out.print("column Family: " + new String(rowKV.getFamily()) + " ");
System.out
.print("Row Name: " + new String(rowKV.getQualifier()) + " ");
System.out.println("Value: " + new String(rowKV.getValue()) + " ");
}
}
}
// main
public static void main(String[] args) {
try {
String tableName = "users2";
// 第一步：创建数据库表：“users2”
String[] columnFamilys = { "info", "course" };
OperateTable.createTable(tableName, columnFamilys);
// 第二步：向数据表的添加数据
// 添加第一行数据
OperateTable.addRow(tableName, "tht", "info", "age", "20");
OperateTable.addRow(tableName, "tht", "info", "sex", "boy");
OperateTable.addRow(tableName, "tht", "course", "china", "97");
OperateTable.addRow(tableName, "tht", "course", "math", "128");
OperateTable.addRow(tableName, "tht", "course", "english", "85");
// 添加第二行数据
OperateTable.addRow(tableName, "xiaoxue", "info", "age", "19");
OperateTable.addRow(tableName, "xiaoxue", "info", "sex", "boy");
OperateTable.addRow(tableName, "xiaoxue", "course", "china", "90");
OperateTable.addRow(tableName, "xiaoxue", "course", "math", "120");
OperateTable
.addRow(tableName, "xiaoxue", "course", "english", "90");
// 添加第三行数据
OperateTable.addRow(tableName, "qingqing", "info", "age", "18");
OperateTable.addRow(tableName, "qingqing", "info", "sex", "girl");
OperateTable
.addRow(tableName, "qingqing", "course", "china", "100");
OperateTable.addRow(tableName, "qingqing", "course", "math", "100");
OperateTable.addRow(tableName, "qingqing", "course", "english",
"99");
// 第三步：获取一条数据
System.out.println("获取一条数据");
OperateTable.getRow(tableName, "tht");
// 第四步：获取所有数据
System.out.println("获取所有数据");
OperateTable.getAllRows(tableName);
// 第五步：删除一条数据
System.out.println("删除一条数据");
OperateTable.delRow(tableName, "tht");
OperateTable.getAllRows(tableName);
// 第六步：删除多条数据
System.out.println("删除多条数据");
String[] rows = { "xiaoxue", "qingqing" };
OperateTable.delMultiRows(tableName, rows);
OperateTable.getAllRows(tableName);
// 第八步：删除数据库
System.out.println("删除数据库");
OperateTable.deleteTable(tableName);
} catch (Exception err) {
err.printStackTrace();
}
}
}
结果：
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:host.name=tht 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_45 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.home=D:\Java\jre6 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.class.path=。。。。。。。。。。。。 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.library.path=。。。。。。。。。。。 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\ 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 7 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:os.arch=x86 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:os.version=6.1 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\Administrator 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:user.dir=D:\workspace\eclipse-workspace-jee-kepler\hadoop-Apps-tht 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=hconnection 13/09/09 22:01:18 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht 13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置) 13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session 13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000b, negotiated timeout = 180000 13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba8602 13/09/09 22:01:18 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht 13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置) 13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session 13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000c, negotiated timeout = 180000 13/09/09 22:01:20 INFO zookeeper.ZooKeeper: Session: 0x14102d851f8000c closed 13/09/09 22:01:20 INFO zookeeper.ClientCnxn: EventThread shut down 13/09/09 22:01:36 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba8602 13/09/09 22:01:36 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置) 13/09/09 22:01:36 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht 13/09/09 22:01:36 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session 13/09/09 22:01:38 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000d, negotiated timeout = 180000 13/09/09 22:01:38 INFO zookeeper.ZooKeeper: Session: 0x14102d851f8000d closed 13/09/09 22:01:38 INFO zookeeper.ClientCnxn: EventThread shut down 创建表成功 获取一条数据 Row Name: tht Timestamp: 1378735285456 column Family: course Row Name: china Value: 97 Row Name: tht Timestamp: 1378735285918 column Family: course Row Name: english Value: 85 Row Name: tht Timestamp: 1378735285591 column Family: course Row Name: math Value: 128 Row Name: tht Timestamp: 1378735285056 column Family: info Row Name: age Value: 20 Row Name: tht Timestamp: 1378735285368 column Family: info Row Name: sex Value: boy 获取所有数据 Row Name: qingqing Timestamp: 1378735286503 column Family: course Row Name: china Value: 100 Row Name: qingqing Timestamp: 1378735286547 column Family: course Row Name: english Value: 99 Row Name: qingqing Timestamp: 1378735286524 column Family: course Row Name: math Value: 100 Row Name: qingqing Timestamp: 1378735286463 column Family: info Row Name: age Value: 18 Row Name: qingqing Timestamp: 1378735286482 column Family: info Row Name: sex Value: girl Row Name: tht Timestamp: 1378735285456 column Family: course Row Name: china Value: 97 Row Name: tht Timestamp: 1378735285918 column Family: course Row Name: english Value: 85 Row Name: tht Timestamp: 1378735285591 column Family: course Row Name: math Value: 128 Row Name: tht Timestamp: 1378735285056 column Family: info Row Name: age Value: 20 Row Name: tht Timestamp: 1378735285368 column Family: info Row Name: sex Value: boy Row Name: xiaoxue Timestamp: 1378735286268 column Family: course Row Name: china Value: 90 Row Name: xiaoxue Timestamp: 1378735286403 column Family: course Row Name: english Value: 90 Row Name: xiaoxue Timestamp: 1378735286343 column Family: course Row Name: math Value: 120 Row Name: xiaoxue Timestamp: 1378735286114 column Family: info Row Name: age Value: 19 Row Name: xiaoxue Timestamp: 1378735286236 column Family: info Row Name: sex Value: boy 删除一条数据 Row Name: qingqing Timestamp: 1378735286503 column Family: course Row Name: china Value: 100 Row Name: qingqing Timestamp: 1378735286547 column Family: course Row Name: english Value: 99 Row Name: qingqing Timestamp: 1378735286524 column Family: course Row Name: math Value: 100 Row Name: qingqing Timestamp: 1378735286463 column Family: info Row Name: age Value: 18 Row Name: qingqing Timestamp: 1378735286482 column Family: info Row Name: sex Value: girl Row Name: xiaoxue Timestamp: 1378735286268 column Family: course Row Name: china Value: 90 Row Name: xiaoxue Timestamp: 1378735286403 column Family: course Row Name: english Value: 90 Row Name: xiaoxue Timestamp: 1378735286343 column Family: course Row Name: math Value: 120 Row Name: xiaoxue Timestamp: 1378735286114 column Family: info Row Name: age Value: 19 Row Name: xiaoxue Timestamp: 1378735286236 column Family: info Row Name: sex Value: boy 删除多条数据 删除数据库 13/09/09 22:01:40 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba8602 13/09/09 22:01:40 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht 13/09/09 22:01:40 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置) 13/09/09 22:01:40 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session 13/09/09 22:01:42 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000e, negotiated timeout = 180000 13/09/09 22:01:44 INFO zookeeper.ZooKeeper: Session: 0x14102d851f8000e closed 13/09/09 22:01:44 INFO zookeeper.ClientCnxn: EventThread shut down 13/09/09 22:01:48 INFO client.HBaseAdmin: Started disable of users2 13/09/09 22:01:50 INFO client.HBaseAdmin: Disabled users2 13/09/09 22:01:51 INFO client.HBaseAdmin: Deleted users2 删除表成功