hbase开发，hbase表操作及其java api实现
开发环境：
hadoop: hadoop-1.1.2
hbase:hbase-0.94.11-security
eclipse:Juno Service Release 2
配置Eclipse
通过 Eclipse 创建一个新 Java 工程，右击项目根目录，选择“Properties> Java Build Path> Library> Add External JARs”，将 HBase 安装文件解压后根目录下的hbase-0.94.11-security.jar、hbase-0.94.11-security-tests.jar 和 lib 子目录下所有的 jar 包添加到本工程的 Build Path下。
示例使用表users及表的基本操作：
创建表
>create 'users','user_id','address','info'添加记录
put 'users','xiaoming','info:age','24';
put 'users','xiaoming','info:birthday','1987-06-17';
put 'users','xiaoming','info:company','alibaba';
put 'users','xiaoming','address:contry','china';
put 'users','xiaoming','address:province','zhejiang';
put 'users','xiaoming','address:city','hangzhou';
put 'users','zhangyifei','info:birthday','1987-4-17';
put 'users','zhangyifei','info:favorite','movie';
put 'users','zhangyifei','info:company','alibaba';
put 'users','zhangyifei','address:contry','china';
put 'users','zhangyifei','address:province','guangdong';
put 'users','zhangyifei','address:city','jieyang';
put 'users','zhangyifei','address:town','xianqiao'获取一条记录
1.取得一个id的所有数据>get 'users','xiaoming'>get 'users','xiaoming','info'>get 'users','xiaoming','info:age'更新记录
>put 'users','xiaoming','info:age' ,'29'
>get 'users','xiaoming','info:age'
>put 'users','xiaoming','info:age' ,'30'
>get 'users','xiaoming','info:age'获取单元格数据的版本数据
>get 'users','xiaoming',{COLUMN=>'info:age',VERSIONS=>1}
>get 'users','xiaoming',{COLUMN=>'info:age',VERSIONS=>2}
>get 'users','xiaoming',{COLUMN=>'info:age',VERSIONS=>3}获取单元格数据的某个版本数据
〉get 'users','xiaoming',{COLUMN=>'info:age',TIMESTAMP=>1364874937056}删除xiaoming值的'info:age'字段
>delete 'users','xiaoming','info:age'
>get 'users','xiaoming'删除整行
>deleteall 'users','xiaoming'统计表的行数
>count 'users'清空表
>truncate 'users'全表扫描
>scan 'users'输出结果：
hbase(main):022:0> scan 'users'
ROW COLUMN+CELL
xiaoming column=address:city, timestamp=1378733106132, value=hangzhou
xiaoming column=address:contry, timestamp=1378733106058, value=china
xiaoming column=address:province, timestamp=1378733106120, value=zhejiang
xiaoming column=info:age, timestamp=1378733105943, value=24
xiaoming column=info:birthday, timestamp=1378733105961, value=1987-06-17
xiaoming column=info:company, timestamp=1378733106006, value=alibaba
zhangyifei column=address:city, timestamp=1378733106184, value=jieyang
zhangyifei column=address:contry, timestamp=1378733106176, value=china
zhangyifei column=address:province, timestamp=1378733106180, value=guangdong
zhangyifei column=address:town, timestamp=1378733106189, value=xianqiao
zhangyifei column=info:birthday, timestamp=1378733106161, value=1987-4-17
zhangyifei column=info:company, timestamp=1378733106171, value=alibaba
zhangyifei column=info:favorite, timestamp=1378733106167, value=movie
2 row(s) in 0.1900 seconds示例一：输出表“users”的列族名称
代码：
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.util.Bytes;
public class ExampleClient {
public static void main(String[] args) throws IOException {
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "master");//使用eclipse时必须添加这个，否则无法定位
conf.set("hbase.zookeeper.property.clientPort", "2181");
HBaseAdmin admin = new HBaseAdmin(conf);// 新建一个数据库管理员
HTableDescriptor tableDescriptor = admin.getTableDescriptor(Bytes.toBytes("users"));
byte[] name = tableDescriptor.getName();
System.out.println("result:");
System.out.println("table name: "+ new String(name));
HColumnDescriptor[] columnFamilies = tableDescriptor
.getColumnFamilies();
for(HColumnDescriptor d : columnFamilies){
System.out.println("column Families: "+ d.getNameAsString());
}
}
}编译结果：
2013-09-09 15:58:51,890 WARN conf.Configuration (Configuration.java:(195)) - DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:host.name=w253245.ppp.asahi-net.or.jp
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.version=1.6.0_10-rc2
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.vendor=Sun Microsystems Inc.
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.home=C:\Java\jre6
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.class.path=...............//太占篇幅此处省略
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.library.path=D:\workspace\Eclipse-jee\hadoop-1.1.21\lib\native;D:\workspace\Eclipse-jee\hadoop-1.1.21\lib\native
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.io.tmpdir=C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.compiler=
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.name=Windows XP
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.arch=x86
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.version=5.1
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.name=hadoop
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.home=C:\Documents and Settings\Administrator
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.dir=D:\workspace\Eclipse-jee\Hadoop_APPs_U_tht
2013-09-09 15:58:55,031 INFO zookeeper.ZooKeeper (ZooKeeper.java:(438)) - Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=hconnection
2013-09-09 15:58:56,171 INFO zookeeper.RecoverableZooKeeper (RecoverableZooKeeper.java:(104)) - The identifier of this process is 6032@tht
2013-09-09 15:58:56,234 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server master/121.1.253.251:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2013-09-09 15:58:56,296 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(849)) - Socket connection established to master/121.1.253.251:2181, initiating session
2013-09-09 15:58:56,484 INFO zookeeper.ClientCnxn (ClientCnxn.java:onConnected(1207)) - Session establishment complete on server master/121.1.253.251:2181, sessionid = 0x141011ad7db000e, negotiated timeout = 180000
result:
table name: users
column Families: address
column Families: info
column Families: user_id 示例二：使用HBase Java API对表users2进行操作：
代码：
import java.util.ArrayList;
import java.util.List;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
public class OperateTable {
// 声明静态配置
private static Configuration conf = null;
static {
conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum",
"master");
conf.set("hbase.zookeeper.property.clientPort", "2181");
}
// 创建数据库表
public static void createTable(String tableName, String[] columnFamilys)
throws Exception {
// 新建一个数据库管理员
HBaseAdmin hAdmin = new HBaseAdmin(conf);
if (hAdmin.tableExists(tableName)) {
System.out.println("表已经存在");
System.exit(0);
} else {
// 新建一个 scores 表的描述
HTableDescriptor tableDesc = new HTableDescriptor(tableName);
// 在描述里添加列族
for (String columnFamily : columnFamilys) {
tableDesc.addFamily(new HColumnDescriptor(columnFamily));
}
// 根据配置好的描述建表
hAdmin.createTable(tableDesc);
System.out.println("创建表成功");
}
}
// 删除数据库表
public static void deleteTable(String tableName) throws Exception {
// 新建一个数据库管理员
HBaseAdmin hAdmin = new HBaseAdmin(conf);
if (hAdmin.tableExists(tableName)) {
// 关闭一个表
hAdmin.disableTable(tableName);
// 删除一个表
hAdmin.deleteTable(tableName);
System.out.println("删除表成功");
} else {
System.out.println("删除的表不存在");
System.exit(0);
}
}
// 添加一条数据
public static void addRow(String tableName, String row,
String columnFamily, String column, String value) throws Exception {
HTable table = new HTable(conf, tableName);
Put put = new Put(Bytes.toBytes(row));
// 参数出分别：列族、列、值
put.add(Bytes.toBytes(columnFamily), Bytes.toBytes(column),
Bytes.toBytes(value));
table.put(put);
}
// 删除一条数据
public static void delRow(String tableName, String row) throws Exception {
HTable table = new HTable(conf, tableName);
Delete del = new Delete(Bytes.toBytes(row));
table.delete(del);
}
// 删除多条数据
public static void delMultiRows(String tableName, String[] rows)
throws Exception {
HTable table = new HTable(conf, tableName);
List list = new ArrayList();
for (String row : rows) {
Delete del = new Delete(Bytes.toBytes(row));
list.add(del);
}
table.delete(list);
}
// get row
public static void getRow(String tableName, String row) throws Exception {
HTable table = new HTable(conf, tableName);
Get get = new Get(Bytes.toBytes(row));
Result result = table.get(get);
// 输出结果
for (KeyValue rowKV : result.raw()) {
System.out.print("Row Name: " + new String(rowKV.getRow()) + " ");
System.out.print("Timestamp: " + rowKV.getTimestamp() + " ");
System.out.print("column Family: " + new String(rowKV.getFamily()) + " ");
System.out.print("Row Name: " + new String(rowKV.getQualifier()) + " ");
System.out.println("Value: " + new String(rowKV.getValue()) + " ");
}
}
// get all records
public static void getAllRows(String tableName) throws Exception {
HTable table = new HTable(conf, tableName);
Scan scan = new Scan();
ResultScanner results = table.getScanner(scan);
// 输出结果
for (Result result : results) {
for (KeyValue rowKV : result.raw()) {
System.out.print("Row Name: " + new String(rowKV.getRow()) + " ");
System.out.print("Timestamp: " + rowKV.getTimestamp() + " ");
System.out.print("column Family: " + new String(rowKV.getFamily()) + " ");
System.out
.print("Row Name: " + new String(rowKV.getQualifier()) + " ");
System.out.println("Value: " + new String(rowKV.getValue()) + " ");
}
}
}
// main
public static void main(String[] args) {
try {
String tableName = "users2";
// 第一步：创建数据库表：“users2”
String[] columnFamilys = { "info", "course" };
OperateTable.createTable(tableName, columnFamilys);
// 第二步：向数据表的添加数据
// 添加第一行数据
OperateTable.addRow(tableName, "tht", "info", "age", "20");
OperateTable.addRow(tableName, "tht", "info", "sex", "boy");
OperateTable.addRow(tableName, "tht", "course", "china", "97");
OperateTable.addRow(tableName, "tht", "course", "math", "128");
OperateTable.addRow(tableName, "tht", "course", "english", "85");
// 添加第二行数据
OperateTable.addRow(tableName, "xiaoxue", "info", "age", "19");
OperateTable.addRow(tableName, "xiaoxue", "info", "sex", "boy");
OperateTable.addRow(tableName, "xiaoxue", "course", "china", "90");
OperateTable.addRow(tableName, "xiaoxue", "course", "math", "120");
OperateTable
.addRow(tableName, "xiaoxue", "course", "english", "90");
// 添加第三行数据
OperateTable.addRow(tableName, "qingqing", "info", "age", "18");
OperateTable.addRow(tableName, "qingqing", "info", "sex", "girl");
OperateTable
.addRow(tableName, "qingqing", "course", "china", "100");
OperateTable.addRow(tableName, "qingqing", "course", "math", "100");
OperateTable.addRow(tableName, "qingqing", "course", "english",
"99");
// 第三步：获取一条数据
System.out.println("获取一条数据");
OperateTable.getRow(tableName, "tht");
// 第四步：获取所有数据
System.out.println("获取所有数据");
OperateTable.getAllRows(tableName);
// 第五步：删除一条数据
System.out.println("删除一条数据");
OperateTable.delRow(tableName, "tht");
OperateTable.getAllRows(tableName);
// 第六步：删除多条数据
System.out.println("删除多条数据");
String[] rows = { "xiaoxue", "qingqing" };
OperateTable.delMultiRows(tableName, rows);
OperateTable.getAllRows(tableName);
// 第八步：删除数据库
System.out.println("删除数据库");
OperateTable.deleteTable(tableName);
} catch (Exception err) {
err.printStackTrace();
}
}
} 结果：
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:host.name=tht
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_45
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.home=D:\Java\jre6
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.class.path=。。。。。。。。。。。。
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.library.path=。。。。。。。。。。。
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:java.compiler=
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:os.name=Windows 7
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:os.arch=x86
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:os.version=6.1
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:user.home=C:\Users\Administrator
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Client environment:user.dir=D:\workspace\eclipse-workspace-jee-kepler\hadoop-Apps-tht
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=hconnection
13/09/09 22:01:18 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht
13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session
13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000b, negotiated timeout = 180000
13/09/09 22:01:18 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba8602
13/09/09 22:01:18 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht
13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session
13/09/09 22:01:18 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000c, negotiated timeout = 180000
13/09/09 22:01:20 INFO zookeeper.ZooKeeper: Session: 0x14102d851f8000c closed
13/09/09 22:01:20 INFO zookeeper.ClientCnxn: EventThread shut down
13/09/09 22:01:36 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba8602
13/09/09 22:01:36 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
13/09/09 22:01:36 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht
13/09/09 22:01:36 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session
13/09/09 22:01:38 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000d, negotiated timeout = 180000
13/09/09 22:01:38 INFO zookeeper.ZooKeeper: Session: 0x14102d851f8000d closed
13/09/09 22:01:38 INFO zookeeper.ClientCnxn: EventThread shut down
创建表成功
获取一条数据
Row Name: tht Timestamp: 1378735285456 column Family: course Row Name: china Value: 97
Row Name: tht Timestamp: 1378735285918 column Family: course Row Name: english Value: 85
Row Name: tht Timestamp: 1378735285591 column Family: course Row Name: math Value: 128
Row Name: tht Timestamp: 1378735285056 column Family: info Row Name: age Value: 20
Row Name: tht Timestamp: 1378735285368 column Family: info Row Name: sex Value: boy
获取所有数据
Row Name: qingqing Timestamp: 1378735286503 column Family: course Row Name: china Value: 100
Row Name: qingqing Timestamp: 1378735286547 column Family: course Row Name: english Value: 99
Row Name: qingqing Timestamp: 1378735286524 column Family: course Row Name: math Value: 100
Row Name: qingqing Timestamp: 1378735286463 column Family: info Row Name: age Value: 18
Row Name: qingqing Timestamp: 1378735286482 column Family: info Row Name: sex Value: girl
Row Name: tht Timestamp: 1378735285456 column Family: course Row Name: china Value: 97
Row Name: tht Timestamp: 1378735285918 column Family: course Row Name: english Value: 85
Row Name: tht Timestamp: 1378735285591 column Family: course Row Name: math Value: 128
Row Name: tht Timestamp: 1378735285056 column Family: info Row Name: age Value: 20
Row Name: tht Timestamp: 1378735285368 column Family: info Row Name: sex Value: boy
Row Name: xiaoxue Timestamp: 1378735286268 column Family: course Row Name: china Value: 90
Row Name: xiaoxue Timestamp: 1378735286403 column Family: course Row Name: english Value: 90
Row Name: xiaoxue Timestamp: 1378735286343 column Family: course Row Name: math Value: 120
Row Name: xiaoxue Timestamp: 1378735286114 column Family: info Row Name: age Value: 19
Row Name: xiaoxue Timestamp: 1378735286236 column Family: info Row Name: sex Value: boy
删除一条数据
Row Name: qingqing Timestamp: 1378735286503 column Family: course Row Name: china Value: 100
Row Name: qingqing Timestamp: 1378735286547 column Family: course Row Name: english Value: 99
Row Name: qingqing Timestamp: 1378735286524 column Family: course Row Name: math Value: 100
Row Name: qingqing Timestamp: 1378735286463 column Family: info Row Name: age Value: 18
Row Name: qingqing Timestamp: 1378735286482 column Family: info Row Name: sex Value: girl
Row Name: xiaoxue Timestamp: 1378735286268 column Family: course Row Name: china Value: 90
Row Name: xiaoxue Timestamp: 1378735286403 column Family: course Row Name: english Value: 90
Row Name: xiaoxue Timestamp: 1378735286343 column Family: course Row Name: math Value: 120
Row Name: xiaoxue Timestamp: 1378735286114 column Family: info Row Name: age Value: 19
Row Name: xiaoxue Timestamp: 1378735286236 column Family: info Row Name: sex Value: boy
删除多条数据
删除数据库
13/09/09 22:01:40 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=master:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba8602
13/09/09 22:01:40 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 9608@tht
13/09/09 22:01:40 INFO zookeeper.ClientCnxn: Opening socket connection to server master/192.168.1.101:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
13/09/09 22:01:40 INFO zookeeper.ClientCnxn: Socket connection established to master/192.168.1.101:2181, initiating session
13/09/09 22:01:42 INFO zookeeper.ClientCnxn: Session establishment complete on server master/192.168.1.101:2181, sessionid = 0x14102d851f8000e, negotiated timeout = 180000
13/09/09 22:01:44 INFO zookeeper.ZooKeeper: Session: 0x14102d851f8000e closed
13/09/09 22:01:44 INFO zookeeper.ClientCnxn: EventThread shut down
13/09/09 22:01:48 INFO client.HBaseAdmin: Started disable of users2
13/09/09 22:01:50 INFO client.HBaseAdmin: Disabled users2
13/09/09 22:01:51 INFO client.HBaseAdmin: Deleted users2
删除表成功