HBase:根据Rowkey批量查询数据

创建链接属性: 

public static Configuration conf = null;  
    public static Connection connection = null;  
    public static Admin admin = null;  
    static {  
        conf = HBaseConfiguration.create();  
   
        conf.set("hbase.zookeeper.quorum", "10.10.1.100,xxxxx");  
        conf.set("hbase.master", "xxxxx:60010");  
        conf.set("hbase.zookeeper.property.clientPort", "2181");   
        conf.set("zookeeper.znode.parent", "/hbase-unsecure");  
        conf.set("hbase.client.keyvalue.maxsize", "1048576000"); //1G  
        conf = HBaseConfiguration.create(conf);  
        try {  
            connection = ConnectionFactory.createConnection(conf);  
            admin = connection.getAdmin();  
        } catch (IOException e) {  
            e.printStackTrace();  
        }  
    } 

开始做查询操作:] vi

public static List qurryTableTest(List rowkeyList) throws IOException {  
    String tableName = "table_a";  
    Table table = connection.getTable( TableName.valueOf(tableName));// 获取表  
    for (String rowkey : rowkeyList){  
        Get get = new Get(Bytes.toBytes(rowkey));  
        Result result = table.get(get);  
        for (Cell kv : result.rawCells()) {  
            String value = Bytes.toString(CellUtil.cloneValue(kv));  
            list.add(value);  
        }  
    }  
    return list;  
}  

但是,每条rowkey都要发起一次请求,这种方法效率十分低,测试了10000条记录查询要用4分钟左右,但是需要是秒回,展示,这是不行的。

所以在HTable.class里看到了这个get方法:

public Result[] get(List gets) throws IOException {  
        if(gets.size() == 1) {  
            return new Result[]{this.get((Get)gets.get(0))};  
        } else {  
            try {  
                Object[] r1 = this.batch(gets);  
                Result[] results = new Result[r1.length];  
                int i = 0;  
                Object[] arr$ = r1;  
                int len$ = r1.length;  
  
                for(int i$ = 0; i$ < len$; ++i$) {  
                    Object o = arr$[i$];  
                    results[i++] = (Result)o;  
                }  
  
                return results;  
            } catch (InterruptedException var9) {  
                throw (InterruptedIOException)(new InterruptedIOException()).initCause(var9);  
            }  
        }  
    }  

这就简单了,对上边的方法进行改进:

public static List qurryTableTestBatch(List rowkeyList) throws IOException {  
        List getList = new ArrayList();  
        String tableName = "table_a";  
        Table table = connection.getTable( TableName.valueOf(tableName));// 获取表  
        for (String rowkey : rowkeyList){//把rowkey加到get里,再把get装到list中  
            Get get = new Get(Bytes.toBytes(rowkey));  
            getList.add(get);  
        }  
        Result[] results = table.get(getList);//重点在这,直接查getList  
        for (Result result : results){//对返回的结果集进行操作  
            for (Cell kv : result.rawCells()) {  
                String value = Bytes.toString(CellUtil.cloneValue(kv));  
                list.add(value);  
            }  
        }  
        return list;  
    }  
根据这种批量的方法,10000个rowkey进行测试,时间为2s,速度提升特别大。

你可能感兴趣的:(大数据~HBase)