hbase实现分页查询

[Author]: kwu 

hbase实现分页查询，实现按时间查询最新的15条，hbase的查询主要是通过rowkey来进行的，保证查询的高效。column的filter查询效率较低。

1、rowkey的设计

以时间的倒序进行查询，如20150818_152130来看，设计rowkey

20150818的hash64值+(999999-152130)。

1）MurmurHash的64的冲突非常小，并实现rowkey的散列。

2）日分秒与999999的差值，可以实现逆序

2、插入数据的操作

 String[] splited = jsData.split("`");

 SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss");
 Date date = new Date();
 String format = sdf.format(date);

 String dateday = format.substring(0, 8);
 String datehour = format.substring(8, 10);
 String dateMin = format.substring(10, 12);
 String dateSec = format.substring(12, 14);

 int datehourSub = 99 - new Integer(datehour);
 int dateMinSub = 99 - new Integer(dateMin);
 int dateSecSub = 99 - new Integer(dateSec);

 String dateHash = MurmurHash.hash64(dateday) + "_" + datehourSub  + dateMinSub  + dateSecSub;

 Put put = new Put((dateHash + "_" + splited[0]).getBytes());
 put.add("cf1".getBytes(), "bdcCookieId".getBytes(), splited[0].getBytes());
 put.add("cf1".getBytes(), "pcScreenRatio".getBytes(), splited[1].getBytes());
 put.add("cf1".getBytes(), "objClassName".getBytes(), splited[2].getBytes());
 put.add("cf1".getBytes(), "pageOpenTime".getBytes(), splited[3].getBytes());
 put.add("cf1".getBytes(), "pageLoadCompleteTime".getBytes(), splited[4].getBytes());
 put.add("cf1".getBytes(), "pageCloseTime".getBytes(), splited[5].getBytes());
 put.add("cf1".getBytes(), "category".getBytes(), splited[6].getBytes());
 put.add("cf1".getBytes(), "browserVersion".getBytes(), splited[7].getBytes());
 put.add("cf1".getBytes(), "currentURL".getBytes(), splited[8].getBytes());
 put.add("cf1".getBytes(), "targetParentURL".getBytes(), splited[9].getBytes());
 put.add("cf1".getBytes(), "referrer".getBytes(), splited[10].getBytes());
 put.add("cf1".getBytes(), "clickX".getBytes(), splited[11].getBytes());
 put.add("cf1".getBytes(), "clickY".getBytes(), splited[12].getBytes());
 put.add("cf1".getBytes(), "targetURL".getBytes(), splited[13].getBytes());
 put.add("cf1".getBytes(), "objID".getBytes(), splited[14].getBytes());
 put.add("cf1".getBytes(), "objName".getBytes(), splited[15].getBytes());
 put.add("cf1".getBytes(), "objValue".getBytes(), objValue.getBytes());
 put.add("cf1".getBytes(), "siteCookieId".getBytes(), splited[17].getBytes());
 put.add("cf1".getBytes(), "scrollValue".getBytes(), splited[18].getBytes());
 put.add("cf1".getBytes(), "sessionId".getBytes(), splited[19].getBytes());
 put.add("cf1".getBytes(), "ip".getBytes(), splited[20].getBytes());
 put.add("cf1".getBytes(), "macAdress".getBytes(), splited[21].getBytes());
 put.add("cf1".getBytes(), "gaId".getBytes(), splited[22].getBytes());
 put.add("cf1".getBytes(), "baiduId".getBytes(), splited[23].getBytes());
 put.add("cf1".getBytes(), "activeTime".getBytes(), splited[24].getBytes());
 put.add("cf1".getBytes(), "status".getBytes(), splited[25].getBytes());
 put.add("cf1".getBytes(), "triggertime".getBytes(), splited[26].getBytes());

 List<Put> list = new ArrayList<Put>();
 list.add(put);

 HTable table = new HTable(configuration, "jsActionPage");
 table.put(list);

 table.close();

3、查询 利用scan 时间倒序

 HTable table = new HTable(configuration, "jsActionPage");
 Scan scan = new Scan();
 scan.setStartRow(startRow.getBytes());
 scan.setStopRow(stopRow.getBytes());

 JSONObject resultJson = new JSONObject();

 JSONObject temp = null;
 ResultScanner scanner = table.getScanner(scan);

 for (Result result : scanner) {
     JSONArray jsonArray = new JSONArray();

     String rowkey = "";
     for (KeyValue keyValue : result.raw()) {
         byte[] row = keyValue.getRow();
         rowkey = new String(row);

         String qualifier = new String(keyValue.getQualifier());
         String value = new String(keyValue.getValue());

         temp = new JSONObject();
         temp.put(qualifier, value);
         jsonArray.add(temp);
     }
     resultJson.put(rowkey, jsonArray);
 }

 table.close();

最后返回json的数据格式