hbase插入数据时,报错java.lang.IllegalArgumentException: KeyValue size too large

报错异常信息:

Exception in thread "main" java.lang.IllegalArgumentException: KeyValue size too large
    at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1399)
    at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1385)
    at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:978)
    at org.apache.hadoop.hbase.client.HTable.put(HTable.java:945)
    at com.hbase.preRegin.InsertDataToTable.main(InsertDataToTable.java:45)

经查看原因,是由于某一列的值大小超过了:hbase-site.xml文件里设置的hbase.client.keyvalue.maxsize值;默认大小为10M,即 10485760;
该值表示某一列的值最大大小值,当大于该值的时候,将会抛出该异常;

  <property>
    <name>hbase.client.keyvalue.maxsizename>
    <value>10485760value>
  property>

可以从下面源码里面看到,它会遍历每一个put里面的每一列的值,判断大小是否超过了maxKeyValueSize;

private void validatePut(final Put put) throws IllegalArgumentException{  
    if (put.isEmpty()) {  
      throw new IllegalArgumentException("No columns to insert");  
    }  
    if (maxKeyValueSize > 0) {  
      for (List list : put.getFamilyMap().values()) {  
        for (KeyValue kv : list) {  
          if (kv.getLength() > maxKeyValueSize) {**重点内容**  
            throw new IllegalArgumentException("KeyValue size too large");  
          }  
        }  
      }  
    }  
  } 

你可能感兴趣的:(hbase-应用)