Phoenix设置时间戳

Phoenix设置时间戳

 

phonex-4.3 hbase-0.98.10

 

环境 phonex-4.3,hbase-0.98.10,hadoop-2.5.2 
设置时间戳 使用hbase的TTL机制用来定时删除过期的数据记录,但是默认的是数据插入是的系统时间戳,显然不可以,需要手动设置时间戳,找了半天,官方文档上提到CurrentSCN 的描述,最终使用java客户端设置时间戳,代买如下:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.Statement;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Properties;

import org.apache.phoenix.util.PhoenixRuntime;
import org.junit.Test;

public class PhoenixTest {
    @Test
    public void createTable(){
        try {
            String table = "TEST5";
            Statement stmt = null;
            Properties props = new Properties();
            // 设置PhoenixRuntime.CURRENT_SCN_ATTRIB的值
            props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(System.currentTimeMillis()-1000*60*60*24+1));
            Connection con = DriverManager.getConnection("jdbc:phoenix:localhost",props);
            stmt = con.createStatement();
            String create = "create table " + table + " (mykey integer not null primary key, mycolumn varchar)";
            stmt.execute(create);
            con.commit();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }   

    @Test
    public void upsert(){
        try {
            String table = "TEST5";
            Statement stmt = null;
            Properties props = new Properties();
            // 设置PhoenixRuntime.CURRENT_SCN_ATTRIB的值
            props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(System.currentTimeMillis()-1000*60*60*24+1));
            Connection con = DriverManager.getConnection("jdbc:phoenix:localhost",props);
            stmt = con.createStatement();
            stmt.executeUpdate("upsert into "+ table+"(mykey,mycolumn) values(13,'13hhhhhhhhhhhhhhhhhh')");
            con.commit();
            con.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

• 先创建表,再upsert数据 
  phoenix的神奇地方:
• upsert数据的时间戳要大于必须要创建表的时间,否则报如下异常:

15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:38783 from root: closed
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:38783 from root: stopped, connections 0
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:45090 from root: wrote request header call_id: 6 method_name: "ExecService" request_param: true
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:45090 from root: got response header call_id: 6, totalSize: 609 bytes
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:45090 from root: wrote request header call_id: 7 method_name: "ExecService" request_param: true
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:45090 from root: got response header call_id: 7, totalSize: 162 bytes
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:45090 from root: wrote request header call_id: 8 method_name: "ExecService" request_param: true
15/04/07 19:03:42 DEBUG ipc.RpcClient: IPC Client (231806250) connection to localhost/127.0.0.1:45090 from root: got response header call_id: 8, totalSize: 162 bytes
org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined. tableName=TEST5
    at org.apache.phoenix.compile.FromCompiler$BaseColumnResolver.createTableRef(FromCompiler.java:359)
    at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.<init>(FromCompiler.java:237)
    at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.<init>(FromCompiler.java:231)
    at org.apache.phoenix.compile.FromCompiler.getResolverForMutation(FromCompiler.java:207)
    at org.apache.phoenix.compile.UpsertCompiler.compile(UpsertCompiler.java:248)
    at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:487)
    at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:478)
    at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:279)
    at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:272)
    at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:270)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1052)
    at test.phoenix.PhoenixTest.batchInsert(PhoenixTest.java:32)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
15/04/07 19:03:42 INFO client.HConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x14c92a833ac0183
15/04/07 19:03:42 DEBUG zookeeper.ZooKeeper: Closing session: 0x14c92a833ac0183
15/04/07 19:03:42 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x14c92a833ac0183
15/04/07 19:03:42 DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@11f569ee
15/04/07 19:03:42 DEBUG ipc.Client: removing client from cache: org.apache.hadoop.ipc.Client@11f569ee

• 建表时间如果大于当前时间,则创建表后 phoenix的sqlline客户端看不见表,需要当前时间大于建表时间才能显示表,这个功能相当超前 
  总结
• phoenix 摒弃了hbase的多个version机制,每次查询出的是最新版本,不能反查历史版本,需要从业务层面设计rowkey来实现;
• 关于时间戳,按照目前的形式,只能在创建connection的时候设置,由此每添加一条记录就要创建一次链接,导致不能批量插入,因此需要从业务层次设计,比如一批数据取一个时间戳等等…
• 总之,phoenix提供了一个可以自定义时间戳的方式已经不错了,比没有强,希望phoenix可以在后续版本中逐渐的完善更加的人性化