从Oracle中按照步长批量查询数据的一种实现方式(java的Date和Oracle的timestamp的类型转换)

场景:

数据管理系统A:
  • 基于SpringMVC,B/S架构;
  • 管理试验过程,从试验计划下达,内容安排,试验设备分配到试验报告生成的全过程;
数据采集系统B:
  • 基于C#,C/S架构
  • Oracle记录各种试验设备的采集数据(时域数据,采样间隔为:0.5s~5s)
  • 数据库的采集时间字段类型为timestamp精确到毫秒级
  • 已停止维护

需求:

用户通过浏览器登陆A系统后,能够随时查看不同试验设备一段时间的试验数据(至少半个小时的),如果全量抓取,数据量过大,显然是不可取的,而且,按照需求也没必要,抓取的这些数据能够绘制折线图,在web端展示,看出大致趋势即可;

一种实现方案

  • 首先,基于springboot搭建一个微服务C,直接连接B系统的oracle数据库
  • C系统向A系统提供restful的接口,供A系统查询
  • 给定开始时间,结束时间,需要的数据个数,然后oracle中执行存储过程,按照一定的间隔返回所有的需要查询的时间;
  • 按照查询时间,执行一次in查询,拿到最终的数据集;

几点争议

  • 关于第三方系统直连其他系统数据库的安全性问题

A系统和B系统本就是一家负责开发和维护的,而且是得到甲方允许的;

  • 为什么A系统不直接连B的数据库

A系统架构较老,模块繁重,扩展性差,不是基于maven,配置多数据源连接可能面临一系列问题...

  • 为什么存储过程不直接返回数据,而只是返回时间集;

因为不同的设备的数据存在不同的表中,表的字段除了采集时间之外,其余的字段各不相同,如果想一次拿出需要的数据,则需要在oracle中定义各种type,一个设备表将需要定义一个type,总之,不利于扩展

相关代码

-- 定义一个array 接受筛选后的时间
CREATE OR REPLACE TYPE "ARRAY_COLLECTION_TIME" as table of TIMESTAMP(3);
-- 声明包
create or replace package pkg_query as
 function get_required_collection_times(filter_str in varchar2,table_name in varchar2,require_count in number,total_count out number) return ARRAY_COLLECTION_TIME;
end pkg_query;
--- end
create or replace package body pkg_query as
 function get_required_collection_times(filter_str in varchar2,table_name in varchar2,require_count in number,total_count out number) return ARRAY_COLLECTION_TIME as
 begin declare type param_cursor is ref cursor;
            v_param_cursour param_cursor;
            required_collection_times ARRAY_COLLECTION_TIME;
            collection_time timestamp(3);
            i_index number;
            counter number;
            step number;
        begin
 i_index := 0;
            counter := 0;
            required_collection_times := ARRAY_COLLECTION_TIME();
            execute immediate 'SELECT count(*) FROM '||table_name||' t  where t.'||filter_str into total_count;
            if total_count = 0 then
 return required_collection_times;
            end if;
            if require_count > total_count then
 open v_param_cursour for 'SELECT COLLECTIONTIME FROM '||table_name||' t where t.'||filter_str;
                loop
 collection_time := '';
                    fetch v_param_cursour into collection_time;
                    exit when v_param_cursour%NOTFOUND;
                    required_collection_times.extend;
                    i_index := i_index + 1;
                    required_collection_times(i_index) := collection_time;
                end loop;
                close v_param_cursour;
                return required_collection_times;
            end if;
            -- 步长
 step:= 0;
            execute immediate 'SELECT trunc('||total_count||'/'||require_count||') FROM dual' into step;
            open v_param_cursour for 'SELECT COLLECTIONTIME FROM '||table_name||' t where t.'||filter_str;
            loop
 collection_time := '';
                fetch v_param_cursour into collection_time;
                exit when v_param_cursour%NOTFOUND;
                counter := counter +1;
                if counter = step then
 required_collection_times.extend;
                    i_index := i_index + 1;
                    required_collection_times(i_index) := collection_time;
                    -- 重置计数器
 counter := 0;
                end if;
            end loop;
            close v_param_cursour;
            return required_collection_times;
        end;
    end get_required_collection_times;
end pkg_query;

查询时,注意oracle的timestampjavaDate的转换

public List getRequiredCollectionTimes(String tableName,Date from,Date to,int totalCount){
    List timestampList = new ArrayList<>();
    SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss:sss");
    Object results = jdbcTemplate.execute(new CallableStatementCreator() {
        @Override
 public CallableStatement createCallableStatement(Connection connection) throws SQLException {
            String executeFunSql = "{? = call pkg_query.get_required_collection_times(?,?,?,?)}";
            CallableStatement cs = connection.prepareCall(executeFunSql);
            cs.registerOutParameter(1, OracleTypes.ARRAY, "ARRAY_COLLECTION_TIME".toUpperCase());
            cs.setString(2, "collectiontime between to_timestamp('" + sdf.format(from) + "','YYYY-MM-DD HH24:MI:SS:ff') and to_timestamp('" + sdf.format(to) + "','YYYY-MM-DD HH24:MI:SS:ff')");
            cs.setString(3, tableName);
            cs.setInt(4, totalCount);
            cs.registerOutParameter(5, OracleTypes.NUMBER);
            return cs;
        }
    }, new CallableStatementCallback() {
        @Override
 public Object doInCallableStatement(CallableStatement cs) throws SQLException, DataAccessException {
            cs.execute();
            Array outArray = cs.getArray(1);
            System.out.println("实际满足的总记录数:"+cs.getLong(5));
            Object[] outObjects = (Object[]) outArray.getArray();
            for (Object outObject : outObjects) {
                timestampList.add((Timestamp) outObject);
            }
            return timestampList;
        }
    });
    return timestampList;
}
public List> getRequiredParamDataList(String tableName,Set paramColumnNames,List collectionTimes){
    List filterStr = new ArrayList<>();
    for (Date filterTime:collectionTimes){
        filterStr.add("to_timestamp('"+new java.sql.Timestamp(filterTime.getTime())+"','YYYY-MM-DD HH24:MI:SS:ff')");
    }
    final String sql = "select collectionTime,"+String.join(",",paramColumnNames)+" from "+tableName+" t where t.collectiontime in ("+String.join(",",filterStr)+") order by collectiontime asc";
    List> dataList = jdbcTemplate.queryForList(sql);
    return dataList;
} 

                            
                        
                    
                    
                    

你可能感兴趣的:(从Oracle中按照步长批量查询数据的一种实现方式(java的Date和Oracle的timestamp的类型转换))