对db4o direct (fast) access的探索

一般情况下，用db4o查询到的数据都是你的POJO的实例（当你读取某个实例的属性时，db4o会调用activate方法激活该实例）。这种机制非常的不利于你读取较大数据量的情形，比如你跑报表的时候，你的某个报表包含10000条记录（其实在odb不应该使用“记录”这一说法），如果只考虑顶层数据对象的激活，就得硬生生实例化10000个对象。如果你执行的是一个聚集操作，比如求10000个数据的平均值，你将得到10000个对象，然后你逐一读取，求均值，很明显，这很不格算。资源都浪费在实例化对象上面了。理论上这会是一个瓶颈，实际上也是如此。

 

db4o目前还没有直接提供此类支持（db4o网站上倒是有一个关于Aggregate Queries的描述，见http://developer.db4o.com/Projects/html/projectspaces/db4o_product_design/aggregate_queries.html，不过一直没人去实现它）

 

那么我可不可以在不实例化对象的前提下，直接读取数据呢？ 我觉得这是可以的。至少有两个途径：

 

1，自己读取db4o的bytes，得到自己需要的数据。在调试过程中，发现db4o在实例化POJO对象之前，是从bytes读取数据的，比如这个bytes：

 

[-1 , -4 , -86 , 11 , 10 , 0 , 0 , 0 , 67 , -92 , -1 , -1 , -1 , -1 , -1 , -1 , 19 , 6 , 0 , 0 , 0 , 67 , 0 , 2 , 95 , -42 , 0 , 28 , -9 , -102 , 0 , 0 , 2 , 49 , 0 , 0 , 0 , 32 , 0 , 1 , 97 , -63 , 0 , 0 , 11 , -123 , 0 , 0 , 0 , -67 , 0 , 0 , 1 , 116 , 0 , 0 , 0 , 113 , 0 , 0 , 0 , 76 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 45 , 7 , -62 , 76 , -22 , 0 , 0 , 0 , 87 , 0 , 0 , 0 , 26 , 70 , 0 , 0 , 0 , 11 , 115 , 0 , 121 , 0 , 115 , 0 , 95 , 0 , 109 , 0 , 105 , 0 , 103 , 0 , 114 , 0 , 97 , 0 , 116 , 0 , 101 , 0 , 0 , 0 , 0 , 36 , 99 , 0 , 98 , 0 , 55 , 0 , 57 , 0 , 53 , 0 , 52 , 0 , 101 , 0 , 49 , 0 , 45 , 0 , 52 , 0 , 55 , 0 , 53 , 0 , 57 , 0 , 45 , 0 , 52 , 0 , 99 , 0 , 53 , 0 , 97 , 0 , 45 , 0 , 98 , 0 , 53 , 0 , 100 , 0 , 56 , 0 , 45 , 0 , 48 , 0 , 49 , 0 , 100 , 0 , 101 , 0 , 102 , 0 , 51 , 0 , 52 , 0 , 56 , 0 , 54 , 0 , 48 , 0 , 51 , 0 , 99 , 0 , 0 , 0 , 0 , -72 , 80 , 0 , 108 , 0 , 101 , 0 , 97 , 0 , 115 , 0 , 101 , 0 , 32 , 0 , 101 , 0 , 45 , 0 , 109 , 0 , 97 , 0 , 105 , 0 , 108 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 101 , 0 , 32 , 0 , 67 , 0 , 68 , 0 , 65 , 0 , 32 , 0 , 116 , 0 , 111 , 0 , 32 , 0 , 100 , 0 , 112 , 0 , 101 , 0 , 100 , 0 , 105 , 0 , 110 , 0 , 101 , 0 , 108 , 0 , 108 , 0 , 105 , 0 , 64 , 0 , 109 , 0 , 105 , 0 , 99 , 0 , 104 , 0 , 105 , 0 , 103 , 0 , 97 , 0 , 110 , 0 , 104 , 0 , 101 , 0 , 97 , 0 , 114 , 0 , 116 , 0 , 46 , 0 , 99 , 0 , 111 , 0 , 109 , 0 , 46 , 0 , 32 , 0 , 83 , 0 , 104 , 0 , 101 , 0 , 32 , 0 , 119 , 0 , 105 , 0 , 108 , 0 , 108 , 0 , 32 , 0 , 115 , 0 , 101 , 0 , 101 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 97 , 0 , 116 , 0 , 32 , 0 , 105 , 0 , 116 , 0 , 32 , 0 , 103 , 0 , 101 , 0 , 116 , 0 , 115 , 0 , 32 , 0 , 116 , 0 , 111 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 101 , 0 , 32 , 0 , 97 , 0 , 112 , 0 , 112 , 0 , 114 , 0 , 111 , 0 , 112 , 0 , 114 , 0 , 105 , 0 , 97 , 0 , 116 , 0 , 101 , 0 , 32 , 0 , 80 , 0 , 73 , 0 , 46 , 0 , 32 , 0 , 73 , 0 , 116 , 0 , 32 , 0 , 109 , 0 , 97 , 0 , 121 , 0 , 32 , 0 , 32 , 0 , 101 , 0 , 110 , 0 , 100 , 0 , 32 , 0 , 117 , 0 , 112 , 0 , 32 , 0 , 98 , 0 , 101 , 0 , 105 , 0 , 110 , 0 , 103 , 0 , 32 , 0 , 115 , 0 , 111 , 0 , 109 , 0 , 101 , 0 , 111 , 0 , 110 , 0 , 101 , 0 , 32 , 0 , 101 , 0 , 108 , 0 , 115 , 0 , 101 , 0 , 32 , 0 , 114 , 0 , 97 , 0 , 116 , 0 , 104 , 0 , 101 , 0 , 114 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 97 , 0 , 110 , 0 , 32 , 0 , 68 , 0 , 114 , 0 , 46 , 0 , 32 , 0 , 79 , 0 , 39 , 0 , 68 , 0 , 111 , 0 , 110 , 0 , 110 , 0 , 101 , 0 , 108 , 0 , 108 , 0 , 46 , 0 , 32 , 0 , 67 , 0 , 68 , 0 , 65 , 0 , 32 , 0 , 101 , 0 , 109 , 0 , 97 , 0 , 105 , 0 , 108 , 0 , 101 , 0 , 100 , 0 , 32 , 0 , 50 , 0 , 47 , 0 , 49 , 0 , 55 , 0 , 47 , 0 , 48 , 0 , 54 , 0 , 0 , 0 , 0 , 14 , 77 , 0 , 105 , 0 , 99 , 0 , 104 , 0 , 105 , 0 , 103 , 0 , 97 , 0 , 110 , 0 , 32 , 0 , 72 , 0 , 101 , 0 , 97 , 0 , 114 , 0 , 116 , 0]

用

 

println new String([-1 , -4 , -86 , 11 , 10 , 0 , 0 , 0 , 67 , -92 , -1 , -1 , -1 , -1 , -1 , -1 , 19 , 6 , 0 , 0 , 0 , 67 , 0 , 2 , 95 , -42 , 0 , 28 , -9 , -102 , 0 , 0 , 2 , 49 , 0 , 0 , 0 , 32 , 0 , 1 , 97 , -63 , 0 , 0 , 11 , -123 , 0 , 0 , 0 , -67 , 0 , 0 , 1 , 116 , 0 , 0 , 0 , 113 , 0 , 0 , 0 , 76 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 1 , 45 , 7 , -62 , 76 , -22 , 0 , 0 , 0 , 87 , 0 , 0 , 0 , 26 , 70 , 0 , 0 , 0 , 11 , 115 , 0 , 121 , 0 , 115 , 0 , 95 , 0 , 109 , 0 , 105 , 0 , 103 , 0 , 114 , 0 , 97 , 0 , 116 , 0 , 101 , 0 , 0 , 0 , 0 , 36 , 99 , 0 , 98 , 0 , 55 , 0 , 57 , 0 , 53 , 0 , 52 , 0 , 101 , 0 , 49 , 0 , 45 , 0 , 52 , 0 , 55 , 0 , 53 , 0 , 57 , 0 , 45 , 0 , 52 , 0 , 99 , 0 , 53 , 0 , 97 , 0 , 45 , 0 , 98 , 0 , 53 , 0 , 100 , 0 , 56 , 0 , 45 , 0 , 48 , 0 , 49 , 0 , 100 , 0 , 101 , 0 , 102 , 0 , 51 , 0 , 52 , 0 , 56 , 0 , 54 , 0 , 48 , 0 , 51 , 0 , 99 , 0 , 0 , 0 , 0 , -72 , 80 , 0 , 108 , 0 , 101 , 0 , 97 , 0 , 115 , 0 , 101 , 0 , 32 , 0 , 101 , 0 , 45 , 0 , 109 , 0 , 97 , 0 , 105 , 0 , 108 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 101 , 0 , 32 , 0 , 67 , 0 , 68 , 0 , 65 , 0 , 32 , 0 , 116 , 0 , 111 , 0 , 32 , 0 , 100 , 0 , 112 , 0 , 101 , 0 , 100 , 0 , 105 , 0 , 110 , 0 , 101 , 0 , 108 , 0 , 108 , 0 , 105 , 0 , 64 , 0 , 109 , 0 , 105 , 0 , 99 , 0 , 104 , 0 , 105 , 0 , 103 , 0 , 97 , 0 , 110 , 0 , 104 , 0 , 101 , 0 , 97 , 0 , 114 , 0 , 116 , 0 , 46 , 0 , 99 , 0 , 111 , 0 , 109 , 0 , 46 , 0 , 32 , 0 , 83 , 0 , 104 , 0 , 101 , 0 , 32 , 0 , 119 , 0 , 105 , 0 , 108 , 0 , 108 , 0 , 32 , 0 , 115 , 0 , 101 , 0 , 101 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 97 , 0 , 116 , 0 , 32 , 0 , 105 , 0 , 116 , 0 , 32 , 0 , 103 , 0 , 101 , 0 , 116 , 0 , 115 , 0 , 32 , 0 , 116 , 0 , 111 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 101 , 0 , 32 , 0 , 97 , 0 , 112 , 0 , 112 , 0 , 114 , 0 , 111 , 0 , 112 , 0 , 114 , 0 , 105 , 0 , 97 , 0 , 116 , 0 , 101 , 0 , 32 , 0 , 80 , 0 , 73 , 0 , 46 , 0 , 32 , 0 , 73 , 0 , 116 , 0 , 32 , 0 , 109 , 0 , 97 , 0 , 121 , 0 , 32 , 0 , 32 , 0 , 101 , 0 , 110 , 0 , 100 , 0 , 32 , 0 , 117 , 0 , 112 , 0 , 32 , 0 , 98 , 0 , 101 , 0 , 105 , 0 , 110 , 0 , 103 , 0 , 32 , 0 , 115 , 0 , 111 , 0 , 109 , 0 , 101 , 0 , 111 , 0 , 110 , 0 , 101 , 0 , 32 , 0 , 101 , 0 , 108 , 0 , 115 , 0 , 101 , 0 , 32 , 0 , 114 , 0 , 97 , 0 , 116 , 0 , 104 , 0 , 101 , 0 , 114 , 0 , 32 , 0 , 116 , 0 , 104 , 0 , 97 , 0 , 110 , 0 , 32 , 0 , 68 , 0 , 114 , 0 , 46 , 0 , 32 , 0 , 79 , 0 , 39 , 0 , 68 , 0 , 111 , 0 , 110 , 0 , 110 , 0 , 101 , 0 , 108 , 0 , 108 , 0 , 46 , 0 , 32 , 0 , 67 , 0 , 68 , 0 , 65 , 0 , 32 , 0 , 101 , 0 , 109 , 0 , 97 , 0 , 105 , 0 , 108 , 0 , 101 , 0 , 100 , 0 , 32 , 0 , 50 , 0 , 47 , 0 , 49 , 0 , 55 , 0 , 47 , 0 , 48 , 0 , 54 , 0 , 0 , 0 , 0 , 14 , 77 , 0 , 105 , 0 , 99 , 0 , 104 , 0 , 105 , 0 , 103 , 0 , 97 , 0 , 110 , 0 , 32 , 0 , 72 , 0 , 101 , 0 , 97 , 0 , 114 , 0 , 116 , 0] as byte[], 'utf8')

把它打出来，是这样的：

 

??C???????C_???1 a???tqL-?L?WFsys_migrate$cb7954e1-4759-4c5a-b5d8-01def348603c?Please e-mail the CDA to [email protected]. She will see that it gets to the appropriate PI. It may  end up being someone else rather than Dr. O'Donnell. CDA emailed 2/17/06Michigan Heart

有一丝希望的曙光。不过，几经调试，我愣是没有理出个头绪，搞得一团乱麻。我长叹一声，算了，想其它的辙吧。

 

2， 在阅读db4o源码的过程中，发现在Evaluation接口中有这么一段文字：

 

db4o 写道

/**
* for implementation of callback evaluations.
* <br><br>
* To constrain a {@link Query} node with your own callback
* <code>Evaluation</code>, construct an object that implements the
* <code>Evaluation</code> interface and register it by passing it
* to {@link Query#constrain(Object)}.
* <br><br>
* Evaluations are called as the last step during query execution,
* after all other constraints have been applied. Evaluations in higher
* level {@link Query} nodes in the query graph are called first.
* <br><br>Java client/server only:<br>
* db4o first attempts to use Java Serialization to allow to pass final
* variables to the server. Please make sure that all variables that are
* used within the {@link #evaluate(Candidate)} method are Serializable. This may include
* the class an anonymous Evaluation object is created in. If db4o is
* not successful at using Serialization, the Evaluation is transported
* to the server in a db4o {@link com.db4o.io.MemoryBin}. In this case final variables can
* not be restored.
*/

 希望的曙光再次出现。

 

啥也不说了，上代码：

 

DirectCandidateAccessor.java :

 

package com.db4o.query;

import com.db4o.internal.query.processor.QCandidate;
import groovy.lang.Closure;

/**
 * Created by IntelliJ IDEA.
 * User: S.C.
 * Date: Jan 4, 2011
 * Time: 11:29:41 AM
 * To change this template use File | Settings | File Templates.
 */
public class DirectCandidateAccessor implements Evaluation {
    private Closure callback;
    public DirectCandidateAccessor(Closure callback) {
        this.callback = callback;
    }

    public void evaluate(Candidate candidate) {
        QCandidate qc = (QCandidate) candidate;
        int _key = qc._key;
        Object _value = qc.getObject();
        //System.out.println("Direct Access :: key: " + _key + ", value: " + _value);
        this.callback.call(new Object[]{_key, _value});
    }

}

 

QQueryBase#constrain :

 

public Constraint constrain(Object example) {
        // +by S.C. 09-Dec-2010 ==>
        if(example instanceof Pattern) {
            return constrain(new RegexConstraint((Pattern)example));
        }
        // +by S.C. 09-Dec-2010 <==

        // +by S.C. 04-Jan-2011 ==>
        if(example instanceof Closure) {
            return constrain(new DirectCandidateAccessor((Closure)example));
        }
        // +by S.C. 04-Jan-2011 <==

        synchronized (streamLock()) {
            ReflectClass claxx = reflectClassForClass(example);             
            if (claxx != null) {                
                return addClassConstraint(claxx);
            }
            
            QConEvaluation eval = Platform4.evaluationCreate(_trans, example);
			if (eval != null) {
                return addEvaluationToAllConstraints(eval);
            }
			
            Collection4 constraints = new Collection4();
            addConstraint(constraints, example);
            return toConstraint(constraints);
        }
    }

 这里就加了这么一句

if(example instanceof Closure) {
    return constrain(new DirectCandidateAccessor((Closure)example));
}

 

 

 

小试牛刀：

 

 

def callback = {k,v->
  println "key: $k, value: $v"
}
Site.findAll(name:callback,'pi.name':callback,id:callback).size()

 

 比起

 

def callback = {k,v->
  println "key: $k, value: $v"
}
Site.findAll(name:callback,'pi.name':callback,id:callback).each{println it}

 来，不是一般的快，因为后者做了实例化，而前者避免了这一操作。