greendao源码解析

NOTE：greendao由Markus Junginger创建（markus另一项大作：eventbus）

新的手机数据库 ObjectBox，代替SQLite,据说快10倍，同时支持kottlin，支持RxJava 2。
http://greenrobot.org/greendao/documentation/objectbox-compat/

greeddao 实现流程图：

image

greendao特点：

1.版本3.0开始，通过注解自动化构建
2.性能好，参考http://greenrobot.org/greendao/features//
3.基于orm 设计，Query builder，方便书写代码。

自动化构建

  在实体的DAO中，您将发现包含数据库表名称的table name
  以及包含所有属性常量的内部类属性（field columnname）。
  能在编译时，就检查错误。

注：操作表的时候，表名和属性最好用常量，或者dao 里面去找。

以下涉及内容：缓存机制，预加载，api使用，线程安全等几个方面讨论。

why 性能好？

1.速度（核心）：[inserted, updated and loaded at several thousand entities per second] 增删改查，每秒几千entities；加上cache，有2万多entities每秒。
对比查询，Greendao使用ID进行的加载操作根本不需要与数据库联系。对缓存实体使用加载操作会导致每秒超过100000个实体查找的速率。
http://greenrobot.org/open-source/current-performance-figures/
对比OrmLite, and ActiveAndroid
greenDAO inserts and updates entities around 2 times faster, and loads entities around 4 times faster than ORMLite

2.session cache 缓存机制
默认多次查询，返回相同的Java objcet。(由identity scope决定)
如果要清除缓存：

daoSession.clear();

只清空当前ID scope
noteDao = daoSession.getNoteDao();
noteDao.detachAll();

实现缓存的方法。dao::loadAllFromCursor()中的关键方法

    if (cursor.moveToFirst()) {
            if (identityScope != null) {
                identityScope.lock();
                identityScope.reserveRoom(count);
            }

            try {
                if (!useFastCursor && window != null && identityScope != null) {
                    loadAllUnlockOnWindowBounds(cursor, window, list);
                } else {
                    do {
                        list.add(loadCurrent(cursor, 0, false));
                    } while (cursor.moveToNext());
                }
            } finally {
                if (identityScope != null) {
                    identityScope.unlock();
                }
            }
        }

只要指针还有数据，往list的添加loadCurrent(cursor, 0, false)

final protected T loadCurrent(Cursor cursor, int offset, boolean lock) {
        if (identityScopeLong != null) {
            if (offset != 0) {
                // Occurs with deep loads (left outer joins)
                if (cursor.isNull(pkOrdinal + offset)) {
                    return null;
                }
            }

            long key = cursor.getLong(pkOrdinal + offset);
            T entity = lock ? identityScopeLong.get2(key) : identityScopeLong.get2NoLock(key);
            if (entity != null) {
                return entity;
            } else {
                entity = readEntity(cursor, offset);
                attachEntity(entity);
                if (lock) {
                    identityScopeLong.put2(key, entity);
                } else {
                    identityScopeLong.put2NoLock(key, entity);
                }
                return entity;
            }
        } else if (identityScope != null) {
            K key = readKey(cursor, offset);
            if (offset != 0 && key == null) {
                // Occurs with deep loads (left outer joins)
                return null;
            }
            T entity = lock ? identityScope.get(key) : identityScope.getNoLock(key);
            if (entity != null) {
                return entity;
            } else {
                entity = readEntity(cursor, offset);
                attachEntity(key, entity, lock);
                return entity;
            }
        } else {
            // Check offset, assume a value !=0 indicating a potential outer join, so check PK
            if (offset != 0) {
                K key = readKey(cursor, offset);
                if (key == null) {
                    // Occurs with deep loads (left outer joins)
                    return null;
                }
            }
            T entity = readEntity(cursor, offset);
            attachEntity(entity);
            return entity;
        }
    }

再看有主键的情况， identityScopeLong.get2(key)

   public T get2(long key) {
        lock.lock();
        Reference ref;
        try {
            ref = map.get(key);
        } finally {
            lock.unlock();
        }
        if (ref != null) {
            return ref.get();
        } else {
            return null;
        }
    }

public void put2(long key, T entity) {
        lock.lock();
        try {
            map.put(key, new WeakReference(entity));
        } finally {
            lock.unlock();
        }
    }

数据缓存是通过identityScopeLong中成员变量map来获取的。现在看identityScopeLong是什么时候初始化的？

伪代码
xxEntityDaoConfig = daoConfigMap.get(xxEntityDao.class).clone();
        xxEntityDaoConfig.initIdentityScope(type);

   public void initIdentityScope(IdentityScopeType type) {
        if (type == IdentityScopeType.None) {
            identityScope = null;
        } else if (type == IdentityScopeType.Session) {
            if (keyIsNumeric) {
                identityScope = new IdentityScopeLong();
            } else {
                identityScope = new IdentityScopeObject();
            }
        } else {
            throw new IllegalArgumentException("Unsupported type: " + type);
        }
    }

由上可以看到，分三种identityScope = null;（identityScope==null）
若为主键，identityScope =IdentityScopeLong；若不为主键identityScope =IdentityScopeObject

参数IdentityScopeType是啥，默认创建的是IdentityScopeType.Session,也可以传NULL. 推荐默认IdentityScopeType.Session，NULL每次都要重新readEntity，创建新的entity，并将获取指针的内容放入entity中，不会获取缓存。

    public DaoSession newSession() {
        return new DaoSession(db, IdentityScopeType.Session, daoConfigMap);
    }

    public DaoSession newSession(IdentityScopeType type) {
        return new DaoSession(db, type, daoConfigMap);
    }

3.eager loading 预加载
daomaster创建的时候，就registerDaoClass，创建各个dao daoconfig()，加载entity表结构。
之后每次新建daosession，就clone之前的dao配置（表结构等）
并初始化initIdentityScope。

  protected void registerDaoClass(Class> daoClass) {
        DaoConfig daoConfig = new DaoConfig(db, daoClass);
        daoConfigMap.put(daoClass, daoConfig);
    }
//daoconfig构造函数
 public DaoConfig(Database db, Class> daoClass) {
        this.db = db;
        try {
            this.tablename = (String) daoClass.getField("TABLENAME").get(null);
            Property[] properties = reflectProperties(daoClass);
            this.properties = properties;

            allColumns = new String[properties.length];

            List pkColumnList = new ArrayList();
            List nonPkColumnList = new ArrayList();
            Property lastPkProperty = null;
            for (int i = 0; i < properties.length; i++) {
                Property property = properties[i];
                String name = property.columnName;
                allColumns[i] = name;
                if (property.primaryKey) {
                    pkColumnList.add(name);
                    lastPkProperty = property;
                } else {
                    nonPkColumnList.add(name);
                }
            }
            String[] nonPkColumnsArray = new String[nonPkColumnList.size()];
            nonPkColumns = nonPkColumnList.toArray(nonPkColumnsArray);
            String[] pkColumnsArray = new String[pkColumnList.size()];
            pkColumns = pkColumnList.toArray(pkColumnsArray);

            pkProperty = pkColumns.length == 1 ? lastPkProperty : null;
            statements = new TableStatements(db, tablename, allColumns, pkColumns);

            if (pkProperty != null) {
                Class type = pkProperty.type;
                keyIsNumeric = type.equals(long.class) || type.equals(Long.class) || type.equals(int.class)
                        || type.equals(Integer.class) || type.equals(short.class) || type.equals(Short.class)
                        || type.equals(byte.class) || type.equals(Byte.class);
            } else {
                keyIsNumeric = false;
            }

        } catch (Exception e) {
            throw new DaoException("Could not init DAOConfig", e);
        }
    }

4.active entities.丰富的API，透明灵活的处理持久层操作，增删改查等。
5.支持数据库加密SQLCipher

DevOpenHelper helper = new DevOpenHelper(this, "notes-db-encrypted.db");
Database db = helper.getEncryptedWritableDb("");
daoSession = new DaoMaster(db).newSession();

测试加密方法：
.getWritableDb() instead of .getEncryptedWritableDb()
加密存在的问题：
1） SQLCipher用了更多的锁，增加了死锁的风险
2）抛出的异常为非android.database.SQLException
3）SQLCipher从3.5.0版本开始正式支持Android N

6.小于100k,对apk size不会产生大的影响
7.开源，社区活跃

为何方便书写

1.代码书写使用builder代替原生sql，具体见http://greenrobot.org/greendao/documentation/queries/
如：
First name is "Joe" AND (year of birth is greater than 1970 OR (year of birth is 1970 AND month of birth is equal to or greater than 10

QueryBuilder qb = userDao.queryBuilder();
qb.where(Properties.FirstName.eq("Joe"),
qb.or(Properties.YearOfBirth.gt(1970),
qb.and(Properties.YearOfBirth.eq(1970), Properties.MonthOfBirth.ge(10))));
List youngJoes = qb.list();

// order by last name
queryBuilder.orderAsc(Properties.LastName);
// in reverse
queryBuilder.orderDesc(Properties.LastName);
// order by last name and year of birth
queryBuilder.orderAsc(Properties.LastName).orderDesc(Properties.YearOfBirth);

** 可以添加个数的限制
[limit(int)]Limits the number of results returned by the query.

[offset(int)]Sets the offset for query results in combination with limit(int). The first offset results are skipped and the total number of results will be limited by limit(int). You cannot use offset withoutlimit(int).
**

2.类型,支持类型转换
默认支持的类型如下：

boolean, Boolean
int, Integer
short, Short
long, Long
float, Float
double, Double
byte, Byte
byte[]
String
Date

其他建议：
1）最好使用数据库原本支持的类型。
2）除了自增长的数据，@Entity最好都用String类型（String 映射到TEXT），否则date等类型在做转换的时候，有可能会出现异常。（或者Date用INTEGER或Long代替）
3）枚举类型映射到int，应该用int类型代替。

4. 使用类型转换,尽量用透明的类型代替，如：
   boolean 映射 INTEGER（0，1）
   Date 映射 (long) INTEGER

@Entity
public class User {
    @Id
    private Long id;

    @Convert(converter = RoleConverter.class, columnType = Integer.class)
    private Role role;

    public enum Role {
        DEFAULT(0), AUTHOR(1), ADMIN(2);
        
        final int id;
        
        Role(int id) {
            this.id = id;
        }
    }

    public static class RoleConverter implements PropertyConverter {
        @Override
        public Role convertToEntityProperty(Integer databaseValue) {
            if (databaseValue == null) {
                return null;
            }
            for (Role role : Role.values()) {
                if (role.id == databaseValue) {
                    return role;
                }
            }
            return Role.DEFAULT;
        }

        @Override
        public Integer convertToDatabaseValue(Role entityProperty) {
            return entityProperty == null ? null : entityProperty.id;
        }
    }
}

3. unique()和uniqueOrThrow()
   unique会返回结果或null，返回单一数据。
   uniqueOrThrow会抛出DaoException

4.list（）listLazy() listLazyUncached() listIterator()
除了list（），其他的方法 必须关闭。正常的使用，
listLazy 使用内存缓存，内存有不会去查库。
listLazyUncached 不用内存缓存，会查库
listIterator，数据不会缓存，使用才会load data。

关闭原因及方法：
To load data on-demand, it holds a reference to a database cursor. This is the reason you must ensure to close the lazy lists and iterators (typically in a try/finally block).

5.builder支持查询多次，复用

// fetch users with Joe as a first name born in 1970
Query query = userDao.queryBuilder().where(
    Properties.FirstName.eq("Joe"), Properties.YearOfBirth.eq(1970)
).build();
List joesOf1970 = query.list();
// using the same Query object, we can change the parameters
// to search for Marias born in 1977 later:
query.setParameter(0, "Maria");
query.setParameter(1, 1977);
List mariasOf1977 = query.list();

6.Raw queries
1）WhereCondition.StringCondition

Query query = userDao.queryBuilder().where(
  new StringCondition("_ID IN " +
    "(SELECT USER_ID FROM USER_MESSAGE WHERE READ_FLAG = 0)")
).build();

2） queryRaw() queryRawCreate() 区别及用法
List queryRaw(String where, String... selectionArg)
Query queryRawCreate(String where, Object... selectionArg)

Query query = userDao.queryRawCreate(
  ", GROUP G WHERE G.NAME=? AND T.GROUP_ID=G._ID", "admin"
);

7.buildDelete()
返回 DeleteQuery，用于之后调用，针对相同规则的entities
注意，bulk deletes不会影响entities，可以恢复。
Keep in mind, that bulk deletes currently do not affect entities in the identity scope, for example you could “resurrect” deleted entities if they have been cached before and are accessed by their ID (load method).

8.调试
可协助打印出sql语句，在其他数据库工具里执行。

QueryBuilder.LOG_SQL = true;
QueryBuilder.LOG_VALUES = true;

9.join :如：一个user多个地址

QueryBuilder queryBuilder = userDao.queryBuilder();
queryBuilder.join(Address.class, AddressDao.Properties.userId)
  .where(AddressDao.Properties.Street.eq("Sesame Street"));
List users = queryBuilder.list();

其他技巧：
1）As a rule of thumb，用application scope操作，保存数据。
2）android sqlite 推荐主键用Long,
若创建唯一索（unique = true）

@Id
private Long id;

@Index(unique = true)
private String key;

3）multiple sessions refer to the same database connection

支持多线程操作

(仅仅指同一个daosession 的query builder以及dao的操作是线程安全的，daosession的创建必须保证时单例同步的，其他地方避免用synchronized)
核心：
i.query builder，通过同步锁和map实现，如果当前线程id
和map中存储的线程ID一致，则复制之前的参数；否则存储在map中。等待下次相同线程调用。(由于有同步锁的存在，建议dao.queryBuilder()最好在同一个方法区里。)
ii.数据缓存，通过dao中的IdentityScopeLong或者IdentityScopeObject来实现。具体通过ReentrantLock来确保线程安全。

  public IdentityScopeObject() {
        map = new HashMap>();
        lock = new ReentrantLock();
    }

1）query builder在创建时会调用**forCurrentThread() **,然后在当前线程创建Query instance 。（这种设置比较安全，原因：
每次调用用forCurrentThread()，the parameters are set to the initial parameters at the time the query was built using its builder.
参数被设置到之前创建query的初始化参数里。）

2）避免用synchronized，竞争性交易（concurrent transactions）会导致死锁

//伪代码，接着看都发生了什么？
dao.queryBuilder()
.where(xx.eq(xx));
.list()

QueryBuilder::list

 public List list() {
        return build().list();
    }

   /**
     * Builds a reusable query object (Query objects can be executed more efficiently than creating a QueryBuilder for
     * each execution.
     */
    public Query build() {
        StringBuilder builder = createSelectBuilder();
        int limitPosition = checkAddLimit(builder);
        int offsetPosition = checkAddOffset(builder);

        String sql = builder.toString();
        checkLog(sql);

        return Query.create(dao, sql, values.toArray(), limitPosition, offsetPosition);
    }

  static  Query create(AbstractDao dao, String sql, Object[] initialValues, int limitPosition,
                                 int offsetPosition) {
        QueryData queryData = new QueryData(dao, sql, toStringArray(initialValues), limitPosition,
                offsetPosition);
        return queryData.forCurrentThread();
    }

每次执行list()--build()--新建QueryData,最终调用QueryData .forCurrentThread()(如果是新的线程就创建Query，否则复制之前配的参数,并返回。通过synchronized，保证各自线程里的querydata互不干扰，从而达到querybuilder复用，且线程安全)

  /**
     * Note: all parameters are reset to their initial values specified in {@link QueryBuilder}.
     */
    Q forCurrentThread() {
        // Process.myTid() seems to have issues on some devices (see Github #376) and Robolectric (#171):
        // We use currentThread().getId() instead (unfortunately return a long, can not use SparseArray).
        // PS.: thread ID may be reused, which should be fine because old thread will be gone anyway.
        long threadId = Thread.currentThread().getId();
        synchronized (queriesForThreads) {
            WeakReference queryRef = queriesForThreads.get(threadId);
            Q query = queryRef != null ? queryRef.get() : null;
            if (query == null) {
                gc();
                query = createQuery();
                queriesForThreads.put(threadId, new WeakReference(query));
            } else {
                System.arraycopy(initialValues, 0, query.parameters, 0, initialValues.length);
            }
            return query;
        }
    }

接着看list（）；

   /** Executes the query and returns the result as a list containing all entities loaded into memory. */
    public List list() {
        checkThread();//检查是否当前线程，否则抛DaoException
        Cursor cursor = dao.getDatabase().rawQuery(sql, parameters);
        return daoAccess.loadAllAndCloseCursor(cursor);
    }

这里的dao.getDatabase()是哪个数据库？回到最开始伪代码调用的地方 dao.queryBuilder()生成QueryBuilder，这里的dao 都是org.greenrobot.greendao.AbstractDao ,最终发现QueryBuilder的成员变量dao就是AbstractDao, 也是QueryData的dao,并通过QueryData ：：createQuery() 获取AbstractDao。

 public QueryBuilder queryBuilder() {
        return QueryBuilder.internalCreate(this);
    }

  protected QueryBuilder(AbstractDao dao, String tablePrefix) {
        this.dao = dao;
        this.tablePrefix = tablePrefix;
        values = new ArrayList