Android 一种在Dalvik虚拟机上多Dex加载优化的方案

在Android源码中，DexFile中有一个方法，其函数原型为：

 native private static int openDexFile(byte[] fileContents);

也就是通过byte数组加载一个Dex，可以达到秒级加载，亲自测了下，如果一个使用Multidex加载的App,第二个Dex如果需要加载耗时2s+，则使用这个函数去加载，只需要300ms以内即可完成加载。

因此可以做的优化就是，app安装后首次加载使用该函数去加载，同时开一个进程使用Multidex加载，当第二次启动的时候，则使用原始的Multidex去加载。这样可以做到：
- 首次加载由2s+的耗时降低到300ms以内
- 首次加载多进程完成Multidex，后续加载通过Multidex加载，耗时10ms以内。

但是这个函数在Android 4.4中java层被删除了，而Native层中的函数还是存在的。因此我们不从Java层中去动手，而是直接从NDK入手。且这种方式不支持art虚拟机。

我们使用Google官方推荐的CMake进行C/C++的编译。本篇文章完整代码的项目地址见：https://github.com/lizhangqu/QuickMultidex

这里就简单介绍一下原理

• 在jni的JNI_OnLoad方法中查找openDexFile函数，获取其指针
• 在jni的JNI_OnLoad方法中注册动态函数，关联java层的native函数。

我们要查找的openDexFile函数在libdvm中，通过dlopen函数获取其指针，然后通过dlsym函数，获取openDexFile的指针。

 //定义
JNINativeMethod *dvm_dalvik_system_DexFile;
void (*openDexFile)(const u4 *args, union JValue *pResult);

JNIEXPORT jint JNICALL
JNI_OnLoad(JavaVM *vm, void *reserved) {
    JNIEnv *env = NULL;
    jint result = -1;

    void *ldvm = (void *) dlopen("libdvm.so", RTLD_LAZY);
    dvm_dalvik_system_DexFile = (JNINativeMethod *) dlsym(ldvm, "dvm_dalvik_system_DexFile");
    if (0 == lookup(dvm_dalvik_system_DexFile, "openDexFile", "([B)I",
                    &openDexFile)) {
        openDexFile = NULL;
        return result;
    }
    if ((*vm)->GetEnv(vm, (void **) &env, JNI_VERSION_1_6) != JNI_OK) {
        return result;
    }
    return JNI_VERSION_1_6;
}
 int lookup(JNINativeMethod *table, const char *name, const char *sig,
           void (**fnPtrout)(u4 const *, union JValue *)) {
    int i = 0;
    while (table[i].name != NULL) {
        LOGD("lookup %d %s", i, table[i].name);
        if ((strcmp(name, table[i].name) == 0)
            && (strcmp(sig, table[i].signature) == 0)) {
            *fnPtrout = table[i].fnPtr;
            return 1;
        }
        i++;
    }
    return 0;
}

关于第二步，java函数和jni函数的关联，你可以使用静态注册，即遵守jni的标准即可。这里使用了动态注册方式，即在JNI_OnLoad完成java和jni函数的关联。

首先声明java函数：

public class Multidex {
    static {
        System.loadLibrary("multidex");
    }

    public static int openDexFile(byte[] dexBytes) throws Exception {
        return openDexFile(dexBytes, dexBytes.length);
    }

    /*
     * Open a DEX file based on a {@code byte[]}. The value returned
     * is a magic VM cookie. On failure, a RuntimeException is thrown.
     */
    private native static int openDexFile(byte[] fileContents, long length);
}

进行注册

JNIEXPORT jint JNICALL
JNI_OnLoad(JavaVM *vm, void *reserved) {
    //....省略n行代码
    if ((*vm)->GetEnv(vm, (void **) &env, JNI_VERSION_1_6) != JNI_OK) {
        return result;
    }
    if (registerNatives(env) != JNI_TRUE) {
        return result;
    }
    return JNI_VERSION_1_6;
}

registerNatives函数的实现如下：

static JNINativeMethod methods[] = {
        {"openDexFile", "([BJ)I", (void *) Multidex_openDexFile}
};
static const char *classPathName = "com/android/quickmultidex/Multidex";

static int registerNativeMethods(JNIEnv *env, const char *className,
                                 JNINativeMethod *gMethods, int numMethods) {
    jclass clazz;

    clazz = (*env)->FindClass(env, className);
    if (clazz == NULL) {
        return JNI_FALSE;
    }
    if ((*env)->RegisterNatives(env, clazz, gMethods, numMethods) < 0) {
        return JNI_FALSE;
    }

    return JNI_TRUE;
}

static int registerNatives(JNIEnv *env) {
    if (!registerNativeMethods(env, classPathName,
                               methods, sizeof(methods) / sizeof(methods[0]))) {
        return JNI_FALSE;
    }
    return JNI_TRUE;
}

最终和java关联的函数为Multidex_openDexFile函数，其函数原型如下:

JNIEXPORT jint JNICALL Multidex_openDexFile(
        JNIEnv *env, jclass jv, jbyteArray dexArray, jlong dexLen) {

}

在这个函数中，我们就需要调用系统的openDexFile函数，获取加载Dex的cookie。这个函数中比较关键的一个问题就是如何构造入参，即ArrayObject相关参数的构造，关于这个构造，请参考一下几篇文章：

• http://taoyuanxiaoqi.com/2015/01/16/apkshell2/
• http://bbs.pediy.com/showthread.php?t=205577
• http://bbs.pediy.com/showthread.php?t=195865
• http://bbs.pediy.com/showthread.php?p=1256698
• http://2013.hackitoergosum.org/presentations/Day1-05.Nifty%20stuff%20that%20you%20can%20still%20do%20with%20Android%20by%20Xavier%20Martin.pdf

看完了上面几篇文章，还有一个问题，就是ArrayObject对象中的contents的偏移，该偏移在arm上是16，在x86上是12，因此需要宏来辅助定义，如下：

#if defined(__i386__)
#define array_object_contents_offset 12
#else
#define array_object_contents_offset 16
#endif

然后就是大小端的判断，大小端也是通过宏来定义

#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#define HAVE_LITTLE_ENDIAN

int getEndian() {
    return 1;
}

#else
#define HAVE_BIG_ENDIAN
int getEndian(){
    return 0;
}
#endif

最后就是所需入参的数据结构构造

#if defined(HAVE_ENDIAN_H)
# include 

#else /*not HAVE_ENDIAN_H*/
# define __BIG_ENDIAN 4321
# define __LITTLE_ENDIAN 1234
# if defined(HAVE_LITTLE_ENDIAN)
#  define __BYTE_ORDER __LITTLE_ENDIAN
# else
#  define __BYTE_ORDER __BIG_ENDIAN
# endif
#endif /*not HAVE_ENDIAN_H*/


//数据结构构造定义
typedef uint8_t u1;
typedef uint16_t u2;
typedef uint32_t u4;
typedef uint64_t u8;
typedef int8_t s1;
typedef int16_t s2;
typedef int32_t s4;
typedef int64_t s8;

union JValue {
#if defined(HAVE_LITTLE_ENDIAN)
    u1 z;
    s1 b;
    u2 c;
    s2 s;
    s4 i;
    s8 j;
    float f;
    double d;
    void *l;
#endif
#if defined(HAVE_BIG_ENDIAN)
    struct {
        u1 _z[3];
        u1 z;
    };
    struct {
        s1 _b[3];
        s1 b;
    };
    struct {
        u2 _c;
        u2 c;
    };
    struct {
        s2 _s;
        s2 s;
    };
    s4 i;
    s8 j;
    float f;
    double d;
    void *l;
#endif
};

typedef struct {
    void *clazz;
    u4 lock;
    u4 length;
    u1 *contents;
} ArrayObject;

最关键的地方就是Multidex_openDexFile函数的实现,具体如何构造可参考上面的文章

JNIEXPORT jint JNICALL Multidex_openDexFile(
        JNIEnv *env, jclass jv, jbyteArray dexArray, jlong dexLen) {
    LOGD("array_object_contents_offset: %d", array_object_contents_offset);
    u1 *dexData = (u1 *) (*env)->GetByteArrayElements(env, dexArray, NULL);
    char *arr;
    arr = (char *) malloc((size_t) (array_object_contents_offset + dexLen));
    ArrayObject *ao = (ArrayObject *) arr;
    ao->length = (u4) dexLen;
    memcpy(arr + array_object_contents_offset, dexData, dexLen);
    u4 args[] = {(u4) ao};
    union JValue pResult;
    jint result = -1;
    if (openDexFile != NULL) {
        openDexFile(args, &pResult);
        result = (jint) pResult.l;
    }
    return result;
}

这样，我们就获取到了通过byte数组加载Dex后返回的cookie，通过这个cookie我们就可以去查找Dex中的类。

• 首先获取到Dex的字节数组
• 其次调用native方法将字节数组传入返回cookie
• 利用cookie构造DexFile

• 将DexFile插入到Classloader中

  第一步，可改造Multidex代码，将其解压Dex的代码进行改造，返回byte数组，改造后的代码如下:

    private static final String DEX_PREFIX = "classes";
    private static final String DEX_SUFFIX = ".dex";
    private static final int MAX_EXTRACT_ATTEMPTS = 3;

    private static List<byte[]> performExtractions(String sourceApk)
            throws IOException {
        List<byte[]> dexDatas = new ArrayList<byte[]>();

        final ZipFile apk = new ZipFile(sourceApk);
        try {
            int secondaryNumber = 2;
            ZipEntry dexFile = apk.getEntry(DEX_PREFIX + secondaryNumber + DEX_SUFFIX);
            while (dexFile != null) {
                int numAttempts = 0;
                boolean isExtractionSuccessful = false;
                while (numAttempts < MAX_EXTRACT_ATTEMPTS && !isExtractionSuccessful) {
                    numAttempts++;
                    byte[] extract = extract(apk, dexFile);
                    if (extract == null) {
                        isExtractionSuccessful = false;
                    } else {
                        dexDatas.add(extract);
                        isExtractionSuccessful = true;
                    }
                }
                if (!isExtractionSuccessful) {
                    throw new IOException("Could not create extra file " +
                            " for secondary dex (" +
                            secondaryNumber + ")");
                }
                secondaryNumber++;
                dexFile = apk.getEntry(DEX_PREFIX + secondaryNumber + DEX_SUFFIX);
            }
            return dexDatas;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                apk.close();
            } catch (IOException e) {
                e.printStackTrace();
                Log.e(TAG, "Failed to close resource", e);
            }
        }
        return null;
    }

    private static byte[] extract(ZipFile apk, ZipEntry dexFile) throws IOException {
        InputStream input = apk.getInputStream(dexFile);
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        byte[] buffer = new byte[4096];
        int n = 0;
        while (-1 != (n = input.read(buffer))) {
            output.write(buffer, 0, n);
        }
        closeQuietly(output);
        closeQuietly(input);
        return output.toByteArray();
    }

    private static void closeQuietly(Closeable closeable) {
        try {
            closeable.close();
        } catch (IOException e) {
            Log.w(TAG, "Failed to close resource", e);
        }
    }

第二步，将byte数组转换为cookie

    private static List loadDex(Context context) throws Exception {

        ArrayList list = new ArrayList<>();

        ApplicationInfo applicationInfo = context.getApplicationInfo();
        String sourceDir = applicationInfo.sourceDir;

        List<byte[]> dexByteslist = performExtractions(sourceDir);
        if (dexByteslist != null && dexByteslist.size() > 0) {
            for (byte[] dexBytes : dexByteslist) {
                int i = openDexFile(dexBytes);
                Log.e(TAG, "loadDex openDexFile cookie:" + i);
                list.add(i);
            }
        } else {
            Log.e(TAG, "loadDex performExtractions null");

        }
        return list;
    }

第三、四步，构造DexFile，这一步比较绕，主要原理就是通过DexPathList的makeDexElements函数，传参数为app的apk包路径，构造一个dexElements出来，然后将该dexElements的所有参数设置为null（除了dexFile），然后将dexFile获取到，设置没有用的参数为null，设置cookie为获取到的cookie，然后插入到classloader中去，怎么插入，和multidex是一样的。

    public static boolean inject(Context base, List cookies) {
        try {
            ApplicationInfo applicationInfo = base.getApplicationInfo();
            String sourceDir = applicationInfo.sourceDir;

            Field pathListField = findField(base.getClassLoader(), "pathList");
            Object pathList = pathListField.get(base.getClassLoader());

            Method makeDexElements = null;
            if (Build.VERSION.SDK_INT < 19) {
                makeDexElements =
                        findMethod(pathList, "makeDexElements", ArrayList.class, File.class);

            } else {
                makeDexElements =
                        findMethod(pathList, "makeDexElements", ArrayList.class, File.class,
                                ArrayList.class);

            }
            Object[] invokeElements = null;
            ArrayList files = new ArrayList<>();
            for (int i = 0; i < cookies.size(); i++) {
                files.add(new File(sourceDir));
            }
            if (Build.VERSION.SDK_INT < 19) {
                invokeElements = (Object[]) makeDexElements.invoke(pathList, files, null);
            } else {
                invokeElements = (Object[]) makeDexElements.invoke(pathList, files, null, null);
            }

            Field dexElementsFiled = Multidex.findField(pathList, "dexElements");
            Object[] originalDexElements = (Object[]) dexElementsFiled.get(pathList);
            Object[] resultDexElements = (Object[]) Array.newInstance(originalDexElements.getClass().getComponentType(), originalDexElements.length + invokeElements.length);

            System.arraycopy(originalDexElements, 0, resultDexElements, 0, originalDexElements.length);
            System.arraycopy(invokeElements, 0, resultDexElements, originalDexElements.length, invokeElements.length);

            int length = originalDexElements.length;
            for (int i = 0; i < cookies.size(); i++) {
                Object dexElements = resultDexElements[length + i];
                Field fileField = Multidex.findField(dexElements, "file");
                fileField.set(dexElements, null);
                Field zipField = Multidex.findField(dexElements, "zip");
                zipField.set(dexElements, null);
                Field zipFileField = Multidex.findField(dexElements, "zipFile");
                zipFileField.set(dexElements, null);
                Field dexFileField = Multidex.findField(dexElements, "dexFile");
                Object o = dexFileField.get(dexElements);
                Field mCookieField = Multidex.findField(o, "mCookie");
                mCookieField.set(o, cookies.get(i));
                Field mFileNameFiled = Multidex.findField(o, "mFileName");
                mFileNameFiled.set(o, null);
            }

            dexElementsFiled.set(pathList, resultDexElements);
            return true;
        } catch (NoSuchFieldException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (NoSuchMethodException e) {
            e.printStackTrace();
        } catch (InvocationTargetException e) {
            e.printStackTrace();
        }
        return false;
    }

    public static Field findField(Object instance, String name) throws NoSuchFieldException {
        Class clazz = instance.getClass();
        while (clazz != null) {
            try {
                Field e = clazz.getDeclaredField(name);
                if (!e.isAccessible()) {
                    e.setAccessible(true);
                }

                return e;
            } catch (NoSuchFieldException var4) {
                clazz = clazz.getSuperclass();
            }
        }

        throw new NoSuchFieldException("Field " + name + " not found in " + instance.getClass());
    }

    public static Method findMethod(Object instance, String name, Class... parameterTypes) throws NoSuchMethodException {
        Class clazz = instance.getClass();

        while (clazz != null) {
            try {
                Method e = clazz.getDeclaredMethod(name, parameterTypes);
                if (!e.isAccessible()) {
                    e.setAccessible(true);
                }

                return e;
            } catch (NoSuchMethodException var5) {
                clazz = clazz.getSuperclass();
            }
        }

        throw new NoSuchMethodException("Method " + name + " with parameters " + Arrays.asList(parameterTypes) + " not found in " + instance.getClass());
    }

最后就是一个对外暴露的函数install

    private static final String TAG = "Multidex";

    public static boolean install(Context context) {
        try {
            long start = System.nanoTime();
            boolean ret = false;
            long startLoadDexData = System.nanoTime();
            List cookies = loadDex(context);
            long endLoadDexData = System.nanoTime();
            Log.e(TAG, "loadDexData time：" + (endLoadDexData - startLoadDexData) + " ns");
            Log.e(TAG, "loadDexData time：" + (endLoadDexData - startLoadDexData) / 1000000 + " ms");

            if (cookies != null && cookies.size() > 0) {
                long startInject = System.nanoTime();
                boolean result = inject(context, cookies);
                long endInject = System.nanoTime();
                Log.e(TAG, "inject time：" + (endInject - startInject) + " ns");
                Log.e(TAG, "inject time：" + (endInject - startInject) / 1000000 + " ms");
                ret = result;
            } else {
                ret = false;
            }
            Log.e(TAG, "install result:" + ret);
            long end = System.nanoTime();
            Log.e(TAG, "install time：" + (end - start) + " ns");
            Log.e(TAG, "install time：" + (end - start) / 1000000 + " ms");
            return ret;
        } catch (Exception e) {
            e.printStackTrace();
            throw new RuntimeException(e);
        }
    }

但是事实并没有那么完美，当你的项目中使用了java.lang.Class.getTypeParameters()等函数时，就会在Android 4.4上crash掉，这个原因可以见

• http://www.jacpy.com/2015/05/10/android-4-4-call-class-gettypeparameters-occur-jni-function-newglobalref-called-with-exception-pending.html

因此本篇文章的适用范围为Android 4.1~4.3