Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃

背景

某一天,测试部大哥从当前最新的软件版本降级到一个月前的版本,然后启动就出现崩溃了。

异常如下:

java.lang.IllegalStateException: A migration from 4 to 3 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.
	at androidx.room.RoomOpenHelper.onUpgrade(RoomOpenHelper.java:117)
	at androidx.room.RoomOpenHelper.onDowngrade(RoomOpenHelper.java:129)
	at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.onDowngrade(FrameworkSQLiteOpenHelper.java:135)
	at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked(SQLiteOpenHelper.java:254)
	at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:163)
	at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.getWritableSupportDatabase(FrameworkSQLiteOpenHelper.java:92)
	at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper.getWritableDatabase(FrameworkSQLiteOpenHelper.java:53)
	at androidx.room.RoomDatabase.inTransaction(RoomDatabase.java:476)
	at androidx.room.RoomDatabase.assertNotSuspendingTransaction(RoomDatabase.java:281)

等等?你跟我说降级?现在有哪个App支持降级的吗?别着急,有句话说的好,只要思想不滑坡,方法总比困难多,降级的肯定是可以降级的,不过这不是重点,今天主要分享下数据库降级导致的问题如何使用ASM修改字节码的方式来解决。

乍一看,这不是很简单吗,添加对应的降级Migration或者配置fallbackToDestructiveMigration方法进行破坏性重建数据库即可。

再看下RoomOpenHelper对应的源码:

    @Override
    public void onUpgrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) {
        boolean migrated = false;
        if (mConfiguration != null) {
            List migrations = mConfiguration.migrationContainer.findMigrationPath(
                    oldVersion, newVersion);
            if (migrations != null) {
                mDelegate.onPreMigrate(db);
                for (Migration migration : migrations) {
                    migration.migrate(db);
                }
                ValidationResult result = mDelegate.onValidateSchema(db);
                if (!result.isValid) {
                    throw new IllegalStateException("Migration didn't properly handle: "
                            + result.expectedFoundMsg);
                }
                mDelegate.onPostMigrate(db);
                updateIdentity(db);
                migrated = true;
            }
        }
        if (!migrated) {
            if (mConfiguration != null
                    && !mConfiguration.isMigrationRequired(oldVersion, newVersion)) {
                mDelegate.dropAllTables(db);
                mDelegate.createAllTables(db);
            } else {
                throw new IllegalStateException("A migration from " + oldVersion + " to "
                        + newVersion + " was required but not found. Please provide the "
                        + "necessary Migration path via "
                        + "RoomDatabase.Builder.addMigration(Migration ...) or allow for "
                        + "destructive migrations via one of the "
                        + "RoomDatabase.Builder.fallbackToDestructiveMigration* methods.");
            }
        }
    }

没错了,既然是没添加对应Migration,那加上不就完了吗?说的好的,确实如此,不过要是这样就结束了,还怎么秀操作?而且为啥不用fallbackToDestructiveMigration?不能用的原因很简单,领导说了,不管是数据库降级还是还是升级,数据都必须要保留。fallbackToDestructiveMigration的操作就是删表重建,这样是不满足要求的。

开干,一顿操作猛如虎,三下五除二就把降级Migration的加上了。

  private fun buildDatabase(context: Context): TestDataBase {

            return Room.databaseBuilder(context, TestDataBase::class.java, "test.db")
                    .addMigrations(MIGRATION_1_2
                    .addMigrations(MIGRATION_2_1)
                    .addCallback(object : RoomDatabase.Callback() {
                        override fun onCreate(db: SupportSQLiteDatabase) {
                            super.onCreate(db)
                            Log.i(TAG, "onCreate")
                        }

                        override fun onOpen(db: SupportSQLiteDatabase) {
                            super.onOpen(db)
                            Log.i(TAG, "onOpen")
                        }
                    })
                    .build()
        }
			
      /**
         * 版本升级1->2
         */
        private val MIGRATION_1_2 = object : Migration(1, 2) {
            override fun migrate(database: SupportSQLiteDatabase) {
                database.execSQL("ALTER TABLE Test ADD COLUMN updatetime TEXT")
            }
        }

        /**
         * 版本升级2->1空实现,防止降级崩溃
         */
        private val MIGRATION_2_1 = object : Migration(2, 1) {
            override fun migrate(database: SupportSQLiteDatabase) {
                Log.i(TAG, "migrate: MIGRATION_2_1 do nothing")
            }
        }

       

重新打包给到测试,嘿,没问题,禅道马上关闭bug。心里美滋滋,把这个问题反馈给领导,让其它项目都修改下这个问题。但是项目这么多,涉及到Room数据库的都要手动改一遍,岂不是很麻烦?有没有更优雅的方式,最好是其它项目都不用动,躺着就把问题解决了。

好家伙,提高难度了,看来要秀出看家本领才行。

再回头看下RoomOpenHelper的源码:

@Override
    public void onUpgrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) {
        boolean migrated = false;
        if (mConfiguration != null) {
            List migrations = mConfiguration.migrationContainer.findMigrationPath(
                    oldVersion, newVersion);
            if (migrations != null) {
                mDelegate.onPreMigrate(db);
                for (Migration migration : migrations) {
                    migration.migrate(db);
                }
                ValidationResult result = mDelegate.onValidateSchema(db);
                if (!result.isValid) {
                    throw new IllegalStateException("Migration didn't properly handle: "
                            + result.expectedFoundMsg);
                }
                mDelegate.onPostMigrate(db);
                updateIdentity(db);
                migrated = true;
            }
        }
        if (!migrated) {
            if (mConfiguration != null
                    && !mConfiguration.isMigrationRequired(oldVersion, newVersion)) {
                mDelegate.dropAllTables(db);
                mDelegate.createAllTables(db);
            } else {
                throw new IllegalStateException("A migration from " + oldVersion + " to "
                        + newVersion + " was required but not found. Please provide the "
                        + "necessary Migration path via "
                        + "RoomDatabase.Builder.addMigration(Migration ...) or allow for "
                        + "destructive migrations via one of the "
                        + "RoomDatabase.Builder.fallbackToDestructiveMigration* methods.");
            }
        }
    }

    @Override
    public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) {
        onUpgrade(db, oldVersion, newVersion);
    }

不看不知道,官方这骚操作,onDowngrade竟然是调用的onUpgrade,我直呼内行。那对应的解决办法也很简单,重写下onDowngrade方法不要调用onUpgrade那不就可以了?先试试?试试就试试。

ASM的修改字节码

按照Greendao或者SupportSQLiteOpenHelper的经验,onDowngrade是可以直接重写,但是到了Room这里,看起来就没办法了,RoomOpenHelper的实例是在对应的XXXDataBaseImpl创建的,根本不给机会啊,而且XXXDataBaseImpl实现类是编译时Room的注解处理器生成的。

/**
 * An open helper that holds a reference to the configuration until the database is opened.
 *
 * @hide
 */
@SuppressWarnings("unused")
@RestrictTo(RestrictTo.Scope.LIBRARY_GROUP_PREFIX)
public class RoomOpenHelper extends SupportSQLiteOpenHelper.Callback {
	...
}

@Override
  protected SupportSQLiteOpenHelper createOpenHelper(DatabaseConfiguration configuration) {
    final SupportSQLiteOpenHelper.Callback _openCallback = new RoomOpenHelper(configuration, new RoomOpenHelper.Delegate(5) {
    	}
    }

怎么办?ASM就派上用场了。

注意:本文不赘述ASM的详细用法,只讲解针对以上问题,如何使用ASM解决,不熟悉AMS或者对ASM有兴趣的朋友可以参考lsieun 大佬的《Java ASM系列》教程

ASM的官网地址:ASM

既然是使用ASM,那我们肯定是希望在项目编译时去动态修改RoomOpenHelper的class文件,达到重写onDowngrade的目的。

开始前需要具备自定义Gradle Plugins的知识,具体可参考:

Android Gradle Plugins系列-01-自定义Gradle插件入门指南

当然,如果只是单纯看看ASM是如何解决问题的也不强求必须懂自定义Gradle Plugins,只查看ASM相关的内容即可。

废话不多说,新建一个工程,添加ASM依赖如下:

dependencies {

    implementation 'androidx.appcompat:appcompat:1.3.1'
    implementation 'com.google.android.material:material:1.3.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'

    implementation("androidx.room:room-runtime:2.3.0")
    annotationProcessor("androidx.room:room-compiler:2.3.0")

    implementation group: 'org.ow2.asm', name: 'asm', version: '7.2'
    implementation group: 'org.ow2.asm', name: 'asm-commons', version: '7.2'
}

安装ASM Bytecode 插件

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第1张图片

为了方便修改字节码,我们需要借助第三方插件,如果使用最新的AndroidStudio版本或者使用了Kotlin建议安装ASM Bytecode Viewer Support Kotlin这个插件。如果安装了另外两个插件,可能会启动报错或者无法使用,解决办法是卸载不能用的插件,重新安装ASM Bytecode Viewer Support Kotlin插件。关于如何卸载插件,某度搜索下就有相关教程了。

单元测试

为方便了调试,我们先使用单元测试来验证ASM修改字节码是够能达到期望的效果。在单元测试的包中新增两个类:ASMTest和RoomOpenHelper。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第2张图片

RoomOpenHelper代码如下:

/**
 * 模拟Room的RoomOpenHelper类
 */
public class RoomOpenHelper {

    public void onUpgrade(SupportSQLiteDatabase db) {

    }

    public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) {
        onUpgrade(db);
    }

    public void updateIdentity(SupportSQLiteDatabase db) {

    }

    public static class SupportSQLiteDatabase {

    }

}

ASMTest代码如下:

package com.nxg.asm;

import org.junit.Test;
import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.ClassWriter;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;
import org.objectweb.asm.commons.AdviceAdapter;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;

public class ASMTest {

    @Test
    public void test() {

        //1、准备待分析的class
        FileInputStream fis;
        try {
            fis = new FileInputStream("/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm/RoomOpenHelper.class");
            //2、执行分析与插桩
            //class字节码的读取与分析引擎
            ClassReader cr = new ClassReader(fis);
            // 写出器 COMPUTE_FRAMES 自动计算所有的内容,后续操作更简单
            ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
            //分析,处理结果写入cw EXPAND_FRAMES:栈图以扩展格式进行访问
            cr.accept(new ClassAdapterVisitor(cw), ClassReader.EXPAND_FRAMES);


            //3、获得结果并输出
            byte[] newClassBytes = cw.toByteArray();
            FileOutputStream fos = new FileOutputStream
                    ("/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm/DstRoomOpenHelper.class");
            fos.write(newClassBytes);
            fos.close();
        } catch (IOException e) {
            e.printStackTrace();
        }


    }

    public static class ClassAdapterVisitor extends ClassVisitor {

        public ClassAdapterVisitor(ClassVisitor cv) {
            super(Opcodes.ASM7, cv);
        }

        @Override
        public MethodVisitor visitMethod(int access, String name, String desc, String signature,
                                         String[] exceptions) {
            System.out.println("方法:" + name + " 签名:" + desc);
            //遇到onDowngrade方法,就是返回对应的MethodAdapterVisitor
            if ("onDowngrade".equals(name)) {
                MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions);
                return new MethodAdapterVisitor(api, mv, access, name, desc);
            }
            return super.visitMethod(access, name, desc, signature, exceptions);
        }
    }

    public static class MethodAdapterVisitor extends AdviceAdapter {

        protected MethodAdapterVisitor(int api, MethodVisitor methodVisitor, int access, String name, String descriptor) {
            super(api, methodVisitor, access, name, descriptor);
        }

        /**
         * 访问方法指令,方法指令是调用方法的指令。
         * 参数:
         * opcode – 要访问的类型指令的操作码。 此操作码是 INVOKEVIRTUAL、INVOKESPECIAL、INVOKESTATIC 或 INVOKEINTERFACE。
         * owner – 方法所有者类的内部名称(请参阅 Type.getInternalName())。
         * name - 方法的名称。
         * 描述符 – 方法的描述符(请参阅类型)。
         * isInterface – 如果方法的所有者类是接口。
         */
        @Override
        public void visitMethodInsn(int opcodeAndSource, String owner, String name, String descriptor, boolean isInterface) {
            System.out.println("opcodeAndSource:" + opcodeAndSource + ", owner:" + owner + ", name:" + name + ", descriptor:" + descriptor);
            //移除onUpgrade方法调用
            if (opcodeAndSource == INVOKEVIRTUAL && "onUpgrade".equals(name)) {
                return;
            }
            super.visitMethodInsn(opcodeAndSource, owner, name, descriptor, isInterface);


        }

        @Override
        protected void onMethodEnter() {
            super.onMethodEnter();

        }

        @Override
        protected void onMethodExit(int opcode) {
            super.onMethodExit(opcode);

        }
    }
}

Javac生成对应的class文件

由于ASM操作的是class文件,所以需要先编译生成对应的class文件。进入RoomOpenHelper所在目录,在终端执行javac RoomOpenHelper.java

root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ ll
total 20
drwxrwxr-x 2 lb lb 4096 11月  5 16:43 ./
drwxrwxr-x 3 lb lb 4096 11月  5 14:31 ../
-rw-rw-r-- 1 lb lb 3578 11月  5 16:15 ASMTest.java
-rw-rw-r-- 1 lb lb  372 11月  5 14:31 ExampleUnitTest.java
-rw-rw-r-- 1 lb lb  400 11月  5 16:22 RoomOpenHelper.java
root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ javac RoomOpenHelper.java 
root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ ll
total 28
drwxrwxr-x 2 lb lb 4096 11月  5 16:43  ./
drwxrwxr-x 3 lb lb 4096 11月  5 14:31  ../
-rw-rw-r-- 1 lb lb 3578 11月  5 16:15  ASMTest.java
-rw-rw-r-- 1 lb lb  372 11月  5 14:31  ExampleUnitTest.java
-rw-rw-r-- 1 lb lb  616 11月  5 16:43  RoomOpenHelper.class
-rw-rw-r-- 1 lb lb  400 11月  5 16:22  RoomOpenHelper.java
-rw-rw-r-- 1 lb lb  323 11月  5 16:43 'RoomOpenHelper$SupportSQLiteDatabase.clas'
root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ 

可以看到,生成了RoomOpenHelper.class文件如下:

//
// Source code recreated from a .class file by IntelliJ IDEA
// (powered by FernFlower decompiler)
//

package com.nxg.asm;

public class RoomOpenHelper {
    public RoomOpenHelper() {
    }

    public void onUpgrade(RoomOpenHelper.SupportSQLiteDatabase var1) {
    }

    public void onDowngrade(RoomOpenHelper.SupportSQLiteDatabase var1, int var2, int var3) {
        this.onUpgrade(var1);
    }

    public void updateIdentity(RoomOpenHelper.SupportSQLiteDatabase var1) {
    }

    public static class SupportSQLiteDatabase {
        public SupportSQLiteDatabase() {
        }
    }
}

此类模拟了Room库的RoomOpenHelper类,可以看到onDowngrade是调用了onUpgrade的,我们的目标是改成这样:

 public void onDowngrade(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1, int var2, int var3) {
        
    }

测试ASM修改字节码

现在来运行下ASMTest,点下图中绿色三角形即可:

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第3张图片

运行完毕,包中多了一个DstRoomOpenHelper.class,打开看它的源码:

//
// Source code recreated from a .class file by IntelliJ IDEA
// (powered by FernFlower decompiler)
//

package com.nxg.asm;

public class RoomOpenHelper {
    public RoomOpenHelper() {
    }

    public void onUpgrade(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1) {
    }

    public void onDowngrade(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1, int var2, int var3)     {
    
    }

    public void updateIdentity(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1) {
    }
}

Nice,目标达成。我们看下ASMTest的关键代码部分:

public static class MethodAdapterVisitor extends AdviceAdapter {

        protected MethodAdapterVisitor(int api, MethodVisitor methodVisitor, int access, String name, String descriptor) {
            super(api, methodVisitor, access, name, descriptor);
        }

        /**
         * 访问方法指令。 方法指令是调用方法的指令。
         * 参数:
         * opcode – 要访问的类型指令的操作码。 此操作码是 INVOKEVIRTUAL、INVOKESPECIAL、INVOKESTATIC 或 INVOKEINTERFACE。
         * owner – 方法所有者类的内部名称(请参阅 Type.getInternalName())。
         * name - 方法的名称。
         * 描述符 – 方法的描述符(请参阅类型)。
         * isInterface – 如果方法的所有者类是接口。
         */
        @Override
        public void visitMethodInsn(int opcodeAndSource, String owner, String name, String descriptor, boolean isInterface) {
            System.out.println("opcodeAndSource:" + opcodeAndSource + ", owner:" + owner + ", name:" + name + ", descriptor:" + descriptor);
            //移除代码
            if (opcodeAndSource == INVOKEVIRTUAL && "onUpgrade".equals(name)) {
                return;
            }
            super.visitMethodInsn(opcodeAndSource, owner, name, descriptor, isInterface);


        }
    }

看起来很简单,由于我们修改的是类的方法,所以使用MethodAdapterVisitor。其中visitMethodInsn可以访问方法指令。

我们的目标是删除onDowngrade方法的onUpgrade调用,我们看下代码是怎么写的?

 //移除代码
 if (opcodeAndSource == INVOKEVIRTUAL && "onUpgrade".equals(name)) {
 				return;
}

就这么简单?就这么简单。一开始笔者也不知道如何删除某个方法内的某句方法调用,直到看到这篇文章深入探索编译插桩技术(四、ASM 探秘)底部的pengion网友的评论,经过一番测试,终于得以实现,感谢这些大佬的无私奉献。

文章评论如下:

pengion

android1年前

求解,怎么在事件模型下替换或删除方法内的某条语句呢?目前看来似乎只能增加代码

1年前

pengion

懂了,看了官方文档,原来是在继承MethodVisitor类的方法里遇到对应语句return就行了

笔者也去官网翻了翻文档,但是没有找到相关说明。

自定义Gradle插件

ASM的代码写好了,就可以拿过来用了,接下来就是编写自定义Gradle插件,在编译期对Room库的RoomOpenHelper类进行字节码修改达到我们的目标。

阅读到此,建议先储备自定义Gradle Plugins的知识,具体可参考:

Android Gradle Plugins系列-01-自定义Gradle插件入门指南

新建Android Library Module

ModuleName和Package 根据需要修改,这里笔者选择Kotlin语言,Bytecode Level选择8。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第4张图片

删掉不必要的包和文件。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第5张图片

配置implementation-class

按照以下路径,建立对应的目录和文件

module/src-main-resources/META-INF/gradle-plugins/asm-gradle-plugin.properties

文件内容如下:

implementation-class=com.nxg.plugins.ASMGradlePlugin

用于配置插件的实现类,这样Gradle才能实例化插件。

新建ASMGradlePlugin插件类

在 com.nxg.plugins包中新建ASMGradlePlugin类实现Plugin接口:

package com.nxg.plugins;

import com.android.build.gradle.AppExtension;

import org.gradle.api.Plugin;
import org.gradle.api.Project;
import org.jetbrains.annotations.NotNull;

public class ASMGradlePlugin implements Plugin {

    @Override
    public void apply(@NotNull Project project) {
       
    }
}

一个简单的啥都不能干的插件就完成了。

配置插件的build.gradle

plugins {
    id 'maven-publish'
    id 'java-library'
    id 'kotlin'
    id 'groovy'
}

java {
    sourceCompatibility = JavaVersion.VERSION_1_8
    targetCompatibility = JavaVersion.VERSION_1_8
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation "com.android.tools.build:gradle:3.5.0"
    implementation gradleApi()
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.5.30"
    implementation group: 'org.ow2.asm', name: 'asm', version: '7.2'
    implementation group: 'org.ow2.asm', name: 'asm-commons', version: '7.2'
}


//定义Maven仓库信息
def MAVEN_GROUP_ID = "com.nxg.plugins"
def MAVEN_ARTIFACT_ID = "asm-gradle-plugin"
def MAVEN_VERSION = "1.0.0"

publishing {
    publications {
        java(MavenPublication) {
            from components.java
            groupId = MAVEN_GROUP_ID
            artifactId = MAVEN_ARTIFACT_ID
            version = MAVEN_VERSION
        }
    }

    repositories {
        mavenLocal()
        maven {
            // change to point to your repo, e.g. http://my.org/repo
            url = layout.buildDirectory.dir('repo')
        }
    }
}

就不多说了,maven-publish和asm都是需要的。

编写ASM代码

package com.nxg.plugins

import com.android.build.api.transform.*
import com.android.build.gradle.internal.pipeline.TransformManager
import com.android.utils.FileUtils
import org.gradle.api.Project
import org.objectweb.asm.*
import org.objectweb.asm.commons.AdviceAdapter
import java.io.*
import java.util.zip.ZipEntry
import java.util.zip.ZipFile
import java.util.zip.ZipOutputStream


const val METHOD_ON_DOWNGRADE = "onDowngrade"
const val METHOD_ON_UPGRADE = "onUpgrade"
const val CLASS_ROOM_OPEN_HELPER = "androidx/room/RoomOpenHelper.class"

/**
 * 访问class
 */
class RoomOpenHelperClassVisitor constructor(
    classWriter: ClassWriter,
    private val className: String
) : ClassVisitor(Opcodes.ASM6, classWriter) {

    private var mClassName: String? = null

    override fun visit(
        version: Int,
        access: Int,
        name: String?,
        signature: String?,
        superName: String?,
        interfaces: Array?
    ) {
        super.visit(version, access, name, signature, superName, interfaces)
        mClassName = name
    }

    override fun visitMethod(
        access: Int,
        name: String?,
        desc: String?,
        signature: String?,
        exceptions: Array?
    ): MethodVisitor {
        //println("RoomOpenHelperClassVisitor mClassName $mClassName className $className name $name")
        val methodVisitor = super.visitMethod(access, name, desc, signature, exceptions)
        if (className == mClassName && METHOD_ON_DOWNGRADE == name) {
            //返回自定义MethodVisitor对象
            return RoomOpenHelperMethodVisitor(methodVisitor, access, name, desc)
        }
        return methodVisitor
    }


}

/**
 * 访问method
 */
class RoomOpenHelperMethodVisitor constructor(
    methodVisitor: MethodVisitor,
    access: Int,
    name: String,
    descriptor: String?
) : AdviceAdapter(Opcodes.ASM6, methodVisitor, access, name, descriptor) {

    override fun visitMethodInsn(
        opcode: Int,
        owner: String?,
        name: String?,
        desc: String?,
        itf: Boolean
    ) {
        //移除METHOD_ON_UPGRADE方法调用
        if (opcode == INVOKEVIRTUAL && METHOD_ON_UPGRADE == name) {
            println("rm onUpgrade(db, oldVersion, newVersion) and add updateIdentity(db)")
            mv.visitVarInsn(ALOAD, 0)
            mv.visitVarInsn(ALOAD, 1)
            mv.visitMethodInsn(
                INVOKEVIRTUAL,
                "androidx/room/RoomOpenHelper",
                "updateIdentity",
                "(Landroidx/sqlite/db/SupportSQLiteDatabase;)V",
                false
            )
            return
        }
        super.visitMethodInsn(opcode, owner, name, desc, itf)

    }
}


/**
 *Transform扫描class
 */
class RoomOpenHelperTransform internal constructor(private val project: Project) : Transform() {
    override fun getName(): String {
        return "RoomOpenHelperTransform"
    }

    override fun getInputTypes(): Set {
        return TransformManager.CONTENT_CLASS
    }

    override fun getScopes(): MutableSet? {
        return TransformManager.SCOPE_FULL_PROJECT
    }

    override fun isIncremental(): Boolean {
        return false
    }

    @Throws(TransformException::class, InterruptedException::class, IOException::class)
    override fun transform(transformInvocation: TransformInvocation) {
        super.transform(transformInvocation)
        println("\nRoomOpenHelperTransform start to transform-------------->>>>>>>")
        val outputProvider = transformInvocation.outputProvider
        val isIncremental = transformInvocation.isIncremental
        println("RoomOpenHelperTransform isIncremental is $isIncremental-------------->>>>>>>")
        //如果非增量,则清空旧的输出内容
        if (!isIncremental) {
            println("RoomOpenHelperTransform outputProvider delete all-------------->>>>>>>")
            outputProvider.deleteAll()
        }
        val inputs = transformInvocation.inputs
        for (transformInput in inputs) {
            //遍历所有的class文件目录
            val directoryInputs = transformInput.directoryInputs
            for (directoryInput in directoryInputs) {
                //必须这样获取输出路径的目录名称
                val destFile = transformInvocation.outputProvider.getContentLocation(
                    directoryInput.name,
                    directoryInput.contentTypes,
                    directoryInput.scopes,
                    Format.DIRECTORY
                )
                FileUtils.copyDirectory(directoryInput.file, destFile)
            }
            val jarInputs = transformInput.jarInputs
            for (jarInput in jarInputs) {
                //获取输出路径下的jar包名称,必须这样获取,得到的输出路径名不能重复,否则会被覆盖
                val destFile = transformInvocation.outputProvider.getContentLocation(
                    jarInput.name,
                    jarInput.contentTypes,
                    jarInput.scopes,
                    Format.JAR
                )
                if (jarInput.file.absolutePath.endsWith(".jar")) {
                    //println("RoomOpenHelperTransform: jarInput.file.absolutePath = " + jarInput.file.absolutePath);
                    val jarFile = jarInput.file
                    //只处理有我们业务逻辑的jar包
                    if (shouldProcessPreDexJar(jarFile.absolutePath)) {
                        handleJar(jarFile, destFile)
                        continue
                    }
                }
                //将输入文件拷贝到输出目录下
                FileUtils.copyFile(jarInput.file, destFile)
            }
        }
    }

    companion object {

        private fun shouldProcessPreDexJar(path: String): Boolean {
            return path.contains("room-runtime")
        }

        private fun handleJar(jarFile: File, destFile: File) {
            println("RoomOpenHelperTransform: handleJar ${jarFile.absolutePath}")
            val zipFile = ZipFile(jarFile)
            val zipOutputStream = ZipOutputStream(FileOutputStream(destFile))
            zipOutputStream.use {
                zipFile.use {
                    val enumeration = zipFile.entries()
                    while (enumeration.hasMoreElements()) {
                        val zipEntry = enumeration.nextElement()
                        val zipEntryName = zipEntry.name
                        //println("RoomOpenHelperTransform: handleJar zipEntryName $zipEntryName")
                        if (CLASS_ROOM_OPEN_HELPER == zipEntryName) {
                            val inputStream = zipFile.getInputStream(zipEntry)
                            val classReader = ClassReader(inputStream)
                            println("RoomOpenHelperTransform: handleJar classReader.className ${classReader.className}")
                            val classWriter = ClassWriter(classReader, ClassWriter.COMPUTE_MAXS)
                            val classVisitor: ClassVisitor =
                                RoomOpenHelperClassVisitor(classWriter, classReader.className)
                            classReader.accept(classVisitor, ClassReader.EXPAND_FRAMES)

                            val data = classWriter.toByteArray()
                            val byteArrayInputStream: InputStream = ByteArrayInputStream(data)

                            val newZipEntry = ZipEntry(zipEntryName)
                            FileUtil.addZipEntry(zipOutputStream, newZipEntry, byteArrayInputStream)

                        } else {
                            val newZipEntry = ZipEntry(zipEntryName)
                            FileUtil.addZipEntry(
                                zipOutputStream,
                                newZipEntry,
                                zipFile.getInputStream(zipEntry)
                            )
                        }

                    }
                }
            }
        }

        private fun eachFileRecurse(file: File) {
            if (file.exists()) {
                val files = file.listFiles()
                if (null != files) {
                    for (tempFile in files) {
                        if (tempFile.isDirectory) {
                            eachFileRecurse(tempFile)
                        } else {
                            if (tempFile.name == "module-info.class") {
                                println("RoomOpenHelperTransform: class file name = module-info.class , delete ")
                            }
                        }
                    }
                }
            }
        }

        private fun handleSources(directoryInput: DirectoryInput) {
            directoryInput.file.walkTopDown().filter { it.isFile }.forEach {
                if (CLASS_ROOM_OPEN_HELPER == it.name) {
                    it.inputStream().use { inputStream ->
                        val classReader = ClassReader(inputStream)
                        println("handleSources->${it.absolutePath}, name->${classReader.className}")
                        val classWriter = ClassWriter(
                            classReader,
                            ClassWriter.COMPUTE_FRAMES or ClassWriter.COMPUTE_MAXS
                        )
                        val classVisitor: ClassVisitor =
                            RoomOpenHelperClassVisitor(classWriter, classReader.className)
                        classReader.accept(classVisitor, ClassReader.EXPAND_FRAMES)
                        it.outputStream().use { outputStream ->
                            outputStream.write(classWriter.toByteArray())
                        }
                    }
                }
            }
        }

    }
}

这里用Kotlin重写了一份,就不赘述ASM的相关怎么用了。主要是RoomOpenHelperTransform这个类,为什么插件可以在编译时获取class文件并且修改字节码,就是靠这个Transform。

Transform的知识点参考:Gradle Transform API 的基本使用

配置插件使用Transform

写好的Transform怎么用?在插件实现类的apply方法中,使用AppExtension的registerTransform注册即可,代码如下:

package com.nxg.plugins;

import com.android.build.gradle.AppExtension;

import org.gradle.api.Plugin;
import org.gradle.api.Project;
import org.jetbrains.annotations.NotNull;

public class ASMGradlePlugin implements Plugin {

    @Override
    public void apply(@NotNull Project project) {
        System.out.println("ASMGradlePlugin: apply------------------>");
        AppExtension appExtension = project.getExtensions().getByType(AppExtension.class);
        appExtension.registerTransform(new RoomOpenHelperTransform(project));
        project.task("hello").doLast(task -> System.out.println("Hello from the com.nxg.plugins.JavaGreetingPlugin(buildSrc)"));
    }
}

到此,插件就写完了,是时候运行看看效果了。

发布Gradle插件

由于上面的插件写法属于独立的插件项目,因此要想使用插件,必须先把插件发布出去,然后其它项目才能使用这个插件。插件的发布也很简单,通过maven-publish插件提供的gradle publish task即可,maven-publish插件使用和配置的关键代码如下:

plugins {
    id 'maven-publish'
    ...
}

...


//定义Maven仓库信息
def MAVEN_GROUP_ID = "com.nxg.plugins"
def MAVEN_ARTIFACT_ID = "asm-gradle-plugin"
def MAVEN_VERSION = "1.0.0"

publishing {
    publications {
        java(MavenPublication) {
            from components.java
            groupId = MAVEN_GROUP_ID
            artifactId = MAVEN_ARTIFACT_ID
            version = MAVEN_VERSION
        }
    }

    repositories {
        mavenLocal()
        maven {
            // change to point to your repo, e.g. http://my.org/repo
            url = layout.buildDirectory.dir('repo')
        }
    }
}

配置好后,点击Sync Projects With Gradle File,即可在右上角的gradl task list中看到对应module的publish tasks。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第6张图片

双击publisToMavenLocal即可发布插件到maven的本地目录,一般是/.m2/repository目录。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第7张图片

当然,你可以发布到指定的目录中:

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第8张图片

使用Gradle插件

配置插件的仓库和classpath

有了插件就可以用了。在工程项目的根目录的build.gradle中添加插件的classpath:

    classpath 'com.nxg.plugins:asm-gradle-plugin:1.0.0'

由于插件是发布到mavenLocal的,所以仓库也要加配置mavenLocal:

  repositories {
        mavenLocal()
        google()
        mavenCentral()
    }

完整的build.gradle:

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        mavenLocal()
        google()
        mavenCentral()
    }
    dependencies {
        classpath "com.android.tools.build:gradle:7.0.1"
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.30"
        classpath 'com.nxg.plugins:asm-gradle-plugin:1.0.0'
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

在app中apply插件

在app的build.gradle中apply插件即可正常使用插件功能。

apply plugin: 'asm-gradle-plugin'

完整的build.gradle:

plugins {
    id 'com.android.application'
    id 'kotlin-android'
    id 'greeting'
}

apply plugin: 'asm-gradle-plugin'

android {
    compileSdk 31

    defaultConfig {
        applicationId "com.nxg.asm"
        minSdk 21
        targetSdk 31
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
}


dependencies {
    implementation 'androidx.core:core-ktx:1.6.0'
    implementation 'androidx.appcompat:appcompat:1.3.1'
    implementation 'com.google.android.material:material:1.3.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'

    implementation("androidx.room:room-runtime:2.3.0")
    annotationProcessor("androidx.room:room-compiler:2.3.0")

    implementation group: 'org.ow2.asm', name: 'asm', version: '7.2'
    implementation group: 'org.ow2.asm', name: 'asm-commons', version: '7.2'
}

注意,由于我们的自定义插件是发布到mavenLocal的,如果mavenLocal的目录中没有这个插件,直接apply插件就会报错,因此建议先注释apply plugin: 'asm-gradle-plugin',待插件发布后再取消注释。

验证

编译项目生成apk

直接build即可,build的过程可以看到相关日志打印。

apktool反编译apk

使用apktool反编译apk得到解压的文件夹,按照图中的包名找到对应的RoomOpenHelper.smali复制待用。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第9张图片

使用smali.jar反编译smail文件

使用smali.jar反编译复制好的RoomOpenHelper.smali得到RoomOpenHelper.dex文件。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第10张图片

使用d2j-dex2jar.sh 反编译dex文件

使用d2j-dex2jar.sh反编译RoomOpenHelper.dex得到RoomOpenHelper-dex2jar.jar文件。

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第11张图片

使用jd-gui 查看jar文件

使用jd-gui查看RoomOpenHelper-dex2jar.jar源码:

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第12张图片

顺便看下RoomOpenHelper.smali:

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第13张图片

没毛病,是想要的效果了。

相关反编译脚本

#!/bin/bash
rm RoomOpenHelper.smali
rm RoomOpenHelper.dex
rm RoomOpenHelper-dex2jar.jar
java -jar apktool.jar d -f app-debug.apk
cp /work/decompile/app-debug/smali/androidx/room/RoomOpenHelper.smali /work/decompile
java -jar smali-2.5.2.jar a RoomOpenHelper.smali -o RoomOpenHelper.dex
sh ./dex-tools-2.1/d2j-dex2jar.sh  --force RoomOpenHelper.dex 

还没结束

到这里,对于通过ASM修改Android SDK源码的方式来解决问题的思路已经全部讲解完了。

identityHash引发的新问题

但是对于Jetpack Room数据库降级引发的崩溃这个问题来说,还是没有彻底解决的,新的异常如下:

Room cannot verify the data integrity. Looks like"
                    + " you've changed schema but forgot to update the version number. You can"
                    + " simply fix this by increasing the version number.

查看源码发现时是checkIdentity方法导致的,为什么会这样?

 private void checkIdentity(SupportSQLiteDatabase db) {
        String identityHash = null;
        if (hasRoomMasterTable(db)) {
            Cursor cursor = db.query(new SimpleSQLiteQuery(RoomMasterTable.READ_QUERY));
            //noinspection TryFinallyCanBeTryWithResources
            try {
                if (cursor.moveToFirst()) {
                    identityHash = cursor.getString(0);
                }
            } finally {
                cursor.close();
            }
        }
        if (!mIdentityHash.equals(identityHash) && !mLegacyHash.equals(identityHash)) {
            throw new IllegalStateException("Room cannot verify the data integrity. Looks like"
                    + " you've changed schema but forgot to update the version number. You can"
                    + " simply fix this by increasing the version number.");
        }
    }

 private static boolean hasRoomMasterTable(SupportSQLiteDatabase db) {
        Cursor cursor = db.query("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name='"
                + RoomMasterTable.TABLE_NAME + "'");
        //noinspection TryFinallyCanBeTryWithResources
        try {
            return cursor.moveToFirst() && cursor.getInt(0) != 0;
        } finally {
            cursor.close();
        }
    }

@SuppressWarnings("WeakerAccess")
@RestrictTo(RestrictTo.Scope.LIBRARY_GROUP)
public class RoomMasterTable {
    /**
     * The master table where room keeps its metadata information.
     */
    public static final String TABLE_NAME = "room_master_table";
    // must match the runtime property Room#MASTER_TABLE_NAME
    public static final String NAME = "room_master_table";
    private static final String COLUMN_ID = "id";
    private static final String COLUMN_IDENTITY_HASH = "identity_hash";
    public static final String DEFAULT_ID = "42";

    public static final String CREATE_QUERY = "CREATE TABLE IF NOT EXISTS " + TABLE_NAME + " ("
            + COLUMN_ID + " INTEGER PRIMARY KEY,"
            + COLUMN_IDENTITY_HASH + " TEXT)";

    public static final String READ_QUERY = "SELECT " + COLUMN_IDENTITY_HASH
            + " FROM " + TABLE_NAME + " WHERE "
            + COLUMN_ID + " = " + DEFAULT_ID + " LIMIT 1";

    /**
     * We don't escape here since we know what we are passing.
     */
    public static String createInsertQuery(String hash) {
        return "INSERT OR REPLACE INTO " + TABLE_NAME + " ("
                + COLUMN_ID + "," + COLUMN_IDENTITY_HASH + ")"
                + " VALUES(" + DEFAULT_ID + ", \"" + hash + "\")";
    }

    private RoomMasterTable() {
    }
}

寻着蛛丝马迹,我们发现Room每次编译的时候会在对应的XXXDataBase_Impl实现类中,生成两个Hash值:identityHash和legacyHash。根据字面翻译结合checkIdentity源码可知,这两个hash值是用来检查数据库发生变化后数据库版本号是否跟着update,如果不是则抛出异常。

其中,identityHash不仅存在XXXDataBase_Impl类和RoomOpenHelper类的identityHash字段中,还保存在sqlite_master表中,代表当前数据库结构的“身份证”;而legacyHash只存在XXXDataBase_Impl类和RoomOpenHelper类的mLegacyHash字段,不能在sqlite_master表中。

为什么要两个hash值?一个不行吗?看下面注释就明白了。原来Room V1版本中存在一个bug,如果表结构的字段排序不一样会导致生成的hash值(mLegacyHash)不一样,这样导致的结果是,如果你只是修改了代码中字段的排序(没有对字段名称和数量都没变),就会导致抛出异常提示你升级版本号,显然这样是没有必要的。之后Android官方修复了这个bug,用新的identityHash来处理,同时保留老的mLegacyHash,并且判断条件是必须identityHash和legacyHash都不一致才会抛出异常,这样就能避免上述提到的bug了。

    @NonNull
    private final String mIdentityHash;
    /**
     * Room v1 had a bug where the hash was not consistent if fields are reordered.
     * The new has fixes it but we still need to accept the legacy hash.
     */
    @NonNull // b/64290754
    private final String mLegacyHash;

回到正题,为什么onDowngrade按下面方式修改了还不行?

//修改前   
 @Override
    public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) {
        onUpgrade(db, oldVersion, newVersion);
    }

//修改后
   @Override
    public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) {
        
    }

原因就是RoomOpenHelper的onOpen方法会在打开数据库时去检查identityHash和legacyHash,由于onDowngrade空实现并没有更新sqlite_master表中的identity_hash字段,这样一来必然会导致checkIdentity判断不通过,因为sqlite_master表中的identity_hash字段是新数据库的,软件降级之后,老版本代码的identityHash和legacyHash是老的数据库的Hash值。

 @Override
    public void onOpen(SupportSQLiteDatabase db) {
        super.onOpen(db);
        checkIdentity(db);
        mDelegate.onOpen(db);
        // there might be too many configurations etc, just clear it.
        mConfiguration = null;
    }

更新identityHash

那怎么解决?很简单,要么一不做二不休把checkIdentity也干掉,要么老老实实更新sqlite_master表中的identity_hash字段。显然干掉checkIdentity是不合适的,这样搞把Room的检测机制都搞没了,风险太大,后果未知。

那如何更新identity_hash字段?调用RoomOpenHelper的updateIdentity就行了,上帝给你关上一扇门又会给你打开一扇窗。

    private void updateIdentity(SupportSQLiteDatabase db) {
        createMasterTableIfNotExists(db);
        db.execSQL(RoomMasterTable.createInsertQuery(mIdentityHash));
    }

关键代码如下:

 override fun visitMethodInsn(
        opcode: Int,
        owner: String?,
        name: String?,
        desc: String?,
        itf: Boolean
    ) {
        //移除METHOD_ON_UPGRADE方法调用
        if (opcode == INVOKEVIRTUAL && METHOD_ON_UPGRADE == name) {
            println("rm onUpgrade(db, oldVersion, newVersion) and add updateIdentity(db)")
            //新增updateIdentity方法调用
            mv.visitVarInsn(ALOAD, 0)
            mv.visitVarInsn(ALOAD, 1)
            mv.visitMethodInsn(
                INVOKEVIRTUAL,
                "androidx/room/RoomOpenHelper",
                "updateIdentity",
                "(Landroidx/sqlite/db/SupportSQLiteDatabase;)V",
                false
            )
            return
        }
        super.visitMethodInsn(opcode, owner, name, desc, itf)

    }

最后再看下反编译的后的RoomOpenHelper:

Android ASM实践系列-01-解决Jetpack Room数据库降级引发的崩溃_第14张图片

最终多次测试验证,问题解决,Bingo!

小结

针对本文的这个问题本身,通过修改SDK源码的方式来解决,看起来有点小题大做了,但是这个思路是完全可以应用到其他场景的。比如第三方的库有bug,项目着急上线,没时间找第三方修改,也没有源码,那能怎么办?绝望啊!好在你现在看到了本文,就有了新的解决方案了。有句话怎么说来着,剑可以不用,但是必须要有!对吧,技多不压身,想进步就得多学。

自定义插件,ASM字节码操作和分析,Transform,这些都不是什么新鲜玩意,只要会使用,重要的就是懂得如何利用工具去高效的解决问题,在这过程中读源码是不可避免的,只有知己知彼,才能百战不殆!

可能遇到的问题

lb@lbpc:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample$ sh gradlew clean build publish

Welcome to Gradle 7.0.2!

Here are the highlights of this release:
 - File system watching enabled by default
 - Support for running with and building Java 16 projects
 - Native support for Apple Silicon processors
 - Dependency catalog feature preview

For more details see https://docs.gradle.org/7.0.2/release-notes.html

Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details

FAILURE: Build failed with an exception.

* Where:
Build file '/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/build.gradle' line: 2

* What went wrong:
An exception occurred applying plugin request [id: 'com.android.application']
> Failed to apply plugin 'com.android.internal.application'.
   > Android Gradle plugin requires Java 11 to run. You are currently using Java 1.8.
     You can try some of the following options:
       - changing the IDE settings.
       - changing the JAVA_HOME environment variable.
       - changing `org.gradle.java.home` in `gradle.properties`.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/7.0.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 11s
lb@lbpc:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample$ 

解决办法是gradle.properties配置JDK的路径:

org.gradle.java.home=/work/android/android-studio-4.0/android-studio/jre

你可能感兴趣的:(Android开发实践,数据库,android,sqlite,jetpack)