某一天,测试部大哥从当前最新的软件版本降级到一个月前的版本,然后启动就出现崩溃了。
异常如下:
java.lang.IllegalStateException: A migration from 4 to 3 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods. at androidx.room.RoomOpenHelper.onUpgrade(RoomOpenHelper.java:117) at androidx.room.RoomOpenHelper.onDowngrade(RoomOpenHelper.java:129) at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.onDowngrade(FrameworkSQLiteOpenHelper.java:135) at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked(SQLiteOpenHelper.java:254) at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:163) at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.getWritableSupportDatabase(FrameworkSQLiteOpenHelper.java:92) at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper.getWritableDatabase(FrameworkSQLiteOpenHelper.java:53) at androidx.room.RoomDatabase.inTransaction(RoomDatabase.java:476) at androidx.room.RoomDatabase.assertNotSuspendingTransaction(RoomDatabase.java:281)
等等?你跟我说降级?现在有哪个App支持降级的吗?别着急,有句话说的好,只要思想不滑坡,方法总比困难多,降级的肯定是可以降级的,不过这不是重点,今天主要分享下数据库降级导致的问题如何使用ASM修改字节码的方式来解决。
乍一看,这不是很简单吗,添加对应的降级Migration或者配置fallbackToDestructiveMigration
方法进行破坏性重建数据库即可。
再看下RoomOpenHelper对应的源码:
@Override public void onUpgrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) { boolean migrated = false; if (mConfiguration != null) { Listmigrations = mConfiguration.migrationContainer.findMigrationPath( oldVersion, newVersion); if (migrations != null) { mDelegate.onPreMigrate(db); for (Migration migration : migrations) { migration.migrate(db); } ValidationResult result = mDelegate.onValidateSchema(db); if (!result.isValid) { throw new IllegalStateException("Migration didn't properly handle: " + result.expectedFoundMsg); } mDelegate.onPostMigrate(db); updateIdentity(db); migrated = true; } } if (!migrated) { if (mConfiguration != null && !mConfiguration.isMigrationRequired(oldVersion, newVersion)) { mDelegate.dropAllTables(db); mDelegate.createAllTables(db); } else { throw new IllegalStateException("A migration from " + oldVersion + " to " + newVersion + " was required but not found. Please provide the " + "necessary Migration path via " + "RoomDatabase.Builder.addMigration(Migration ...) or allow for " + "destructive migrations via one of the " + "RoomDatabase.Builder.fallbackToDestructiveMigration* methods."); } } }
没错了,既然是没添加对应Migration,那加上不就完了吗?说的好的,确实如此,不过要是这样就结束了,还怎么秀操作?而且为啥不用fallbackToDestructiveMigration
?不能用的原因很简单,领导说了,不管是数据库降级还是还是升级,数据都必须要保留。fallbackToDestructiveMigration
的操作就是删表重建,这样是不满足要求的。
开干,一顿操作猛如虎,三下五除二就把降级Migration的加上了。
private fun buildDatabase(context: Context): TestDataBase { return Room.databaseBuilder(context, TestDataBase::class.java, "test.db") .addMigrations(MIGRATION_1_2 .addMigrations(MIGRATION_2_1) .addCallback(object : RoomDatabase.Callback() { override fun onCreate(db: SupportSQLiteDatabase) { super.onCreate(db) Log.i(TAG, "onCreate") } override fun onOpen(db: SupportSQLiteDatabase) { super.onOpen(db) Log.i(TAG, "onOpen") } }) .build() } /** * 版本升级1->2 */ private val MIGRATION_1_2 = object : Migration(1, 2) { override fun migrate(database: SupportSQLiteDatabase) { database.execSQL("ALTER TABLE Test ADD COLUMN updatetime TEXT") } } /** * 版本升级2->1空实现,防止降级崩溃 */ private val MIGRATION_2_1 = object : Migration(2, 1) { override fun migrate(database: SupportSQLiteDatabase) { Log.i(TAG, "migrate: MIGRATION_2_1 do nothing") } }
重新打包给到测试,嘿,没问题,禅道马上关闭bug。心里美滋滋,把这个问题反馈给领导,让其它项目都修改下这个问题。但是项目这么多,涉及到Room数据库的都要手动改一遍,岂不是很麻烦?有没有更优雅的方式,最好是其它项目都不用动,躺着就把问题解决了。
好家伙,提高难度了,看来要秀出看家本领才行。
再回头看下RoomOpenHelper的源码:
@Override public void onUpgrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) { boolean migrated = false; if (mConfiguration != null) { Listmigrations = mConfiguration.migrationContainer.findMigrationPath( oldVersion, newVersion); if (migrations != null) { mDelegate.onPreMigrate(db); for (Migration migration : migrations) { migration.migrate(db); } ValidationResult result = mDelegate.onValidateSchema(db); if (!result.isValid) { throw new IllegalStateException("Migration didn't properly handle: " + result.expectedFoundMsg); } mDelegate.onPostMigrate(db); updateIdentity(db); migrated = true; } } if (!migrated) { if (mConfiguration != null && !mConfiguration.isMigrationRequired(oldVersion, newVersion)) { mDelegate.dropAllTables(db); mDelegate.createAllTables(db); } else { throw new IllegalStateException("A migration from " + oldVersion + " to " + newVersion + " was required but not found. Please provide the " + "necessary Migration path via " + "RoomDatabase.Builder.addMigration(Migration ...) or allow for " + "destructive migrations via one of the " + "RoomDatabase.Builder.fallbackToDestructiveMigration* methods."); } } } @Override public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) { onUpgrade(db, oldVersion, newVersion); }
不看不知道,官方这骚操作,onDowngrade
竟然是调用的onUpgrade
,我直呼内行。那对应的解决办法也很简单,重写下onDowngrade
方法不要调用onUpgrade
那不就可以了?先试试?试试就试试。
按照Greendao或者SupportSQLiteOpenHelper的经验,onDowngrade
是可以直接重写,但是到了Room这里,看起来就没办法了,RoomOpenHelper的实例是在对应的XXXDataBaseImpl创建的,根本不给机会啊,而且XXXDataBaseImpl实现类是编译时Room的注解处理器生成的。
/** * An open helper that holds a reference to the configuration until the database is opened. * * @hide */ @SuppressWarnings("unused") @RestrictTo(RestrictTo.Scope.LIBRARY_GROUP_PREFIX) public class RoomOpenHelper extends SupportSQLiteOpenHelper.Callback { ... } @Override protected SupportSQLiteOpenHelper createOpenHelper(DatabaseConfiguration configuration) { final SupportSQLiteOpenHelper.Callback _openCallback = new RoomOpenHelper(configuration, new RoomOpenHelper.Delegate(5) { } }
怎么办?ASM就派上用场了。
注意:本文不赘述ASM的详细用法,只讲解针对以上问题,如何使用ASM解决,不熟悉AMS或者对ASM有兴趣的朋友可以参考lsieun 大佬的《Java ASM系列》教程
ASM的官网地址:ASM
既然是使用ASM,那我们肯定是希望在项目编译时去动态修改RoomOpenHelper的class文件,达到重写onDowngrade
的目的。
开始前需要具备自定义Gradle Plugins的知识,具体可参考:
Android Gradle Plugins系列-01-自定义Gradle插件入门指南
当然,如果只是单纯看看ASM是如何解决问题的也不强求必须懂自定义Gradle Plugins,只查看ASM相关的内容即可。
废话不多说,新建一个工程,添加ASM依赖如下:
dependencies { implementation 'androidx.appcompat:appcompat:1.3.1' implementation 'com.google.android.material:material:1.3.0' implementation 'androidx.constraintlayout:constraintlayout:2.0.4' testImplementation 'junit:junit:4.13.2' androidTestImplementation 'androidx.test.ext:junit:1.1.3' androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0' implementation("androidx.room:room-runtime:2.3.0") annotationProcessor("androidx.room:room-compiler:2.3.0") implementation group: 'org.ow2.asm', name: 'asm', version: '7.2' implementation group: 'org.ow2.asm', name: 'asm-commons', version: '7.2' }
为了方便修改字节码,我们需要借助第三方插件,如果使用最新的AndroidStudio版本或者使用了Kotlin建议安装ASM Bytecode Viewer Support Kotlin这个插件。如果安装了另外两个插件,可能会启动报错或者无法使用,解决办法是卸载不能用的插件,重新安装ASM Bytecode Viewer Support Kotlin插件。关于如何卸载插件,某度搜索下就有相关教程了。
为方便了调试,我们先使用单元测试来验证ASM修改字节码是够能达到期望的效果。在单元测试的包中新增两个类:ASMTest和RoomOpenHelper。
RoomOpenHelper代码如下:
/** * 模拟Room的RoomOpenHelper类 */ public class RoomOpenHelper { public void onUpgrade(SupportSQLiteDatabase db) { } public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) { onUpgrade(db); } public void updateIdentity(SupportSQLiteDatabase db) { } public static class SupportSQLiteDatabase { } }
ASMTest代码如下:
package com.nxg.asm; import org.junit.Test; import org.objectweb.asm.ClassReader; import org.objectweb.asm.ClassVisitor; import org.objectweb.asm.ClassWriter; import org.objectweb.asm.MethodVisitor; import org.objectweb.asm.Opcodes; import org.objectweb.asm.commons.AdviceAdapter; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; public class ASMTest { @Test public void test() { //1、准备待分析的class FileInputStream fis; try { fis = new FileInputStream("/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm/RoomOpenHelper.class"); //2、执行分析与插桩 //class字节码的读取与分析引擎 ClassReader cr = new ClassReader(fis); // 写出器 COMPUTE_FRAMES 自动计算所有的内容,后续操作更简单 ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES); //分析,处理结果写入cw EXPAND_FRAMES:栈图以扩展格式进行访问 cr.accept(new ClassAdapterVisitor(cw), ClassReader.EXPAND_FRAMES); //3、获得结果并输出 byte[] newClassBytes = cw.toByteArray(); FileOutputStream fos = new FileOutputStream ("/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm/DstRoomOpenHelper.class"); fos.write(newClassBytes); fos.close(); } catch (IOException e) { e.printStackTrace(); } } public static class ClassAdapterVisitor extends ClassVisitor { public ClassAdapterVisitor(ClassVisitor cv) { super(Opcodes.ASM7, cv); } @Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { System.out.println("方法:" + name + " 签名:" + desc); //遇到onDowngrade方法,就是返回对应的MethodAdapterVisitor if ("onDowngrade".equals(name)) { MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions); return new MethodAdapterVisitor(api, mv, access, name, desc); } return super.visitMethod(access, name, desc, signature, exceptions); } } public static class MethodAdapterVisitor extends AdviceAdapter { protected MethodAdapterVisitor(int api, MethodVisitor methodVisitor, int access, String name, String descriptor) { super(api, methodVisitor, access, name, descriptor); } /** * 访问方法指令,方法指令是调用方法的指令。 * 参数: * opcode – 要访问的类型指令的操作码。 此操作码是 INVOKEVIRTUAL、INVOKESPECIAL、INVOKESTATIC 或 INVOKEINTERFACE。 * owner – 方法所有者类的内部名称(请参阅 Type.getInternalName())。 * name - 方法的名称。 * 描述符 – 方法的描述符(请参阅类型)。 * isInterface – 如果方法的所有者类是接口。 */ @Override public void visitMethodInsn(int opcodeAndSource, String owner, String name, String descriptor, boolean isInterface) { System.out.println("opcodeAndSource:" + opcodeAndSource + ", owner:" + owner + ", name:" + name + ", descriptor:" + descriptor); //移除onUpgrade方法调用 if (opcodeAndSource == INVOKEVIRTUAL && "onUpgrade".equals(name)) { return; } super.visitMethodInsn(opcodeAndSource, owner, name, descriptor, isInterface); } @Override protected void onMethodEnter() { super.onMethodEnter(); } @Override protected void onMethodExit(int opcode) { super.onMethodExit(opcode); } } }
由于ASM操作的是class文件,所以需要先编译生成对应的class文件。进入RoomOpenHelper所在目录,在终端执行javac RoomOpenHelper.java
root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ ll total 20 drwxrwxr-x 2 lb lb 4096 11月 5 16:43 ./ drwxrwxr-x 3 lb lb 4096 11月 5 14:31 ../ -rw-rw-r-- 1 lb lb 3578 11月 5 16:15 ASMTest.java -rw-rw-r-- 1 lb lb 372 11月 5 14:31 ExampleUnitTest.java -rw-rw-r-- 1 lb lb 400 11月 5 16:22 RoomOpenHelper.java root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ javac RoomOpenHelper.java root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$ ll total 28 drwxrwxr-x 2 lb lb 4096 11月 5 16:43 ./ drwxrwxr-x 3 lb lb 4096 11月 5 14:31 ../ -rw-rw-r-- 1 lb lb 3578 11月 5 16:15 ASMTest.java -rw-rw-r-- 1 lb lb 372 11月 5 14:31 ExampleUnitTest.java -rw-rw-r-- 1 lb lb 616 11月 5 16:43 RoomOpenHelper.class -rw-rw-r-- 1 lb lb 400 11月 5 16:22 RoomOpenHelper.java -rw-rw-r-- 1 lb lb 323 11月 5 16:43 'RoomOpenHelper$SupportSQLiteDatabase.clas' root:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/src/test/java/com/nxg/asm$
可以看到,生成了RoomOpenHelper.class文件如下:
// // Source code recreated from a .class file by IntelliJ IDEA // (powered by FernFlower decompiler) // package com.nxg.asm; public class RoomOpenHelper { public RoomOpenHelper() { } public void onUpgrade(RoomOpenHelper.SupportSQLiteDatabase var1) { } public void onDowngrade(RoomOpenHelper.SupportSQLiteDatabase var1, int var2, int var3) { this.onUpgrade(var1); } public void updateIdentity(RoomOpenHelper.SupportSQLiteDatabase var1) { } public static class SupportSQLiteDatabase { public SupportSQLiteDatabase() { } } }
此类模拟了Room库的RoomOpenHelper类,可以看到onDowngrade
是调用了onUpgrade
的,我们的目标是改成这样:
public void onDowngrade(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1, int var2, int var3) { }
现在来运行下ASMTest,点下图中绿色三角形即可:
运行完毕,包中多了一个DstRoomOpenHelper.class,打开看它的源码:
// // Source code recreated from a .class file by IntelliJ IDEA // (powered by FernFlower decompiler) // package com.nxg.asm; public class RoomOpenHelper { public RoomOpenHelper() { } public void onUpgrade(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1) { } public void onDowngrade(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1, int var2, int var3) { } public void updateIdentity(com.nxg.asm.RoomOpenHelper.SupportSQLiteDatabase var1) { } }
Nice,目标达成。我们看下ASMTest的关键代码部分:
public static class MethodAdapterVisitor extends AdviceAdapter { protected MethodAdapterVisitor(int api, MethodVisitor methodVisitor, int access, String name, String descriptor) { super(api, methodVisitor, access, name, descriptor); } /** * 访问方法指令。 方法指令是调用方法的指令。 * 参数: * opcode – 要访问的类型指令的操作码。 此操作码是 INVOKEVIRTUAL、INVOKESPECIAL、INVOKESTATIC 或 INVOKEINTERFACE。 * owner – 方法所有者类的内部名称(请参阅 Type.getInternalName())。 * name - 方法的名称。 * 描述符 – 方法的描述符(请参阅类型)。 * isInterface – 如果方法的所有者类是接口。 */ @Override public void visitMethodInsn(int opcodeAndSource, String owner, String name, String descriptor, boolean isInterface) { System.out.println("opcodeAndSource:" + opcodeAndSource + ", owner:" + owner + ", name:" + name + ", descriptor:" + descriptor); //移除代码 if (opcodeAndSource == INVOKEVIRTUAL && "onUpgrade".equals(name)) { return; } super.visitMethodInsn(opcodeAndSource, owner, name, descriptor, isInterface); } }
看起来很简单,由于我们修改的是类的方法,所以使用MethodAdapterVisitor。其中visitMethodInsn可以访问方法指令。
我们的目标是删除onDowngrade方法的onUpgrade调用,我们看下代码是怎么写的?
//移除代码 if (opcodeAndSource == INVOKEVIRTUAL && "onUpgrade".equals(name)) { return; }
就这么简单?就这么简单。一开始笔者也不知道如何删除某个方法内的某句方法调用,直到看到这篇文章深入探索编译插桩技术(四、ASM 探秘)底部的pengion网友的评论,经过一番测试,终于得以实现,感谢这些大佬的无私奉献。
文章评论如下:
pengion
android1年前
求解,怎么在事件模型下替换或删除方法内的某条语句呢?目前看来似乎只能增加代码
1年前
pengion
懂了,看了官方文档,原来是在继承MethodVisitor类的方法里遇到对应语句return就行了
笔者也去官网翻了翻文档,但是没有找到相关说明。
ASM的代码写好了,就可以拿过来用了,接下来就是编写自定义Gradle插件,在编译期对Room库的RoomOpenHelper类进行字节码修改达到我们的目标。
阅读到此,建议先储备自定义Gradle Plugins的知识,具体可参考:
Android Gradle Plugins系列-01-自定义Gradle插件入门指南
ModuleName和Package 根据需要修改,这里笔者选择Kotlin语言,Bytecode Level选择8。
按照以下路径,建立对应的目录和文件
module/src-main-resources/META-INF/gradle-plugins/asm-gradle-plugin.properties
文件内容如下:
implementation-class=com.nxg.plugins.ASMGradlePlugin
用于配置插件的实现类,这样Gradle才能实例化插件。
在 com.nxg.plugins包中新建ASMGradlePlugin类实现Plugin接口:
package com.nxg.plugins; import com.android.build.gradle.AppExtension; import org.gradle.api.Plugin; import org.gradle.api.Project; import org.jetbrains.annotations.NotNull; public class ASMGradlePlugin implements Plugin{ @Override public void apply(@NotNull Project project) { } }
一个简单的啥都不能干的插件就完成了。
plugins { id 'maven-publish' id 'java-library' id 'kotlin' id 'groovy' } java { sourceCompatibility = JavaVersion.VERSION_1_8 targetCompatibility = JavaVersion.VERSION_1_8 } dependencies { implementation fileTree(dir: 'libs', include: ['*.jar']) implementation "com.android.tools.build:gradle:3.5.0" implementation gradleApi() implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.5.30" implementation group: 'org.ow2.asm', name: 'asm', version: '7.2' implementation group: 'org.ow2.asm', name: 'asm-commons', version: '7.2' } //定义Maven仓库信息 def MAVEN_GROUP_ID = "com.nxg.plugins" def MAVEN_ARTIFACT_ID = "asm-gradle-plugin" def MAVEN_VERSION = "1.0.0" publishing { publications { java(MavenPublication) { from components.java groupId = MAVEN_GROUP_ID artifactId = MAVEN_ARTIFACT_ID version = MAVEN_VERSION } } repositories { mavenLocal() maven { // change to point to your repo, e.g. http://my.org/repo url = layout.buildDirectory.dir('repo') } } }
就不多说了,maven-publish和asm都是需要的。
package com.nxg.plugins import com.android.build.api.transform.* import com.android.build.gradle.internal.pipeline.TransformManager import com.android.utils.FileUtils import org.gradle.api.Project import org.objectweb.asm.* import org.objectweb.asm.commons.AdviceAdapter import java.io.* import java.util.zip.ZipEntry import java.util.zip.ZipFile import java.util.zip.ZipOutputStream const val METHOD_ON_DOWNGRADE = "onDowngrade" const val METHOD_ON_UPGRADE = "onUpgrade" const val CLASS_ROOM_OPEN_HELPER = "androidx/room/RoomOpenHelper.class" /** * 访问class */ class RoomOpenHelperClassVisitor constructor( classWriter: ClassWriter, private val className: String ) : ClassVisitor(Opcodes.ASM6, classWriter) { private var mClassName: String? = null override fun visit( version: Int, access: Int, name: String?, signature: String?, superName: String?, interfaces: Array? ) { super.visit(version, access, name, signature, superName, interfaces) mClassName = name } override fun visitMethod( access: Int, name: String?, desc: String?, signature: String?, exceptions: Array ? ): MethodVisitor { //println("RoomOpenHelperClassVisitor mClassName $mClassName className $className name $name") val methodVisitor = super.visitMethod(access, name, desc, signature, exceptions) if (className == mClassName && METHOD_ON_DOWNGRADE == name) { //返回自定义MethodVisitor对象 return RoomOpenHelperMethodVisitor(methodVisitor, access, name, desc) } return methodVisitor } } /** * 访问method */ class RoomOpenHelperMethodVisitor constructor( methodVisitor: MethodVisitor, access: Int, name: String, descriptor: String? ) : AdviceAdapter(Opcodes.ASM6, methodVisitor, access, name, descriptor) { override fun visitMethodInsn( opcode: Int, owner: String?, name: String?, desc: String?, itf: Boolean ) { //移除METHOD_ON_UPGRADE方法调用 if (opcode == INVOKEVIRTUAL && METHOD_ON_UPGRADE == name) { println("rm onUpgrade(db, oldVersion, newVersion) and add updateIdentity(db)") mv.visitVarInsn(ALOAD, 0) mv.visitVarInsn(ALOAD, 1) mv.visitMethodInsn( INVOKEVIRTUAL, "androidx/room/RoomOpenHelper", "updateIdentity", "(Landroidx/sqlite/db/SupportSQLiteDatabase;)V", false ) return } super.visitMethodInsn(opcode, owner, name, desc, itf) } } /** *Transform扫描class */ class RoomOpenHelperTransform internal constructor(private val project: Project) : Transform() { override fun getName(): String { return "RoomOpenHelperTransform" } override fun getInputTypes(): Set { return TransformManager.CONTENT_CLASS } override fun getScopes(): MutableSet ? { return TransformManager.SCOPE_FULL_PROJECT } override fun isIncremental(): Boolean { return false } @Throws(TransformException::class, InterruptedException::class, IOException::class) override fun transform(transformInvocation: TransformInvocation) { super.transform(transformInvocation) println("\nRoomOpenHelperTransform start to transform-------------->>>>>>>") val outputProvider = transformInvocation.outputProvider val isIncremental = transformInvocation.isIncremental println("RoomOpenHelperTransform isIncremental is $isIncremental-------------->>>>>>>") //如果非增量,则清空旧的输出内容 if (!isIncremental) { println("RoomOpenHelperTransform outputProvider delete all-------------->>>>>>>") outputProvider.deleteAll() } val inputs = transformInvocation.inputs for (transformInput in inputs) { //遍历所有的class文件目录 val directoryInputs = transformInput.directoryInputs for (directoryInput in directoryInputs) { //必须这样获取输出路径的目录名称 val destFile = transformInvocation.outputProvider.getContentLocation( directoryInput.name, directoryInput.contentTypes, directoryInput.scopes, Format.DIRECTORY ) FileUtils.copyDirectory(directoryInput.file, destFile) } val jarInputs = transformInput.jarInputs for (jarInput in jarInputs) { //获取输出路径下的jar包名称,必须这样获取,得到的输出路径名不能重复,否则会被覆盖 val destFile = transformInvocation.outputProvider.getContentLocation( jarInput.name, jarInput.contentTypes, jarInput.scopes, Format.JAR ) if (jarInput.file.absolutePath.endsWith(".jar")) { //println("RoomOpenHelperTransform: jarInput.file.absolutePath = " + jarInput.file.absolutePath); val jarFile = jarInput.file //只处理有我们业务逻辑的jar包 if (shouldProcessPreDexJar(jarFile.absolutePath)) { handleJar(jarFile, destFile) continue } } //将输入文件拷贝到输出目录下 FileUtils.copyFile(jarInput.file, destFile) } } } companion object { private fun shouldProcessPreDexJar(path: String): Boolean { return path.contains("room-runtime") } private fun handleJar(jarFile: File, destFile: File) { println("RoomOpenHelperTransform: handleJar ${jarFile.absolutePath}") val zipFile = ZipFile(jarFile) val zipOutputStream = ZipOutputStream(FileOutputStream(destFile)) zipOutputStream.use { zipFile.use { val enumeration = zipFile.entries() while (enumeration.hasMoreElements()) { val zipEntry = enumeration.nextElement() val zipEntryName = zipEntry.name //println("RoomOpenHelperTransform: handleJar zipEntryName $zipEntryName") if (CLASS_ROOM_OPEN_HELPER == zipEntryName) { val inputStream = zipFile.getInputStream(zipEntry) val classReader = ClassReader(inputStream) println("RoomOpenHelperTransform: handleJar classReader.className ${classReader.className}") val classWriter = ClassWriter(classReader, ClassWriter.COMPUTE_MAXS) val classVisitor: ClassVisitor = RoomOpenHelperClassVisitor(classWriter, classReader.className) classReader.accept(classVisitor, ClassReader.EXPAND_FRAMES) val data = classWriter.toByteArray() val byteArrayInputStream: InputStream = ByteArrayInputStream(data) val newZipEntry = ZipEntry(zipEntryName) FileUtil.addZipEntry(zipOutputStream, newZipEntry, byteArrayInputStream) } else { val newZipEntry = ZipEntry(zipEntryName) FileUtil.addZipEntry( zipOutputStream, newZipEntry, zipFile.getInputStream(zipEntry) ) } } } } } private fun eachFileRecurse(file: File) { if (file.exists()) { val files = file.listFiles() if (null != files) { for (tempFile in files) { if (tempFile.isDirectory) { eachFileRecurse(tempFile) } else { if (tempFile.name == "module-info.class") { println("RoomOpenHelperTransform: class file name = module-info.class , delete ") } } } } } } private fun handleSources(directoryInput: DirectoryInput) { directoryInput.file.walkTopDown().filter { it.isFile }.forEach { if (CLASS_ROOM_OPEN_HELPER == it.name) { it.inputStream().use { inputStream -> val classReader = ClassReader(inputStream) println("handleSources->${it.absolutePath}, name->${classReader.className}") val classWriter = ClassWriter( classReader, ClassWriter.COMPUTE_FRAMES or ClassWriter.COMPUTE_MAXS ) val classVisitor: ClassVisitor = RoomOpenHelperClassVisitor(classWriter, classReader.className) classReader.accept(classVisitor, ClassReader.EXPAND_FRAMES) it.outputStream().use { outputStream -> outputStream.write(classWriter.toByteArray()) } } } } } } }
这里用Kotlin重写了一份,就不赘述ASM的相关怎么用了。主要是RoomOpenHelperTransform这个类,为什么插件可以在编译时获取class文件并且修改字节码,就是靠这个Transform。
Transform的知识点参考:Gradle Transform API 的基本使用
写好的Transform怎么用?在插件实现类的apply方法中,使用AppExtension的registerTransform注册即可,代码如下:
package com.nxg.plugins; import com.android.build.gradle.AppExtension; import org.gradle.api.Plugin; import org.gradle.api.Project; import org.jetbrains.annotations.NotNull; public class ASMGradlePlugin implements Plugin{ @Override public void apply(@NotNull Project project) { System.out.println("ASMGradlePlugin: apply------------------>"); AppExtension appExtension = project.getExtensions().getByType(AppExtension.class); appExtension.registerTransform(new RoomOpenHelperTransform(project)); project.task("hello").doLast(task -> System.out.println("Hello from the com.nxg.plugins.JavaGreetingPlugin(buildSrc)")); } }
到此,插件就写完了,是时候运行看看效果了。
由于上面的插件写法属于独立的插件项目,因此要想使用插件,必须先把插件发布出去,然后其它项目才能使用这个插件。插件的发布也很简单,通过maven-publish插件提供的gradle publish task即可,maven-publish插件使用和配置的关键代码如下:
plugins { id 'maven-publish' ... } ... //定义Maven仓库信息 def MAVEN_GROUP_ID = "com.nxg.plugins" def MAVEN_ARTIFACT_ID = "asm-gradle-plugin" def MAVEN_VERSION = "1.0.0" publishing { publications { java(MavenPublication) { from components.java groupId = MAVEN_GROUP_ID artifactId = MAVEN_ARTIFACT_ID version = MAVEN_VERSION } } repositories { mavenLocal() maven { // change to point to your repo, e.g. http://my.org/repo url = layout.buildDirectory.dir('repo') } } }
配置好后,点击Sync Projects With Gradle File,即可在右上角的gradl task list中看到对应module的publish tasks。
双击publisToMavenLocal即可发布插件到maven的本地目录,一般是/.m2/repository目录。
当然,你可以发布到指定的目录中:
有了插件就可以用了。在工程项目的根目录的build.gradle中添加插件的classpath:
classpath 'com.nxg.plugins:asm-gradle-plugin:1.0.0'
由于插件是发布到mavenLocal的,所以仓库也要加配置mavenLocal:
repositories { mavenLocal() google() mavenCentral() }
完整的build.gradle:
// Top-level build file where you can add configuration options common to all sub-projects/modules. buildscript { repositories { mavenLocal() google() mavenCentral() } dependencies { classpath "com.android.tools.build:gradle:7.0.1" classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.30" classpath 'com.nxg.plugins:asm-gradle-plugin:1.0.0' // NOTE: Do not place your application dependencies here; they belong // in the individual module build.gradle files } } task clean(type: Delete) { delete rootProject.buildDir }
在app的build.gradle中apply插件即可正常使用插件功能。
apply plugin: 'asm-gradle-plugin'
完整的build.gradle:
plugins { id 'com.android.application' id 'kotlin-android' id 'greeting' } apply plugin: 'asm-gradle-plugin' android { compileSdk 31 defaultConfig { applicationId "com.nxg.asm" minSdk 21 targetSdk 31 versionCode 1 versionName "1.0" testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" } buildTypes { release { minifyEnabled false proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' } } compileOptions { sourceCompatibility JavaVersion.VERSION_1_8 targetCompatibility JavaVersion.VERSION_1_8 } kotlinOptions { jvmTarget = '1.8' } } dependencies { implementation 'androidx.core:core-ktx:1.6.0' implementation 'androidx.appcompat:appcompat:1.3.1' implementation 'com.google.android.material:material:1.3.0' implementation 'androidx.constraintlayout:constraintlayout:2.0.4' testImplementation 'junit:junit:4.13.2' androidTestImplementation 'androidx.test.ext:junit:1.1.3' androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0' implementation("androidx.room:room-runtime:2.3.0") annotationProcessor("androidx.room:room-compiler:2.3.0") implementation group: 'org.ow2.asm', name: 'asm', version: '7.2' implementation group: 'org.ow2.asm', name: 'asm-commons', version: '7.2' }
注意,由于我们的自定义插件是发布到mavenLocal的,如果mavenLocal的目录中没有这个插件,直接apply插件就会报错,因此建议先注释apply plugin: 'asm-gradle-plugin'
,待插件发布后再取消注释。
直接build即可,build的过程可以看到相关日志打印。
使用apktool反编译apk得到解压的文件夹,按照图中的包名找到对应的RoomOpenHelper.smali复制待用。
使用smali.jar反编译复制好的RoomOpenHelper.smali得到RoomOpenHelper.dex文件。
使用d2j-dex2jar.sh反编译RoomOpenHelper.dex得到RoomOpenHelper-dex2jar.jar文件。
使用jd-gui查看RoomOpenHelper-dex2jar.jar源码:
顺便看下RoomOpenHelper.smali:
没毛病,是想要的效果了。
#!/bin/bash rm RoomOpenHelper.smali rm RoomOpenHelper.dex rm RoomOpenHelper-dex2jar.jar java -jar apktool.jar d -f app-debug.apk cp /work/decompile/app-debug/smali/androidx/room/RoomOpenHelper.smali /work/decompile java -jar smali-2.5.2.jar a RoomOpenHelper.smali -o RoomOpenHelper.dex sh ./dex-tools-2.1/d2j-dex2jar.sh --force RoomOpenHelper.dex
到这里,对于通过ASM修改Android SDK源码的方式来解决问题的思路已经全部讲解完了。
但是对于Jetpack Room数据库降级引发的崩溃这个问题来说,还是没有彻底解决的,新的异常如下:
Room cannot verify the data integrity. Looks like" + " you've changed schema but forgot to update the version number. You can" + " simply fix this by increasing the version number.
查看源码发现时是checkIdentity方法导致的,为什么会这样?
private void checkIdentity(SupportSQLiteDatabase db) { String identityHash = null; if (hasRoomMasterTable(db)) { Cursor cursor = db.query(new SimpleSQLiteQuery(RoomMasterTable.READ_QUERY)); //noinspection TryFinallyCanBeTryWithResources try { if (cursor.moveToFirst()) { identityHash = cursor.getString(0); } } finally { cursor.close(); } } if (!mIdentityHash.equals(identityHash) && !mLegacyHash.equals(identityHash)) { throw new IllegalStateException("Room cannot verify the data integrity. Looks like" + " you've changed schema but forgot to update the version number. You can" + " simply fix this by increasing the version number."); } } private static boolean hasRoomMasterTable(SupportSQLiteDatabase db) { Cursor cursor = db.query("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name='" + RoomMasterTable.TABLE_NAME + "'"); //noinspection TryFinallyCanBeTryWithResources try { return cursor.moveToFirst() && cursor.getInt(0) != 0; } finally { cursor.close(); } } @SuppressWarnings("WeakerAccess") @RestrictTo(RestrictTo.Scope.LIBRARY_GROUP) public class RoomMasterTable { /** * The master table where room keeps its metadata information. */ public static final String TABLE_NAME = "room_master_table"; // must match the runtime property Room#MASTER_TABLE_NAME public static final String NAME = "room_master_table"; private static final String COLUMN_ID = "id"; private static final String COLUMN_IDENTITY_HASH = "identity_hash"; public static final String DEFAULT_ID = "42"; public static final String CREATE_QUERY = "CREATE TABLE IF NOT EXISTS " + TABLE_NAME + " (" + COLUMN_ID + " INTEGER PRIMARY KEY," + COLUMN_IDENTITY_HASH + " TEXT)"; public static final String READ_QUERY = "SELECT " + COLUMN_IDENTITY_HASH + " FROM " + TABLE_NAME + " WHERE " + COLUMN_ID + " = " + DEFAULT_ID + " LIMIT 1"; /** * We don't escape here since we know what we are passing. */ public static String createInsertQuery(String hash) { return "INSERT OR REPLACE INTO " + TABLE_NAME + " (" + COLUMN_ID + "," + COLUMN_IDENTITY_HASH + ")" + " VALUES(" + DEFAULT_ID + ", \"" + hash + "\")"; } private RoomMasterTable() { } }
寻着蛛丝马迹,我们发现Room每次编译的时候会在对应的XXXDataBase_Impl实现类中,生成两个Hash值:identityHash和legacyHash。根据字面翻译结合checkIdentity源码可知,这两个hash值是用来检查数据库发生变化后数据库版本号是否跟着update,如果不是则抛出异常。
其中,identityHash不仅存在XXXDataBase_Impl类和RoomOpenHelper类的identityHash字段中,还保存在sqlite_master表中,代表当前数据库结构的“身份证”;而legacyHash只存在XXXDataBase_Impl类和RoomOpenHelper类的mLegacyHash字段,不能在sqlite_master表中。
为什么要两个hash值?一个不行吗?看下面注释就明白了。原来Room V1版本中存在一个bug,如果表结构的字段排序不一样会导致生成的hash值(mLegacyHash)不一样,这样导致的结果是,如果你只是修改了代码中字段的排序(没有对字段名称和数量都没变),就会导致抛出异常提示你升级版本号,显然这样是没有必要的。之后Android官方修复了这个bug,用新的identityHash来处理,同时保留老的mLegacyHash,并且判断条件是必须identityHash和legacyHash都不一致才会抛出异常,这样就能避免上述提到的bug了。
@NonNull private final String mIdentityHash; /** * Room v1 had a bug where the hash was not consistent if fields are reordered. * The new has fixes it but we still need to accept the legacy hash. */ @NonNull // b/64290754 private final String mLegacyHash;
回到正题,为什么onDowngrade按下面方式修改了还不行?
//修改前 @Override public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) { onUpgrade(db, oldVersion, newVersion); } //修改后 @Override public void onDowngrade(SupportSQLiteDatabase db, int oldVersion, int newVersion) { }
原因就是RoomOpenHelper的onOpen方法会在打开数据库时去检查identityHash和legacyHash,由于onDowngrade空实现并没有更新sqlite_master表中的identity_hash字段,这样一来必然会导致checkIdentity判断不通过,因为sqlite_master表中的identity_hash字段是新数据库的,软件降级之后,老版本代码的identityHash和legacyHash是老的数据库的Hash值。
@Override public void onOpen(SupportSQLiteDatabase db) { super.onOpen(db); checkIdentity(db); mDelegate.onOpen(db); // there might be too many configurations etc, just clear it. mConfiguration = null; }
那怎么解决?很简单,要么一不做二不休把checkIdentity也干掉,要么老老实实更新sqlite_master表中的identity_hash字段。显然干掉checkIdentity是不合适的,这样搞把Room的检测机制都搞没了,风险太大,后果未知。
那如何更新identity_hash字段?调用RoomOpenHelper的updateIdentity就行了,上帝给你关上一扇门又会给你打开一扇窗。
private void updateIdentity(SupportSQLiteDatabase db) { createMasterTableIfNotExists(db); db.execSQL(RoomMasterTable.createInsertQuery(mIdentityHash)); }
关键代码如下:
override fun visitMethodInsn( opcode: Int, owner: String?, name: String?, desc: String?, itf: Boolean ) { //移除METHOD_ON_UPGRADE方法调用 if (opcode == INVOKEVIRTUAL && METHOD_ON_UPGRADE == name) { println("rm onUpgrade(db, oldVersion, newVersion) and add updateIdentity(db)") //新增updateIdentity方法调用 mv.visitVarInsn(ALOAD, 0) mv.visitVarInsn(ALOAD, 1) mv.visitMethodInsn( INVOKEVIRTUAL, "androidx/room/RoomOpenHelper", "updateIdentity", "(Landroidx/sqlite/db/SupportSQLiteDatabase;)V", false ) return } super.visitMethodInsn(opcode, owner, name, desc, itf) }
最后再看下反编译的后的RoomOpenHelper:
最终多次测试验证,问题解决,Bingo!
针对本文的这个问题本身,通过修改SDK源码的方式来解决,看起来有点小题大做了,但是这个思路是完全可以应用到其他场景的。比如第三方的库有bug,项目着急上线,没时间找第三方修改,也没有源码,那能怎么办?绝望啊!好在你现在看到了本文,就有了新的解决方案了。有句话怎么说来着,剑可以不用,但是必须要有!对吧,技多不压身,想进步就得多学。
自定义插件,ASM字节码操作和分析,Transform,这些都不是什么新鲜玩意,只要会使用,重要的就是懂得如何利用工具去高效的解决问题,在这过程中读源码是不可避免的,只有知己知彼,才能百战不殆!
lb@lbpc:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample$ sh gradlew clean build publish Welcome to Gradle 7.0.2! Here are the highlights of this release: - File system watching enabled by default - Support for running with and building Java 16 projects - Native support for Apple Silicon processors - Dependency catalog feature preview For more details see https://docs.gradle.org/7.0.2/release-notes.html Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details FAILURE: Build failed with an exception. * Where: Build file '/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample/app/build.gradle' line: 2 * What went wrong: An exception occurred applying plugin request [id: 'com.android.application'] > Failed to apply plugin 'com.android.internal.application'. > Android Gradle plugin requires Java 11 to run. You are currently using Java 1.8. You can try some of the following options: - changing the IDE settings. - changing the JAVA_HOME environment variable. - changing `org.gradle.java.home` in `gradle.properties`. * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/7.0.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 11s lb@lbpc:/home/work/AndroidStudioProjects/AndroidDevelopmentPractices/AndroidASMSample$
解决办法是gradle.properties配置JDK的路径:
org.gradle.java.home=/work/android/android-studio-4.0/android-studio/jre