项目依赖关系分析中的数据结构

项目中常见问题

  1. 源码依赖可能导致相互直接或间接依赖形成环,对此应该如何快速检测呢?
  2. 源码依赖对于如commonLib大版本升级需要放开依赖该commonLib的其他aar源码验证,我们如何快速获取呢?
思路
  • 针对以上问题,对于同城,同镇目前通过config维护,我们可以从该文件着手提出解决方案;项目相关的敏感信息一律以XX代替!版本信息一律为1.0.0
  1. aar与源码切换关系由switchs集合设置:lib总计41
switchs = [
        "58AppDependenciesLib"    : dependencyMode == "aar",
        "58ClientHybridLib"       : dependencyMode == "aar",
        "58HouseLib"              : dependencyMode == "aar",
        "58HuangyeLib"            : dependencyMode == "aar",
        "58PincheLib"             : dependencyMode == "aar",
        "58PartTimeLib"           : dependencyMode == "",
        "WubaCommonsLib"          : dependencyMode == "aar",
        "WubaHybridBusinessLib"   : dependencyMode == "aar"
        ...省略...
]
  1. 若设置为源码存储位置设置在setting配置:地址为当前项目路径下
include switchs['58ClientHybridLib']?'':'58ClientHybridLib'
include switchs['58HouseLib']?'':'58HouseLib'
include switchs['58AnjukeLib'] ? '' : '58AnjukeLib'
include switchs['58CarLib']?'':'58CarLib'
 ...省略...
  1. 每个module即lib的依赖关系也在config中配置
 WubaPartTimeLib = [
            switchs['58PartTimeLib'] ? "com.wuba.XXX:" + "$WubaPartTimeLibVersion" : findProject(':58PartTimeLib'),
            rootProject.ext.WubaAppDependenciesLib,
            rootProject.ext.WubaTradelineLib,
            rootProject.ext.WubaWebBusinessLib,
            "com.wuba.XXX:$TtSdkVersion",
            'com.wuba.XXX:1.0.0',
            'com.wuba.XXX:1.0.0',
            rootProject.ext.ZCMPublish,
            rootProject.ext.WubaCommonsLib,
            rootProject.ext.WubaPermissionSDK
    ]
  • 通过观察兼职侧依赖,有如ttsdk这种底层库,还有跟兼职一样可配置aar的module如WubaCommonsLib,其中WubaCommonsLib中也可能依赖其他可配置aar的module,如此嵌套很容易想到一种解决方式:递归调用
实施方案
  1. 首先需要获取需要获取aar与源码切换项目名称,即switchs集合数据,使用map存储
  2. 递归调用获取每个lib依赖其他lib的情况关系如下
key : WubaClientHybridLib  -- > [
com.wubaXXX:100.2.36, 
com.wubaXXXLib:100.22.41, 
com.wubanessLib:101.22.82, 
com.wubaILib:100.20.18, 
com.wubaALib:10.19.2, 
]   
key : WubaCommonsLib  -- > 
[com.wubaCommonsLib:10.26.1, 
com.wuba.SourcesLib:1.0.0, 
com.wuba.LogLib:1.0.1
]
  • 对比以上数据,由于是递归调用当WubaClientHybridLib引用WubaCommonsLib则前者将获取到后者所有的依赖库,则我们使用何种数据格式保存他们直接的关系最为方便直接呢?
  • 要求: 方便快速查看各个lib的依赖关系,即比如给WubaCommonsLib可快速查看其依赖的底层库及它被那些lib所使用,经过对比我们选择使用图!下面简单介绍一下为何使用该方式吧!
  • 定义: 图是一种复杂的非线性结构,图结构中,节点之间的关系是任意的,图中任意两个数据元素之间都有可能相关,图G由两个集合V(顶点Vertex)和E(边Edge)组成,定义为G=(V,E)
  1. 还是使用上方数据举例,比如WubaClientHybridLib,WubaCommonsLib和WubaLogLib三者之间的有向图
//定义三者的顶点集合,这里使用数组也是一样儿的,数组的下标表示对应的lib名称
  String[] strArr = new String[]{
                "WubaClientHybridLib", //0 ->V0
                "WubaCommonsLib", //1->V1
                "WubaLogLib" //2 ->V2
        };
  • 无向图:顶点的度表示以该顶点作为一个端点的边的数目;
  • 有向图:顶点的度分为入度和出度。入度表示以该顶点为终点的入边数目,出度是以该顶点为起点的出边数目,该顶点的度等于其入度和出度之和;
    image
  1. 根据以上分析,如果我们需要获取WubaClientHybridLib的依赖库只需要查找以顶点WubaClientHybridLib即顶点V0出度的边的另一个顶点。相同的如果想获取WubaLogLib被那些lib所依赖只需要获取V2入度的边的另一个顶点。
  • 解决了如果WubaLogLib的aar被删除,可以设置只有此lib为源码依赖还是其被依赖的所有lib均设置为源码依赖,只需要读取WubaLogLib入度就可以啦!
边的表示
  • 图的实际信息都保存在边上,它们描述了图的结构,了解数据结构的都知道,无非就是数组和链表两种形式,图也不例外:表示图的边的方法称为邻接矩阵或邻接表
邻接表
  • 是一种顺序分配和链式分配相结合的存储结构,如这个表头结点所对应的顶点存在相邻顶点,则把相邻顶点依次存放于表头结点所指向的单向链表中;
邻接矩阵
  1. 原理就是用两个数组,一个数组保存顶点集,一个数组保存边集;
- V0 V1 V2
V0 0 1 1
V1 0 0 1
V2 0 0 0
  • 对于每行表示当前lib依赖的其他库关系,如果为1表示存在依赖关系
  • 对于每列表示当前lib被哪些库所依赖,如果1表示也存在依赖关系
  1. 对于config运行获取邻接矩阵关系如下,有兴趣的同学可以自己选择一个查看是否跟总结的一致;
当前key : HouseLib  -- > position :   4
当前key : PincheLib  -- > position :   12
当前key : RxDataSourcesLib  -- > position :   36
当前key : BaseUILib  -- > position :   35
当前key : VideoLib  -- > position :   31
当前key : HybridBusinessLib  -- > position :   40
当前key : FinanceLib  -- > position :   14
当前key : TribeLib  -- > position :   16
当前key : JiaoyouLib  -- > position :   25
当前key : AnjukeLib  -- > position :   20
当前key : JobCommonLib  -- > position :   8
当前key : PartTimeLib  -- > position :   13
当前key : TradelineLib  -- > position :   28
当前key : TribeABLib  -- > position :   17
当前key : TownLib  -- > position :   23
当前key : AOPLib  -- > position :   37
当前key : LogLib  -- > position :   38
当前key : WalleExtLib  -- > position :   32
当前key : IMLib  -- > position :   15
当前key : WalleLib  -- > position :   33
当前key : JobLib  -- > position :   6
当前key : HuangyeLib  -- > position :   11
当前key : RNBusinessLib  -- > position :   18
当前key : SaleLib  -- > position :   10
当前key : HouseLib  -- > position :   2
当前key : BasicBusinessLib  -- > position :   29
当前key : CarLib  -- > position :   5
当前key : LoginLib  -- > position :   21
当前key : ClientHybridABLib  -- > position :   19
当前key : TownPowerLib  -- > position :   24
当前key : LocationLib  -- > position :   22
当前key : QigsawLib  -- > position :   39
当前key : JiaoyouIMLib  -- > position :   26
当前key : JobABLib  -- > position :   7
当前key : ClientHybridLib  -- > position :   1
当前key : AppDependenciesLib  -- > position :   0
当前key : DatabaseLib  -- > position :   30
当前key : NewCarLib  -- > position :   9
当前key : WebBusinessLib  -- > position :   27
当前key : HouseAJKMixLib  -- > position :   3
当前key : CommonsLib  -- > position :   34
查看有向图表示为:int[][] map = new int[][]{            
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,0,1,0,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,1,0,1,1,1,1,1,1,1,1,0,1,1,1},
{0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,1,0},
{1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,1,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,1,1,1,1,1,0,1,0,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,0,1,0,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,1,1,1,1,1,1,0,1,0,0},
{1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,1,0,1,1,1,1,1,1,1,1,0,1,1,1},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,0,1},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,1,0,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,1,1,1,1,1,0,1,0,0},
{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,1,1,1,1,1,1,0,1,0,0}
};
  • 举个例子:对于WubaClientHybridLib 下标为1,即第二列只有一个数据为1在第19行即WubaClientHybridABLib所依赖,查看config中数据果然如此
 ClientHybridABLib = [
      switchs['ClientHybridABLib'] ? "com.wuba.XXX.ClientHybridABLib:$ClientHybridABLibVersion" : findProject('ClientHybridABLib'),
      rootProject.ext.ClientHybridLib
    ]
  • 优化点: 上方邻接矩阵由于我们aar设置集合大小为41,内存占比int[41][41]大致为1681 * 4B = 6.56KB,看起来还可以接受,但随着集合数据增加,内存占比是指数级增长的,比如分析数据量为200,内存占比 200 * 200 * 4B = 156.25KB;如何减少呢?
  • 不使用int改为byte数组,byte还是占1字节,通过观察上方数据:0表示无关,1表示存在依赖关系。那么为什么不使用bite位表示呢,对于上方41大小使用long类型占8字节 = 64bit位搓搓有余了,使用long[41]数组即可表示,内存占比64 * 8B = 0.32KB。获取相互直接关系即判断long中某一位是否为1 即Vi & (1 << (j -1)) != 0 , 判断Vi依赖关系即查询long[i]中二进制为1的位置集合,判断Vi被依赖关系即遍历long[]数组获取二进制第i 位置为1的集合;
    • 注意: 若使用bite位表示将对应增加位运算量,对于数据量较小情况建议使用byte数组(仅一次读取内存操作)
int size = 41 ;
long[] nums = new long[size];
//在long数组中修改arr[i][j]存在依赖关系
nums[i] |= 1L << (size - 1 - j);
//获取long数组中i,j 处数据是否为1
nums[i] &= 1L << (size - 1 - j);
assert(nums[i] != 0)

解决

问题一
  • 通过上方我们可以获取依赖关系的邻接矩阵的关系图,如何快速检测是否存在环呢?经常刷leetCode的同学回忆一下是否见过该问题: 207 课程表,使用两种遍历方式广度遍历BFS和深度遍历DFS
    image
  1. 分析如何检测环,首先获取每个节点的入度集合degrees[n],对于入度为0的加入栈(DFS)或者队列(BFS),这里以BFS举例;
  2. 遍历集合degrees将degrees[i]为0,即入度为0的全部节点入队列后,遍历队列取出队头元素,即断开以该节点出度的边,此时需要修改节点出度边的另一个节点的入度 - 1 后 判断入度是否0,若为0加入队列,循环即可;
//储存节点的入度,统计j列中所有1的个数
int[] degrees = new int[map.length];
   for (int i = 0; i < map.length; i++) {
      for (int j = 0; j < map[0].length; j++) {
         degrees[i] += map[j][i];
      }
}
BFSSearch(map , degrees);

/**
 * BFS广度遍历:使用队列 ; DFS 深度遍历:使用栈
 * LinkedList可实现栈和队列数据:
 * 栈 : push表示入栈,在头部添加元素 , pop表示出栈,返回头部元素 ,peek查看栈头部元素
 * 队列: offer 尾部添加元素, poll 返回头部元素,并且从队列中删除, peek返回头部元素
 */
public void BFSSearch(int[][] map , int[] indegree){
    int count = 0; //判断有无回路(是否成环)
    LinkedList stack = new LinkedList();
    int n = indegree.length ;
    for (int i = 0; i < n; i++) {
      if (indegree[i] == 0) {
      stack.offer(i);     // --------标记点1------------
      indegree[i] = -1;
      }
    }

    while (!stack.isEmpty()) {
      Integer p = (Integer) stack.poll();  //--------标记点2------------
      System.out.print(p + " ");
      count++;
      for (int j = 0; j < n; j++) { //p列入度对应的以p出度的为p行数据
        if (map[p][j] == 1) {
            map[p][j] = 0;
            indegree[j]--;
            if (indegree[j] == 0) {
              stack.offer(j);    // --------标记点3------------
              indegree[j] = -1;
            }
          }
        }
    }
    if(count < n)
    System.out.println("当前存在回路信息!");
    else
    System.out.println("当前没有回路信息!");
}
  1. DFS只需要简单修改几行代码即可实现
标记点1和标记点3
stack.push 入栈
标记点2
stack.pop 出栈
问题二
  • 该问题可以提取为寻找以commonLib入度的边的集合,即统计邻接矩阵commonLib所在列值为1的集合,增加task将集合中的lib修改为源码依赖!

思考

  • 邻接矩阵和临界表均可以表示图,我们以后用到了该如何选择,它们有什么区别呢?
  1. 邻接矩阵
    • 优点: 可以快速判断两个顶点之间是否存在边(查询快);快速添加边或者删除边;
    • 缺点: 如果过多的无关联数据(稀松图),即数组中过多0导致白白浪费空间;同时新增顶点比较麻烦,需要修改,移动数组;
  2. 临界表
    • 优点: 节省空间,只存储实际存在的边。新增顶点比较简单,在入度链表中添加数据即可;
    • 缺点: 关注顶点的度时,就可能需要遍历这一整个链表,有解决方式:1)增加一个反临界表存储该顶底的入度列表;2)使用十字链表表示出度,入度信息

你可能感兴趣的:(项目依赖关系分析中的数据结构)