CommunityGAN Community Detection with Generative Adversarial Nets

CommunityGAN Community Detection with Generative Adversarial Nets

这篇文章主要的目标是在建模一个关于图节点的表征。在这个表征中，希望能够把节点的社区性质表达出来。

这样的话，社区的区分之间就可以有重叠。

Introduction

社区检测问题：识别和分析共享高度相似属性或函数的顶点组

传统的问题假设是一个点只属于一个社区。如果把这个限制放开，重叠的社区检测问题是有很大研究价值的。

现在关于图的一些问题（比如说链接预测、推荐、节点分类等）是通过对节点进行表征，即先对整个图进行建模，之后的第二个模型再进行进一步的预测。

然而，由于社区的密集重叠，这种嵌入在重叠的社区检测问题中的应用仍然存在许多局限性。

Generally, the useful information of vertex embedding vectors is the relevant distance of these vectors, while the specific value in vertex embedding vectors has no meanings.一般来说，顶点嵌入向量的有用信息是这些向量的相关距离，而顶点嵌入向量中的特定值没有意义。（比如说node2vec，LINE等等）

如果要给一些顶点特定的向量值，我们可能要考虑用一些其他的方法（比如说逻辑回归等等）。关于检测社区，一般考虑一些邻近算法，NN、k-NN。我们试图在一个统一的框架中同时进行社区检测和网络嵌入，并且去解决社区重叠的问题（解决密集的重叠问题to solve the dense overlapping problem）。

关注motif，也就是图的结构（比如说环）

Thus in this paper, unlike most previous works considering relationship between only two vertices (the relationship between a center vertex and one of the other vertices in a window), we try to generate and discriminate motifs.在本文中，不像以往的大多数工作只考虑两个顶点之间的关系（一个中心顶点和窗口中的另一个顶点之间的关系），我们试图生成和区分motifs。

RELATEDWORK

Community Detection

多种不同角度的社区检测算法：

• One direction is to design some measure of the quality of a community like modularity, and community structure can be uncovered by optimizing such measures.一个方向是设计一些度量社区质量的方法，比如模块化，通过优化这些度量方法可以发现社区结构。

• Another direction is to adopt the generative models to describe the generation of the graphs, and the communities can be inferred by fitting graphs to such models.另一个方向是采用生成模型来描述图的生成，通过拟合图来推断群落。

• Moreover, some models focus on the graph adjacency matrix and output the relationship between vertices and communities by adopting matrix factorization algorithms on the graph adjacency matrix.此外，一些模型侧重于图的邻接矩阵，通过对图的邻接矩阵采用矩阵因子分解算法，输出顶点与社区之间的关系。

• These models often consider the dense community overlapping problem and detect overlapping communities.这些模型经常考虑密集的社区重叠问题，并检测出重叠的社区。

  However, the performance of these methods are restricted by performing pair reconstruction with bi-linear models.然而，由于采用双线性模型进行对重建，限制了这些方法的性能。

Graph representation learning图表示学习

对于一般图的表征学习：

• DeepWalk：shows the random walk in a graph is similar to the text sequence in natural language.显示图中的随机游走类似于自然语言中的文本序列。
• Node2vec：extends the idea of DeepWalk by proposing a biased random walk algorithm, which provides more flexibility when generating the sampled vertex sequence.进一步扩展了深度步的思想，提出了一种有偏随机步算法，在生成采样顶点序列时提供了更大的灵活性。
• LINE：learns the vertex representation preserving both the first and second order proximities.首先学习保留一阶近似和二阶近似的顶点表示。
• GraRep：applies different loss functions defined on graphs to capture different k-order proximities and the global structural properties of the graph.应用图上定义的不同损失函数来捕捉图的不同k阶近似性和全局结构特性。
• GraphGAN：proposes a unified adversarial learning framework, which naturally captures structural information from graphs to learn the graph representation.提出了一种统一的对抗性学习框架，它可以自然地从图中获取结构信息来学习图的表示形式。
• ANE：utilizes GAN as a regularizer for learning stable and robust feature extractor.利用GAN作为正则化器来学习稳定鲁棒的特征提取器。

For community detection tasks, we have to adopt other clustering algorithms on vertex embeddings, which cannot handle the dense community overlapping problem.如果用表征来解决社区检测问题，得采用其他基于顶点嵌入的聚类算法，但是这样无法处理密集的社区重叠问题。

CommunityGAN可以直接将顶点的成员关系输出到社区。

Unified framework for graph representation learning and community detection图表示学习和社区检测的统一框架

最初的想法是从矩阵、张量分解开始的。但是，这样没办法处理大规模数据。

EMPIRICAL OBSERVATION经验观察

通过分析一些真实的网络来得到一些信息。

数据：在线社交网络（online social network），协作网络（collaboration network），产品网络（product network）

Empirical Observations经验观测值

两个关键问题：

• How do the communities contribute to the generation of motifs?社区如何为图结构的产生做出贡献？
• What is the change in motif generation with communities overlapping？随着群落的重叠，图结构产生的变化是什么？

第一个问题

How do the communities contribute to the generation of motifs?社区如何为图结构的产生做出贡献？

研究方法：

• randomly select one community.随机选择一个社区。

• sample 2/3/4 vertices from this community and judge whether they could compose a motif or not.从这个群落中选取2/3/4个顶点作为样本，判断它们是否可以构成一个motif。

  • 为什么是只选取2/3/4个顶点呢？

    we mostly focus on a particular kind of motifs (clique), we only demonstrate the occurrence probability of 2/3/4-vertex cliques.我们主要关注一种特殊的结构（群），我们只证明2/3/4顶点群的出现概率。

    数据表明，一个群落中顶点群的平均出现概率远高于从整个网络中随机选取的顶点群。the average occurrence probabilities of cliques for vertices in one community are much higher than that for vertices randomly selected from the whole network.

    

    （表2：从所有顶点或从一个社区采样的顶点的团出现的概率。R:从所有顶点。C:来自一个社区。）

• 重复很多次，得到在一个社区中某一个motif发生的概率

第二个问题

What is the change in motif generation with communities overlapping？随着群落的重叠，图结构产生的变化是什么？

有些人的研究2-clique概率和社区重叠之间的关系，表明：两个顶点共有的社区越多，它们成为2-clique的概率就越高。

我们进一步探究3-clique和4-clique：the probability curve increases in the overall trend as the number of shared communities increases.随着共享社区数量的增加，总体趋势的概率曲线呈上升趋势。（在下图中有直观的感受）

Such observation accords with the base assumption of AGM framework that vertices residing in communities’ overlaps are more densely connected to each other than the vertices in a single community.这种观察符合AGM框架的基本假设，即居住在社区重叠部分的顶点之间的联系比单个社区中的顶点更为紧密。

METHODOLOGY方法论

（前方数学预警！！！）

CommunityGAN Framework

本篇论文我们只关心cliques这种结构。we only focus on a particular kind of motifs: cliques

根据上面的定义，$M(v_c)$可以被视为从$p_{true}(m|v_c)$中的一组观察到的结构。

生成器G和判别器D在做一个minimax游戏，目标函数：
$$\underset{{\theta }_G}{\min }\underset{{\theta }_D}{\max }V(G,D)=\sum _{c=1}^V(E_{m\sim p_{true}(\cdot |v_c)}[\log D(m;{\theta }_D)]+E_{s\sim G(s|v_c;{\theta }_G)[\log (1-D(s;{\theta }_D))]})$$
CommunityGAN的框架：

判别器D：从$p_{true}(\cdot |v_c)$中采正样例，从$G(\cdot |v_c;{\theta }_G)$中采负样例

CommunityGAN Optimization

$${\nabla }_{{\theta }_D}V(G,D)=\sum _{c=1}^V(E_{m\sim p_{true}(\cdot |vc)}[{\nabla }_{{\theta }_D}\log D(m;{\theta }_D)]+E_{s\sim G(s|v_c;{\theta }_G)}[{\nabla }_{{\theta }_D}\log (1-D(s;{\theta }_D))])$$

s是离散的：
$${\nabla }_{{\theta }_D}V(G,D)={\nabla }_{{\theta }_D}\sum _{c=1}^V{E}_{s\sim G(\cdot |v_c)}[\log (1-D(s))]=\sum _{c=1}^V{E}_{s\sim G(\cdot |v_c)}[{\nabla }_{{\theta }_D}\log G(s|v_c)\log (1-D(s))])$$
（上面这个式子是怎么变的没有看懂……）

A Naive Implementation of D and G

最简单的想法就是直接上各种函数（sigmod，softmax）

For the discriminator D, intuitively we can define it as the multiplication of the sigmoid function of the inner product of every two vertices in the input vertex subset s.对于判别器D，我们可以直观地将其定义为输入顶点子集s中每两个顶点的内积的sigmoid函数的乘积。
$$D(s)=\prod _{(u,v)\in s,u̸=v}\sigma (d_u^T\cdot d_v)$$
其中$d_u,d_v\in {\mathbb{R}}^k$是在判别器中节点$u$和$v$的k维向量表示。用${\theta }_D$来表示判别器对于顶点所生成的空间。

对于生成器G：

to generate a vertex subset s covering vertex vc , we can regard the subset as a sequence of vertices为了生成一个覆盖顶点vc的顶点子集s，我们可以把这个子集看作一个顶点序列：$(v_{s_1},v_{s_2},...,v_{s_m})$，$v_{s_1}=v_c$。G的定义如下：
$$G(s|v_c)=G_v(v_{s_2}|v_{s_1})G_v(v_{s_3}|v_{s_1},v_{s_2})...G_v(v_{s_m}|v_{s_1},...,v_{s_{m-1}})$$
**我们是基于$v_{s_1}$到$v_{s_{m-1}}$来生成的$v_m$，而不仅仅是$v_{s_{m-1}}$。如果我们只基于$v_{s_{m-1}}$来生成，那么$v_{s_m}$很有可能和其他顶点不属于同一个社区。**举个例子，$v_{s_1}$到$v_{s_{m-1}}$都是同一所大学的学生，而只基于$v_{s_{m-1}}$生成的$v_{s_m}$很有可能是$v_{s_{m-1}}$的家长。

Simply, we can know the probability of the vertex subset s being a clique will be very low.简单地说，我们知道顶点子集$s$是一个小集团的概率非常低。

For the implementation of the vertex generator $G_v$, straightforwardly, we can define it as a softmax function over all other vertices.对于顶点生成器$G_v$的实现，我们可以直接将它定义为所有顶点上的一个softmax函数：
$$G_v(v_{s_m}|v_{s_1},v_{s_2},...,v_{s_{m-1}})=\frac{\exp ({\sum }_{i=1}^{m-1}{g}_{v_{s_m}}^T{g}_{v_{s_i}})}{{\sum }_{v̸\in (v_{s_1},v_{s_2},...,v_{s_{m-1}})}\exp ({\sum }_{i=1}^{m-1}{g}_v^T{g}_{v_{s_i}})}$$
这里$g_v\in {\mathbb{R}}^k$是在生成器G中关于节点$v$的向量表征，${\theta }_G$是生成器G的k维空间。

Graph AGM

naive想法的局限性

（上面那种naive的想法由什么问题呢？）

Sigmoid and softmax function provide concise and intuitive definitions for the motif discrimination in Ds and vertex generation in Gv , but they have three limitations in community detection task.sigmoid和softmax函数关于在Ds和顶点生成图结构，歧视提供简洁和直观的定义。但是他们有三个局限：

• To detect community, after learning the vertex representation vectors based on Eq. (4) and (6), we still need to adopt some clustering algorithms to detect the communities.生成器G和判别器D学习后的顶点表征向量，为了检测社区，仍需要采取一些聚类算法来检测的社区。
  • the overlap is indeed a significant feature of many real-world social networks.重叠确实是许多现实社会网络的一个重要特征。
    • The authors showed that in some real-world datasets, one vertex might belong to tens of communities simultaneously.在一些实际数据集中，一个顶点可能同时属于几十个群落。
    • general clustering algorithms cannot handle such dense overlapping.一般的聚类算法无法处理如此密集的重叠。
• sigmoid和softmax函数计算效率低
• The graph structure encodes rich information of proximity among vertices, but the softmax in Eq. (6) completely ignores it.图结构编码了丰富的顶点间接近性信息，但式(6)（生成器G的那个式子）中的softmax完全忽略了这一点。

AGM

基本框架

AGM (Affiliation Graph Model) is based on the idea that communities arise due to shared group affiliation, and views the whole network as a result generated by a community-affiliation graph model.AGM(隶属关系图模型)是基于共享的群体隶属关系而产生社区的思想，将整个网络看作是隶属关系图模型生成的结果。

The framework of AGM is illustrated in Figure 4, which can be either seen as a bipartite network between vertices and communities, or written as a nonnegative affiliation weight matrix.AGM的框架如图4（上图）所示，既可以看作是顶点与群体之间的二部网络，也可以写成非负隶属权矩阵（每个点$v$都有一个隶属于某个社区$c$的强度：$F_{vc}$）。在AGM中，每个顶点可以隶属于0个、一个或多个社区。

For any community $c\in C$, it connects its member vertices $u,v$ with probability $1-\exp (F_{uc}\cdot F_{vc})$.对于任何社区，它连接其成员顶点u,v与概率。

Moreover, each community c creates edges independently.此外，每个社区c都独立地创建边。（所以，当有两个点通过多个社区相连的时候，是可能直接相加考虑的）

If the pair of vertices $u,v$ are connected multiple times through different communities, the probability is $1-\exp ({\sum }_c{F}_{uc}\cdot F_{vc})$.如果$u、v$这对顶点通过不同的社区多次连接，则概率为$1-\exp ({\sum }_c{F}_{uc}\cdot F_{vc})$。

So that the probability that vertices u,v are connected (through any possible communities) is $p(u,v)=1-exp(F_u\cdot F_v)$, where $F_u$ and $F_v$ are the nonnegative C-dimensional affiliation vectors for vertices $u$ and $v$ respectively.因此，顶点$u,v$(通过任何可能的社团)连接的概率为$p(u,v)=1-exp(F_u\cdot F_v)$，其中$F_u$和$F_v$分别为顶点$u$和$v$的非负$c$维向量。

从边生成扩展到结构生成

For any $m$ vertices $v_1$ to $v_m$, we assume that the probability of them to be a clique through of community $c$ is defined as $p_c(v_1,v_2,...,v_m)=1-\exp (-{\prod }_{i=1}^m{F}_{vi}c)$.对于任意$m$个顶点$v_1$到$v_m$，我们假设它们通过社区$c$成为一个小团体的概率定义为……

Then the probability that these $m$ vertices compose a clique through any possible communities can be calculated via：然后这些m个顶点通过任何可能的社区组成一个小团体的概率可以通过下面这个式子进行计算：
$$\begin{gathered} p(v_1,v_2,...,v_m)=1-\prod _c(1-p_c(v_1,v_2,...,v_m)) \\ =1-\exp (-\odot (F_{v_1},F_{v_2},...,F_{v_m})) \end{gathered}$$
⊙表示从上到下的乘积的和：$\odot (F_{v_1},F_{v_2},...,F_{v_m})={\sum }_{c=1}^C{\prod }_{i=1}^m{F}_{v_{i}c}$

Then the discriminator, which was defined as the product of sigmoid in a straightforward way, can be redefined as然后将判别器（之前的naive想法是直接定义为sigmoid的乘积）重新定义为：$D(s)=1-\exp (-\odot (d_{v_1},d_{v_2},...,d_{v_m}))$

Moreover, the generator $G_v$ can be redefined as the softmax function over all other possible vertices to compose a clique with $m-1$ chosen vertices此外，生成器$G_v$可以被重新定义为所有其他可能顶点上的softmax函数，以组成一个包含$m-1$选择顶点的小团体：$G_v(v_{s_m}|v_{s_1},v_{s_2},...,v_{s_{m-1}})=\frac{1-\exp (-\odot (g_{v_1},g_{v_2},...,g_{v_m}))}{{\sum }_{v̸\in (v_{s_1},v_{s_2},...,v_{s_{m-1}})}1-\exp (-\odot (g_{v_{s_1}},g_{v_{s_2}},...,g_{v_{s_{m-1}}},g_v)}$

With this setting, the learned vertex representation vectors $g_v$ will represent the affiliation weight between vertex $v$ and communities, which means we need no additional clustering algorithms to find the communities and the first aforementioned limitation is omitted.通过这个设置，学习到的顶点表示向量$g_v$将表示顶点$v$与社团之间的隶属权，这意味着我们不需要额外的聚类算法来寻找社团，并且省略了前面提到的第一个限制。

下一步克服其他两个限制

sigmoid和softmax函数计算效率低

we first assume there is a virtual vertex $v_v$ which is connected to all the vertices in the union of neighbors of vertices from $v_{s_1}$ to $v_{s_{m-1}}$.我们首先假设有一个虚拟顶点$v_v$，它连接到从$v_{s_1}$到$v_{s_{m-1}}$，$N(v_v)=N(v_{s_1})\cup \cdot \cdot \cdot \cup N(v_{s_{m-1}})$。

关于这个虚拟节点的向量表征是什么？$g_{v_v}=g_{v_1}◦g_{v_2}◦...◦g_{v_{s_{m-1}}}$。the representation vector of $v_v$ is the entrywise product of the representation vectors of vertices from $v_{s_1}$ to $v_{s_{m-1}}$.

同时将$G_v(v_{s_m}|v_{s_1},v_{s_2},...,v_{s_{m-1}})$简化成了$G_v(v_{s_m}|v_v)$。

然后根据一个概率函数，从$v_v$进行随机游走：During the process of random walk, if the currently visited vertex is $v$ and generator $G$ decides to visit $v$'s previous vertex, then $v$ is chosen as the generated vertex and the random walk process stops.在随机游走过程中，如果当前访问的顶点为$v$，生成器$G$决定访问v之前的顶点，则选择v作为生成的顶点，随机游走过程停止。（下面是一个例子，顶点子集s的生成过程。蓝色箭头表示随机游走的路径。在两个蓝色顶点处，因为顶点生成器决定访问前一个顶点，所以随机游走结束，并选择蓝色顶点。）

（那随机游走的概率函数是怎么算的？）

Moreover, in the random walk process, we wish the walk path is always relevant to the root vertex $v_v$ for maximizing the probability of the generated vertex subset to be a motif.此外，在随机游走过程中，为了使生成的顶点子集成为基元的概率最大化，我们希望行走路径始终与根$v_v$相关。

So that for a given vertex $v_c$ and one of its neighbors $v_i\in N(v_c)$, we define the relevance probability of $v_i$ given $v_c$ as.对于一个给定的顶点$v_c$和它的一个邻居$v_i\in N(v_c)$，我们定义$v_i$在$v_c$下的关联概率为：
$$p_{v_v}(v_i|v_c)=\frac{1-\exp (-\odot (g_{v_i},g_{v_c},g_{v_v}))}{{\sum }_{v_j\in N(v_c)}1-\exp (-\odot (g_{v_j},g_{v_c},g_{v_v}))}$$
这个函数的本质是一个softmax除以N(vc)用顶点vv和vc组成小团体？？

解决不能反映图结构的问题

（还是通过随机游走来采样解决对图建模的问题）

If we denote the path of random walk as $Pr=(v_{r_1},v_{r_2},...,v_{r_n})$ where $v_{r_1}=v_v$ , the probability for selecting this path will be $p_{v_v}(v_{r_{n-1}}|v_{r_n})\cdot {\prod }_{i=1}^{n-1}{p}_{v_v}(v_{r_{i+1}}|v_{r_i})$.如果我们用$Pr=(v_{r_1},v_{r_2},...,v_{r_n})$来表示$v_{r_1}=v_v$的随机游走路径，那么选择这条路径的概率为。

In the policy gradient, we regard the selection of this path as an action and the target is to maximize its reward from D. Thus, although there may be multiple paths between $v_{r_1}$ and $v_{r_n}$ , if we have selected the path $P_r$ , we will optimize the policy gradient on it and neglect other paths.在策略迭代中，我们认为选择这条路径作为一个行动和目标是为了最大化从D的奖励。因此，尽管会有多条路径在$v_{r_1}$和$v_{r_n}$之间，如果我们选择路径$P_r$，我们将很久它来优化，同时忽视其他路径。

In other words, if we select the path $P_r$ , we assign $G_v(v_{s_m}|v_v)$ as follows.换句话说，如果我们选择路径$P_r$，我们分配$G_v(v_{s_m}|v_v)=p_{v_v}(v_{r_{n-1}}|v_{r_n})\cdot {\prod }_{i=1}^{n-1}{p}_{v_v}(v_{r_{i+1}}|v_{r_i})$

Other Issues

Model Initialization

两种方法：

• deploy AGM model on the graph to learn a community affiliation vector $F_i$ for each vertex $v_i$ , and then we can set $g_{v_i}=d_{v_i}=F_i$ directly.在图上部署AGM模型，学习关于每一个点$v_i$的社区关系向量$F_i$，直接将$g_{v_i}=d_{v_i}=F_i$。

• use locally minimal neighborhoods to initialize ${\theta }_G$ and ${\theta }_D$.使用局部最小邻域

  • 什么是使用局部最小邻域：
    • We can regard each vertex $v_i$ along with its neighbors $N(v_i)$, denoted as $C(v_i)$, as a community.我们可以将每个顶点$v_i$及其邻居$N(v_i)$(记作$C(v_i$))看作一个社区。
    • Community $C(v_i)$ is called locally minimal if $C(v_i)$ has lower conductance than all the $C(v_j)$ for vertices $v_j$ who are connected to vertex $v_i$ .对于与顶点$v_i$相连的顶点$v_j$，如果$C(v_i)$的conductance比所有的$C(v_j)$都低，则称为局部极小。
    • conductance在这里怎么理解？？？
  • For a node $v_i$ who belongs to a locally minimal neighborhood $c$, we initialize $F_{v_{i}c}=1$, otherwise $F_{v_{i}c}=0$.对于属于局部最小邻域$c$的节点$v_i$
  • 这次方法省时，但性能低于第一种

Determining community membership确定社区成员

对于每一个点，我们想决定一个连接最紧密的点（the “hard” community membership）

通过加阈值来达到这一点。

The basic intuition is that if two nodes belong to the same community $c$, then the probability of having a link between them through community $c$ should be larger than the background edge probability.基本的直觉是，如果两个节点属于同一个社区c，那么它们之间通过社区c链接的概率应该大于背景边缘概率$ϵ=\frac{2E}{V(V-1)}$（这是一个常数，这个常数是怎么设定的？？？）

于是，我们设置这个阈值为$\delta =\sqrt{-\log (1-ϵ)}$

当一个点的$g_{v_{i}c}\ge \delta$或者$d_{v_{i}c}\ge \delta$时，可以认为$v_i$在社区$c$里面。

Choosing the number of communities选择社区的数量

We follow the method proposed in [1] to choose the number of communities C. Specifically, we reserve 20% of links for validation and learn the model parameters with the remaining 80% of links for different C. After that, we use the learned parameters to predict the links in validation set and select the C with the maximum prediction score as the number of communities.

我们遵循[1]中提出的方法选择的社区数量C。具体地说,我们保留20%的链接进行验证和学习剩下的80%的模型参数链接不同的C之后,我们使用学习参数预测中的链接验证设置,选择C与最大预测分数社区的数量。