分布式推荐系统中，通信消耗优化方案

Distributed Mean Estimation with Limited Communication

在分布式推荐系统中，矩阵分解MF 之后，在传输数据的时候，随着数据维度的增大，则通信资源的消耗IO流的消耗也增大，降低通信消耗对算法运行的时间有着明显的提升，本文通过以上论文中的部分描述，实现了基于量化函数的相关优化点。同时本文的工作也参考了论文：

DS-ADMM++: A Novel Distributed Quantized  ADMM to Speed up Differentially Private Matrix

Factorization

这两篇文章的算法实现比较简单，

说那么多直接上代码：

import sys
import time
import numpy as np
import random
import pyopencl  # parallel calculation.
import numba  # invoking CUDA api.
from numba import vectorize
from numba import cuda

'''
@name: kenny adelaide
@email；[email protected]
@description:  quantization protocol πsb
'''

class DS_ADMM_MF:


   def mean_GLOBLE_V(self,matrix):
       '''
       calkculating the mean vector via column.
       '''

       return np.array(matrix).sum() / np.array(matrix).shape[0]


   def generate_normal_distribution_matrix(self,shape=[100, 100]):
       '''
       this is a random init function for a matrix , just for a new normal distribution and zero mean value.
       '''

       matrix = np.array(np.random.normal(0, 1, size=shape[0] * shape[1])).reshape(shape[0], shape[1])
       return matrix


   def calculate_Z_subscript_i(self,matrix, X_subscript_i):
       '''
       this is a function for calculating Z-subscript(i) = R*X-subscript(i). i had denoted the any client machine.
       result discription: matrix's dimension is d*d, hence, returning value is a vector.
       '''

       X_subscript_i = np.array(X_subscript_i).reshape(X_subscript_i.shape[0], 1)

       return np.dot(matrix.T, X_subscript_i).T


   def stochastic_torated_binary_quantization(self,matrix, X_subscript_i):
       '''
        matrix is a init-normal-distribution  and zero-mean-value variable,
        X is input variable.
        attention: matrix has denoted R, that is a global random rotation matrix.
       '''

       # 1.0  first, to calculate X-subscript(i)'s max and min value, attention: X-subscript(i) is a vector.
       # invoking stochastic_torated_binary_quantization method and return Z-subscript(i).

       Z_subscript_i = self.calculate_Z_subscript_i(matrix, X_subscript_i)
       # print(Z_subscript_i)

       max_min_values = []
       max_min_values.append(np.max(Z_subscript_i))
       max_min_values.append(np.min(Z_subscript_i))

       Y_subscript_i_j = np.zeros(shape=Z_subscript_i.shape)

       # 2.0, calculate the quantized vector.
       for index, val in enumerate(Z_subscript_i):
           Y_subscript_i_j[index] = (Z_subscript_i[index] - max_min_values[1]) / (max_min_values[0] - max_min_values[1])

       return Y_subscript_i_j, max_min_values


   def number_of_certain_probability(self,sequence, probability):
       '''
       this is a method to calculate some two value with probs.
       sequence stroes two numbers.
       '''

       # result = []
       #
       # for index, val in enumerate(probability[0]):
       #     probs = [val, 1 - val]
       #     x = random.uniform(0, 1)
       #     cumulative_prob = 0.0
       #     for item, item_prob in zip(sequence, probs):
       #         cumulative_prob += item_prob
       #         if x <= cumulative_prob:
       #             result.append(sequence[1])
       #         else:
       #             result.append(sequence[0])
       #
       # return result[0:int(len(np.array(result)) / 2)]

       result = []

       for index, val in enumerate(probability[0]):
            probs = [val, 1 - val]
            nums = np.random.choice(sequence, size=1, p=probs)
            result.append(nums)

       return result


   def mean(self,matrix):
       matrix = np.array(matrix)
       vector = []
       for i in range(0, matrix.shape[1]):
           vector.append(np.sum(matrix[:, i])/ matrix.shape[0])

       return vector


   def error(self,d,n, Z_subscript_i_max, Z_subscript_i_min):
      '''
      this is a certion for the result of estimation.
      '''

      for i in range(1,n):
         pass

if __name__ == "__main__":

    model = DS_ADMM_MF()
    matrix = model.generate_normal_distribution_matrix(shape=[10, 10])
    X = np.array([[1, 2, 3, 4, 5, 3, 2, 1, 2, 3], [2, 3, 4, 4, 7, 3, 1, 9, 0, 2], [4, 1, 1, 6, 7, 3, 0, 6, 8, 4]])

    Y = []
    for _, val in enumerate(X):
        Y_subscript_i_j, max_min_values = model.stochastic_torated_binary_quantization(matrix, X[_])
        Y_subscript_i_j = model.number_of_certain_probability(max_min_values, Y_subscript_i_j)
        Y.append(Y_subscript_i_j)
    result = model.mean(Y)
    _X = np.dot(np.matrix(matrix).T, np.matrix(result).T).T
    print(_X)

在以上的代码中，没有实现有关算法的标准衡量，可以参考论文自行实现。

以上是最主要的部分，算法的实现不难，难点在于对于公式的理解。  w.p 是以某种概率生成指定的数值。   该算法结合sgd 针对用户评分矩阵实现分布式计算有着良好的效果。