描述统计的python实现
统计相关
- 1 次序统计
- 1.1 计算最小值
- 1.2 计算最大值
- 1.3 计算极差
- 1.4 计算分位数
- 2 均值与方差
- 2.1 计算中位数
- 2.2 计算平均值
- 2.3 计算加权平均值
- 2.4 计算方差
- 2.5 计算标准差
- 3 相关
- 3.1 计算相关系数
- 4 直方图
次序统计
计算最小值
- numpy.amin(a[, axis=None, out=None, keepdims=np._NoValue, initial=np._NoValue,
where=np._NoValue]) Return the minimum of an array or minimum along an axis.
import numpy as np
x = np.array([[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35]])
y = np.amin(x)
print(y) # 11 #整个数组所有元素的最小值
y = np.amin(x, axis=0) #axis=0代表跨行，每一列运算
print(y) # [11 12 13 14 15]
y = np.amin(x, axis=1) #axis=1代表跨列，每一行运算
print(y) # [11 16 21 26 31]
11
[11 12 13 14 15]
[11 16 21 26 31]
计算最大值
- numpy.amax(a[, axis=None, out=None, keepdims=np._NoValue, initial=np._NoValue,
where=np._NoValue]) Return the maximum of an array or maximum along an axis.
import numpy as np
x = np.array([[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35]])
y = np.amax(x)
print(y) # 35
y = np.amax(x, axis=0)
print(y) # [31 32 33 34 35]
y = np.amax(x, axis=1)
print(y) # [15 20 25 30 35]
35
[31 32 33 34 35]
[15 20 25 30 35]
计算极差
- numpy.ptp(a, axis=None, out=None, keepdims=np._NoValue) Range of values (maximum -
minimum) along an axis. The name of the function comes from the acronym for ‘peak to
peak’.
import numpy as np
np.random.seed(20200623)
x = np.random.randint(0, 20, size=[4, 5])
print(x)
# [[10 2 1 1 16]
# [18 11 10 14 10]
# [11 1 9 18 8]
# [16 2 0 15 16]]
print(np.ptp(x)) # 18
print(np.ptp(x, axis=0)) # [ 8 10 10 17 8]
print(np.ptp(x, axis=1)) # [15 8 17 16]
[[10 2 1 1 16]
[18 11 10 14 10]
[11 1 9 18 8]
[16 2 0 15 16]]
18
[ 8 10 10 17 8]
[15 8 17 16]
计算分位数
- numpy.percentile(a, q, axis=None, out=None, overwrite_input=False,
interpolation=‘linear’, keepdims=False) Compute the q-th percentile of the data along
the specified axis. Returns the q-th percentile(s) of the array elements.
a：array，用来算分位数的对象，可以是多维的数组。
q：介于0-100的float，用来计算是几分位的参数，如四分之一位就是25，如要算两个位置
的数就[25,75]。
axis：坐标轴的方向，一维的就不用考虑了，多维的就用这个调整计算的维度方向，取值范
围0/1
import numpy as np
np.random.seed(20200623)
x = np.random.randint(0, 20, size=[4, 5])
print(x)# [[10 2 1 1 16]
# [18 11 10 14 10]
# [11 1 9 18 8]
# [16 2 0 15 16]]
print(np.percentile(x, [25, 50])) #上四分位数和中位数
# [ 2. 10.]
print(np.percentile(x, [25, 50], axis=0))
# [[10.75 1.75 0.75 10.75 9.5 ]
# [13.5 2. 5. 14.5 13. ]]
print(np.percentile(x, [25, 50], axis=1))
# [[ 1. 10. 8. 2.]
# [ 2. 11. 9. 15.]]
[[10 2 1 1 16]
[18 11 10 14 10]
[11 1 9 18 8]
[16 2 0 15 16]]
[ 2. 10.]
[[10.75 1.75 0.75 10.75 9.5 ]
[13.5 2. 5. 14.5 13. ]]
[[ 1. 10. 8. 2.]
[ 2. 11. 9. 15.]]
均值与方差
计算中位数
- numpy.median(a, axis=None, out=None, overwrite_input=False, keepdims=False) Compute
the median along the specified axis. Returns the median of the array elements.
import numpy as np
np.random.seed(20200623)
x = np.random.randint(0, 20, size=[4, 5])
print(x)
# [[10 2 1 1 16]
# [18 11 10 14 10]
# [11 1 9 18 8]
# [16 2 0 15 16]]
print(np.percentile(x, 50))
print(np.median(x)) #两种函数都可输出中位数
# 10.0
print(np.percentile(x, 50, axis=0))
print(np.median(x, axis=0))
# [13.5 2. 5. 14.5 13. ]
print(np.percentile(x, 50, axis=1))
print(np.median(x, axis=1))
# [ 2. 11. 9. 15.]
[[10 2 1 1 16]
[18 11 10 14 10]
[11 1 9 18 8]
[16 2 0 15 16]]
10.0
10.0
[13.5 2. 5. 14.5 13. ]
[13.5 2. 5. 14.5 13. ]
[ 2. 11. 9. 15.]
[ 2. 11. 9. 15.]
计算平均值
- numpy.mean(a[, axis=None, dtype=None, out=None, keepdims=np._NoValue)]) Compute the
arithmetic mean along the specified axis
import numpy as np
x = np.array([[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35]])
y = np.mean(x)
print(y) # 23.0
y = np.mean(x, axis=0)
print(y) # [21. 22. 23. 24. 25.]
y = np.mean(x, axis=1)
print(y) # [13. 18. 23. 28. 33.]
23.0
[21. 22. 23. 24. 25.]
[13. 18. 23. 28. 33.]
计算加权平均值
- numpy.average(a[, axis=None, weights=None, returned=False]) Compute the weighted
average along the specified axis.
mean 和 average 都是计算均值的函数，在不指定权重的时候 average 和 mean 是一样的。指定权重
后， average 可以计算加权平均值。
import numpy as np
x = np.array([[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35]])
y = np.average(x)
print(y) # 23.0
y = np.average(x, axis=0)
print(y) # [21. 22. 23. 24. 25.]
y = np.average(x, axis=1)
print(y) # [13. 18. 23. 28. 33.]
y = np.arange(1, 26).reshape([5, 5])
print(y)
# [[ 1 2 3 4 5]# [ 6 7 8 9 10]
# [11 12 13 14 15]
# [16 17 18 19 20]
# [21 22 23 24 25]]
z = np.average(x, weights=y)
print(z) # 27.0
z = np.average(x, axis=0, weights=y)
print(z)
# [25.54545455 26.16666667 26.84615385 27.57142857 28.33333333]
z = np.average(x, axis=1, weights=y)
print(z)
# [13.66666667 18.25 23.15384615 28.11111111 33.08695652]|
23.0
[21. 22. 23. 24. 25.]
[13. 18. 23. 28. 33.]
[[ 1 2 3 4 5]
[ 6 7 8 9 10]
[11 12 13 14 15]
[16 17 18 19 20]
[21 22 23 24 25]]
27.0
[25.54545455 26.16666667 26.84615385 27.57142857 28.33333333]
[13.66666667 18.25 23.15384615 28.11111111 33.08695652]
计算方差
- numpy.var(a[, axis=None, dtype=None, out=None, ddof=0, keepdims=np._NoValue]) Compute
the variance along the specified axis.
ddof=0：是“Delta Degrees of Freedom”，表示自由度的个数。
要注意方差和样本方差的无偏估计，方差公式中分母上是 n ；样本方差无偏估计公式中分母上是 n‐
1 （ n 为样本个数）
import numpy as np
x = np.array([[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35]])
y = np.var(x)
print(y) # 52.0
y = np.mean((x -np.mean(x)) ** 2)
print(y) # 52.0
y = np.var(x, ddof=1) #ddof=1时选取的是修正后的自由度n-1，=0选区的是n
print(y) # 54.166666666666664
y = np.sum((x - np.mean(x)) ** 2) / (x.size-1)
print(y) # 54.166666666666664
y = np.var(x, axis=0)
print(y) # [50. 50. 50. 50. 50.]
y = np.var(x, axis=1)
print(y) # [2. 2. 2. 2. 2.]
52.0
52.0
54.166666666666664
54.166666666666664
[50. 50. 50. 50. 50.]
[2. 2. 2. 2. 2.]
计算标准差
- numpy.std(a[, axis=None, dtype=None, out=None, ddof=0, keepdims=np._NoValue]) Compute
the standard deviation along the specified axis.
标准差是一组数据平均值分散程度的一种度量，是方差的算术平方根
import numpy as np
x = np.array([[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
[26, 27, 28, 29, 30],
[31, 32, 33, 34, 35]])
#计算标准差的两种方法
y = np.std(x)
print(y) # 7.211102550927978
y = np.sqrt(np.var(x))
print(y) # 7.211102550927978
y = np.std(x, axis=0)
print(y)
# [7.07106781 7.07106781 7.07106781 7.07106781 7.07106781]
y = np.std(x, axis=1)
print(y)
# [1.41421356 1.41421356 1.41421356 1.41421356 1.41421356]
7.211102550927978
7.211102550927978
[7.07106781 7.07106781 7.07106781 7.07106781 7.07106781]
[1.41421356 1.41421356 1.41421356 1.41421356 1.41421356]
相关
- numpy.cov(m, y=None, rowvar=True, bias=False, ddof=None, fweights=None,aweights=None)
Estimate a covariance matrix, given data and weights.
import numpy as np
x = [1, 2, 3, 4, 6]
y = [0, 2, 5, 6, 7]
print(np.cov(x)) # 3.7 #样本方差
print(np.cov(y)) # 8.5 #样本方差
print(np.cov(x, y))
# [[3.7 5.25]# [5.25 8.5 ]]
print(np.var(x)) # 2.96 #方差，默认为总体方差
print(np.var(x, ddof=1)) # 3.7 #ddof=1为样本方差
print(np.var(y)) # 6.8 #方差
print(np.var(y, ddof=1)) # 8.5 #样本方差
z = np.mean((x - np.mean(x)) * (y - np.mean(y))) #协方差（此处为矩阵乘法）
print(z) # 4.2
z = np.sum((x - np.mean(x)) * (y - np.mean(y))) / (len(x) - 1) #样本协方差，其中用于计算协方差的方差用的是样本的无偏估计量 样本方差 自由度为n-1
print(z) # 5.25
z = np.dot(x - np.mean(x), y - np.mean(y)) / (len(x) - 1) #样本协方差
print(z) # 5.25
3.7
8.5
[[3.7 5.25]
[5.25 8.5 ]]
2.96
3.7
6.8
8.5
4.2
5.25
5.25
计算相关系数
- numpy.corrcoef(x, y=None, rowvar=True, bias=np._NoValue, ddof=np._NoValue) Return
Pearson product-moment correlation coefficients.
理解了 np.cov() 函数之后，很容易理解 np.correlate() ，二者参数几乎一模一样。
np.cov() 描述的是两个向量协同变化的程度，它的取值可能非常大，也可能非常小，这就导致没法
直观地衡量二者协同变化的程度。相关系数实际上是正则化的协方差， n 个变量的相关系数形成一
个 n 维方阵。
import numpy as np
np.random.seed(20200623)
x, y = np.random.randint(0, 20, size=(2, 4))
print(x) # [10 2 1 1]
print(y) # [16 18 11 10]
z = np.corrcoef(x, y)
print(z)
# [[1. 0.48510096]
# [0.48510096 1. ]]
a = np.dot(x - np.mean(x), y - np.mean(y))
b = np.sqrt(np.dot(x - np.mean(x), x - np.mean(x)))
c = np.sqrt(np.dot(y - np.mean(y), y - np.mean(y)))
print(a / (b * c)) # 0.4851009629263671
[10 2 1 1]
[16 18 11 10]
[[1. 0.48510096]
[0.48510096 1. ]]
0.4851009629263671
直方图
- numpy.digitize(x, bins, right=False) Return the indices of the bins to which each value
in input array belongs.
x：numpy数组
bins：一维单调数组，必须是升序或者降序
right：间隔是否包含最右
返回值：x在bins中的位置。
import numpy as np
x = np.array([0.2, 6.4, 3.0, 1.6])
bins = np.array([0.0, 1.0, 2.5, 4.0, 10.0])
inds = np.digitize(x, bins)
print(inds) # [1 4 3 2]
for n in range(x.size):
print(bins[inds[n] - 1], "<=", x[n], "<", bins[inds[n]])
[1 4 3 2]
0.0 <= 0.2 < 1.0
4.0 <= 6.4 < 10.0
2.5 <= 3.0 < 4.0
1.0 <= 1.6 < 2.5
import numpy as np
x = np.array([1.2, 10.0, 12.4, 15.5, 20.])
bins = np.array([0, 5, 10, 15, 20])
inds = np.digitize(x, bins, right=True)
print(inds) # [1 2 3 4 4]
inds = np.digitize(x, bins, right=False)
print(inds) # [1 3 3 4 5]
[1 2 3 4 4]
[1 3 3 4 5]