[二] Numpy数据存取与函数

Numpy数据存取与函数

• 一维和二维数据的存取

  • CSV文件, np.loadtxt(), np.savetxt()

• 多维数据的存取

  • a.tofile(), np.fromfile()

  • np.save(), np.load()

• numpy的随机函数(np.random)

  • rand(), randn(), randint(), seed()

  • shuffle(), permutation(), choice()

  • uniform(), normal(), poisson()

• numpy的统计函数

  • sum(), mean(), average(), std(), var()

  • min(), max(), argmin(), argmax(), unravel_index(), ptp(), median()

• numpy的梯度函数

  • np.gradient()

CSV文件：存取一维和二维数据

• CSV文件是一种常见的文件格式，用来存储批量数据。
• CSV文件只能有效的存储一维和二维数组。
• np.savetxt(), np.loadtxt()只能有效的存取一维和二维数组。

Numpy将数据存入CSV文件，函数如下：

import numpy as np

a = np.arange(100).reshape(5, 20)
np.savetxt('a.csv', a, fmt="%d", delimiter=',')

Numpy读入CSV文件，函数如下：

b = np.loadtxt('a.csv', delimiter=',')
b

array([[ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12.,
        13., 14., 15., 16., 17., 18., 19.],
       [20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31., 32.,
        33., 34., 35., 36., 37., 38., 39.],
       [40., 41., 42., 43., 44., 45., 46., 47., 48., 49., 50., 51., 52.,
        53., 54., 55., 56., 57., 58., 59.],
       [60., 61., 62., 63., 64., 65., 66., 67., 68., 69., 70., 71., 72.,
        73., 74., 75., 76., 77., 78., 79.],
       [80., 81., 82., 83., 84., 85., 86., 87., 88., 89., 90., 91., 92.,
        93., 94., 95., 96., 97., 98., 99.]])

b = np.loadtxt('a.csv', dtype=np.int32, delimiter=',')
b

array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15,
        16, 17, 18, 19],
       [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
        36, 37, 38, 39],
       [40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,
        56, 57, 58, 59],
       [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75,
        76, 77, 78, 79],
       [80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95,
        96, 97, 98, 99]])

多维数据的存取

• 数据存入:tofile，数据加载loadfile
• 便捷的文件存取方法：np.save，np.load

多维数据的存入：tofile函数

a = np.arange(100).reshape(5, 10, 2)
a.tofile("b.dat", sep=",", format='%d')

结果将会平铺为一维数据，如下

多维数据的加载：np.fromfile函数

c = np.fromfile("b.dat", dtype=np.int32, sep=",")
c

array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
       51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67,
       68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
       85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99])

# 要想还原数据，还要知道原先数组的尺寸
c = c.reshape(5, 10, 2)
c

array([[[ 0,  1],
        [ 2,  3],
        [ 4,  5],
        [ 6,  7],
        [ 8,  9],
        [10, 11],
        [12, 13],
        [14, 15],
        [16, 17],
        [18, 19]],

       [[20, 21],
        [22, 23],
        [24, 25],
        [26, 27],
        [28, 29],
        [30, 31],
        [32, 33],
        [34, 35],
        [36, 37],
        [38, 39]],

       [[40, 41],
        [42, 43],
        [44, 45],
        [46, 47],
        [48, 49],
        [50, 51],
        [52, 53],
        [54, 55],
        [56, 57],
        [58, 59]],

       [[60, 61],
        [62, 63],
        [64, 65],
        [66, 67],
        [68, 69],
        [70, 71],
        [72, 73],
        [74, 75],
        [76, 77],
        [78, 79]],

       [[80, 81],
        [82, 83],
        [84, 85],
        [86, 87],
        [88, 89],
        [90, 91],
        [92, 93],
        [94, 95],
        [96, 97],
        [98, 99]]])

便捷的文件存取：np.save，np.load

np.save("a.npy", a)

b = np.load("a.npy")
b

array([[[ 0,  1],
        [ 2,  3],
        [ 4,  5],
        [ 6,  7],
        [ 8,  9],
        [10, 11],
        [12, 13],
        [14, 15],
        [16, 17],
        [18, 19]],

       [[20, 21],
        [22, 23],
        [24, 25],
        [26, 27],
        [28, 29],
        [30, 31],
        [32, 33],
        [34, 35],
        [36, 37],
        [38, 39]],

       [[40, 41],
        [42, 43],
        [44, 45],
        [46, 47],
        [48, 49],
        [50, 51],
        [52, 53],
        [54, 55],
        [56, 57],
        [58, 59]],

       [[60, 61],
        [62, 63],
        [64, 65],
        [66, 67],
        [68, 69],
        [70, 71],
        [72, 73],
        [74, 75],
        [76, 77],
        [78, 79]],

       [[80, 81],
        [82, 83],
        [84, 85],
        [86, 87],
        [88, 89],
        [90, 91],
        [92, 93],
        [94, 95],
        [96, 97],
        [98, 99]]])

两种存取方法比较

Numpy的随机数函数

np.random的随机数函数（一）

a = np.random.rand(3, 4, 5)
a

array([[[0.15257507, 0.02480371, 0.62325075, 0.30554174, 0.71039419],
        [0.16793472, 0.11053303, 0.18702078, 0.59347635, 0.18655296],
        [0.34498871, 0.82380281, 0.62474943, 0.09706273, 0.84723209],
        [0.32304167, 0.97391336, 0.34865744, 0.69825689, 0.33477139]],

       [[0.97218235, 0.547869  , 0.01222512, 0.47606408, 0.55055613],
        [0.18585335, 0.06653516, 0.32867498, 0.59954051, 0.36819215],
        [0.23493346, 0.75768867, 0.48641578, 0.98568222, 0.83763441],
        [0.44208099, 0.96234619, 0.93853069, 0.23875454, 0.64855734]],

       [[0.21927259, 0.54326216, 0.5728762 , 0.30291199, 0.30875352],
        [0.70463617, 0.16616126, 0.9389809 , 0.99866047, 0.84941496],
        [0.66782308, 0.80883132, 0.83030115, 0.21593487, 0.11052866],
        [0.50227729, 0.7373498 , 0.08622482, 0.1546465 , 0.7673344 ]]])

sn = np.random.randn(3, 4, 5)
sn

array([[[-0.13899166,  0.81753401, -1.51661175,  1.12756715,
          0.47075786],
        [ 1.58396047, -0.21436527,  1.60210739,  0.50196767,
         -0.70752227],
        [-1.07846307,  0.24308953,  1.31379971,  1.37007777,
          1.0744217 ],
        [ 0.91036789,  1.22711205, -0.36849642,  0.32428547,
          0.14387649]],

       [[-1.02127348, -0.41833756, -0.95767588,  0.51808076,
          0.96952932],
        [-0.86687153,  1.51265849,  0.10725677,  0.54024379,
         -1.16591356],
        [ 1.45679455, -1.12580397,  0.70885802, -1.13800884,
          0.46957394],
        [ 2.03218182, -0.53277405, -2.19688155, -0.20649949,
          2.00457427]],

       [[-0.04384466, -0.67094456, -0.50440545, -1.38355559,
         -1.75376971],
        [-0.52912614,  0.44385732, -1.89686488,  0.16396965,
          0.96562041],
        [ 1.37031847, -0.20085887,  0.56562333, -0.27319559,
         -1.34236485],
        [-0.28952533, -0.5397941 , -0.61748276,  1.35898526,
         -1.09242035]]])

b = np.random.randint(100, 200, (3, 4))
b

array([[178, 153, 195, 183],
       [197, 160, 146, 157],
       [161, 134, 120, 156]])

np.random.seed(10)
np.random.randint(100, 200, (3, 4))

array([[109, 115, 164, 128],
       [189, 193, 129, 108],
       [173, 100, 140, 136]])

# 使用同一个种子，可以发现生成结果是相同的
np.random.seed(10)
np.random.randint(100, 200, (3, 4))

array([[109, 115, 164, 128],
       [189, 193, 129, 108],
       [173, 100, 140, 136]])

np.random的随机数函数（二）

a = np.random.randint(100, 200, (3, 4))
a

array([[116, 111, 154, 188],
       [162, 133, 172, 178],
       [149, 151, 154, 177]])

# 原来的数组a改变了
np.random.shuffle(a)
a

array([[116, 111, 154, 188],
       [149, 151, 154, 177],
       [162, 133, 172, 178]])

a = np.random.randint(100, 200, (3, 4))
a

array([[125, 113, 192, 186],
       [130, 130, 189, 112],
       [165, 131, 157, 136]])

np.random.permutation(a)

array([[125, 113, 192, 186],
       [130, 130, 189, 112],
       [165, 131, 157, 136]])

# 使用permutation乱序数组时，不会改变原来的数组a
a

array([[125, 113, 192, 186],
       [130, 130, 189, 112],
       [165, 131, 157, 136]])

np.random的随机数函数（三）

u = np.random.uniform(0, 10, (3,4))
u

array([[6.01038953, 8.05223197, 5.21647152, 9.08648881],
       [3.19236089, 0.90459349, 3.00700057, 1.13984362],
       [8.28681326, 0.46896319, 6.26287148, 5.47586156]])

n = np.random.normal(0, 1, (3,3))
n

array([[-0.49561818,  0.52563742, -0.38964647],
       [ 0.9372965 , -0.8169742 ,  1.01961544],
       [-0.34143633,  0.7512814 , -0.36225179]])

p = np.random.poisson(0.5, (3, 3))
p

array([[0, 1, 0],
       [2, 1, 0],
       [1, 0, 0]])

### Numpy的统计函数

numpy统计函数（一）

a = np.arange(15).reshape(3, 5)
a

array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14]])

np.sum(a)

105

np.mean(a, axis=1)

array([ 2.,  7., 12.])

np.mean(a, axis=0)

array([5., 6., 7., 8., 9.])

np.average(a, axis=0, weights=[10, 5, 1])

array([2.1875, 3.1875, 4.1875, 5.1875, 6.1875])

np.std(a)

4.320493798938574

np.var(a)

18.666666666666668

numpy统计函数（二）

b = np.arange(15, 0, -1).reshape(3, 5)
b

array([[15, 14, 13, 12, 11],
       [10,  9,  8,  7,  6],
       [ 5,  4,  3,  2,  1]])

np.max(b)

15

np.min(b)

1

np.argmax(b)

0

# 生成多维下标
np.unravel_index(np.argmax(b), b.shape)

(0, 0)

np.ptp(b)

14

np.median(b)

8.0

numpy的梯度函数

a = np.random.randint(0, 20, (5))
a

array([18,  3, 16, 18, 16])

np.gradient(a)

array([-15. ,  -1. ,   7.5,   0. ,  -2. ])

b = np.random.randint(0, 50,(3, 5))
b

array([[14, 15, 17, 25, 46],
       [48, 42, 17, 32, 17],
       [41, 16, 41, 26, 12]])

np.gradient(b)

[array([[ 34. ,  27. ,   0. ,   7. , -29. ],
        [ 13.5,   0.5,  12. ,   0.5, -17. ],
        [ -7. , -26. ,  24. ,  -6. ,  -5. ]]),
 array([[  1. ,   1.5,   5. ,  14.5,  21. ],
        [ -6. , -15.5,  -5. ,   0. , -15. ],
        [-25. ,   0. ,   5. , -14.5, -14. ]])]