pytorch学习笔记（2）--Tensor

系列文章

pytorch学习笔记（1）–QUICKSTART
pytorch学习笔记（2）–Tensor
pytorch学习笔记（3）–数据集与数据导入
pytorch学习笔记（4）–创建模型(Build Model)
pytorch学习笔记（5）–Autograd

---

Tensor（张量）

import torch
import numpy as np

1. 初始化张量

• 直接用列表数据建立：
• • torch.tensor(data)
• 用numpy数组建立：
• • torch.from_numpy(np_array)
• 用另一个Tensor建立：
• • torch.ones_like(x_data)
• • torch.rand_like(x_data, dtype=torch.float)
• 使用随机变量或常量建立：
• • torch.rand(shape)
• • torch.ones(shape)
• • torch.zeros(shape)
• 在GPU上创建张量：
• • tensor.to(‘cuda’)

#直接用列表数据建立
data = [[1,2],[3,4]]
x_data = torch.tensor(data)
# 用numpy数组建立
np_array = np.array(data)
x_np = torch.from_numpy(np_array)
print(f"Numpy np_array value: \n {np_array} \n")
print(f"Tensor x_np value: \n {x_np} \n")

np.multiply(np.array, 2, out=np_array)

print(f"Numpy np_array after * 2 operation: \n {np_array} \n")
print(f"Tensor x_np value after modifying numpy array: \n {x_np} \n")

Numpy np_array value: 
 [[1 2]
 [3 4]] 

Tensor x_np value: 
 tensor([[1, 2],
        [3, 4]]) 
Numpy np_array after * 2 operation: 
 [[2 4]
 [6 8]] 

Tensor x_np value after modifying numpy array: 
 tensor([[2, 4],
        [6, 8]]) 

用numpy数组建立是浅拷贝：如果改变np_array，x_np随之改变

#用另一个Tensor建立
x_ones = torch.ones_like(x_data) # retains the properties of x_data
print(f"Ones Tensor: \n {x_ones} \n")

x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data
print(f"Random Tensor: \n {x_rand} \n")

Ones Tensor: 
 tensor([[1, 1],
        [1, 1]]) 

Random Tensor: 
 tensor([[0.2476, 0.2297],
        [0.6623, 0.8990]]) 

#使用随机变量或常量建立
shape = (2,3)
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)

print(f"Random Tensor:\n {rand_tensor} \n")
print(f"Ones Tensor:\n {ones_tensor} \n")
print(f"Zeros Tensor:\n{zeros_tensor} \n")

输出：

Random Tensor:
 tensor([[0.6404, 0.4636, 0.4383],
        [0.5824, 0.2888, 0.8682]]) 

Ones Tensor:
 tensor([[1., 1., 1.],
        [1., 1., 1.]]) 

Zeros Tensor:
tensor([[0., 0., 0.],
        [0., 0., 0.]]) 

2. 张量的属性

• 形状：tensor.shape
• 数据类型：tensor.dtype
• 存储设备: tensor.device

tensor = torch.rand(3,4)

print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device tensor is stored on: {tensor.device}")

#在GPU上创建张量，默认创建的张量在CPU上
# We move our tensor to the GPU if available
if torch.cuda.is_available():
  tensor = tensor.to('cuda')

Shape of tensor: torch.Size([3, 4])
Datatype of tensor: torch.float32
Device tensor is stored on: cpu

3.张量的操作

张量有100多种操作，包括算术运算、矩阵操作（如转置、切片、索引等），具体见官网。

1. 类似numpy的索引和切片

tensor = torch.ones(4, 4)
print('First row: ',tensor[0])
print("First column: ",tensor [:, 0])
print('Last column: ',tensor[: ,-1])

tensor[:,1] = 0
print(tensor)

First row:  tensor([1., 1., 1., 1.])
First column:  tensor([1., 1., 1., 1.])
Last column:  tensor([1., 1., 1., 1.])
tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]])

2. 拼接

• torch.cat([tensor,tensor1,tensor2],dim=1)
• torch.stack([tensor,tensor1,tensor2],dim=1)
• • 会创建一个新的dimension

t1 = torch.cat([torch.rand(4,4), torch.ones(4,4), torch.zeros(4,4)], dim=1)
print(t1)
t2 = torch.stack([torch.rand(4,4), torch.ones(4,4), torch.zeros(4,4)], dim=0)
print(t2)

tensor([[0.9215, 0.2204, 0.4412, 0.6765, 1.0000, 1.0000, 1.0000, 1.0000, 0.0000,
         0.0000, 0.0000, 0.0000],
        [0.0694, 0.2806, 0.1362, 0.0758, 1.0000, 1.0000, 1.0000, 1.0000, 0.0000,
         0.0000, 0.0000, 0.0000],
        [0.8816, 0.0996, 0.2149, 0.7842, 1.0000, 1.0000, 1.0000, 1.0000, 0.0000,
         0.0000, 0.0000, 0.0000],
        [0.7649, 0.9358, 0.9627, 0.7705, 1.0000, 1.0000, 1.0000, 1.0000, 0.0000,
         0.0000, 0.0000, 0.0000]])
tensor([[[0.7450, 0.0089, 0.4026, 0.4642],
         [0.2541, 0.8307, 0.8477, 0.0690],
         [0.6198, 0.7501, 0.8160, 0.3058],
         [0.7141, 0.6078, 0.6394, 0.6942]],

        [[1.0000, 1.0000, 1.0000, 1.0000],
         [1.0000, 1.0000, 1.0000, 1.0000],
         [1.0000, 1.0000, 1.0000, 1.0000],
         [1.0000, 1.0000, 1.0000, 1.0000]],

        [[0.0000, 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000, 0.0000],
         [0.0000, 0.0000, 0.0000, 0.0000]]])

3.算数操作

• 矩阵乘法：
• • tensor @ tensor1
• • tensor.matmul(tensor1)
• 元素点乘：
• • tensor * tensor1
• • tensor.mul(tensor1)

# This computes the matrix multiplication between two tensors. y1, y2, y3 will have the same value
tensor = torch.ones(4,4)
tensor[2, :] = 0
y1 = tensor @ tensor.T
y2 = tensor.matmul(tensor.T)

y3 = torch.rand_like(tensor)
torch.matmul(tensor, tensor.T, out=y3)
print(y3)

# This computes the element-wise product. z1, z2, z3 will have the same value
z1 = tensor * tensor
z2 = tensor.mul(tensor)

z3 = torch.rand_like(tensor)
torch.mul(tensor, tensor, out=z3)
print(z3)

tensor([[4., 4., 0., 4.],
        [4., 4., 0., 4.],
        [0., 0., 0., 0.],
        [4., 4., 0., 4.]])
tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.],
        [0., 0., 0., 0.],
        [1., 1., 1., 1.]])

4.单元素张量

• 聚合张量的所有值为一个值：tensor.sum()
• 把张量转变为python常数值：agg.item()

agg = tensor.sum()
print(agg)
agg_item = agg.item()  
print(agg_item, type(agg_item))

tensor(12.)
12.0 

5. In-place 操作

将结果存储到操作数中的操作称为就地操作。 它们由 _ 后缀表示。 例如：x.copy_(y)、x.t_()，会改变x。

就地操作可以节省一些内存，但在计算导数时可能会出现问题，因为它们会立即丢失历史记录。 因此，不鼓励使用它们。

tensor = torch.ones(4,4)
tensor [:,1] = 0
print(tensor, "\n")
tensor.add_(5)
print(tensor)
ones = torch.rand(4,4)
tensor.copy_(ones)
print(tensor)
tensor.t_()
print(tensor)

tensor([[1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.],
        [1., 0., 1., 1.]]) 

tensor([[6., 5., 6., 6.],
        [6., 5., 6., 6.],
        [6., 5., 6., 6.],
        [6., 5., 6., 6.]])
        
tensor([[0.3871, 0.4316, 0.9952, 0.7004],
        [0.9453, 0.5613, 0.7772, 0.6667],
        [0.4810, 0.6807, 0.5161, 0.9277],
        [0.9168, 0.6124, 0.4841, 0.5865]])
        
tensor([[0.3871, 0.9453, 0.4810, 0.9168],
        [0.4316, 0.5613, 0.6807, 0.6124],
        [0.9952, 0.7772, 0.5161, 0.4841],
        [0.7004, 0.6667, 0.9277, 0.5865]])

6.Bridge with Numpy

CPU 和 NumPy 数组上的张量可以共享其底层内存位置，改变其中一个就会改变另一个。

• Tensor to Numpy array：n = t.numpy()
• NumPy array to tensor：t = torch.from_numpy(n)

#Tensor to Numpy array
t = torch.ones(3)
print(f"t: {t}")
n = t.numpy()
print(f"n: {n}\n")

t.add_(1)
print(f"t: {t}")
print(f"n: {n}")

t: tensor([1., 1., 1.])
n: [1. 1. 1.]

t: tensor([2., 2., 2.])
n: [2. 2. 2.]

#NumPy array to tensor
n = np.ones(5)
t = torch.from_numpy(n)
print(n)
print(t)
#change the NumPy
np.add(n, 1, out=n)
print(f"t: {t}")
print(f"n: {n}")

[1. 1. 1. 1. 1.]
tensor([1., 1., 1., 1., 1.], dtype=torch.float64)

t: tensor([2., 2., 2., 2., 2.], dtype=torch.float64)
n: [2. 2. 2. 2. 2.]