pytorch基础操作学习笔记（autograd，Tensor）
简述
简单讲讲关于torch.autograd内容（因为我也有点菜）
文章目录
- 简述
- 简单讲讲
- Tensor
- 介绍Tensor
- 创建Tensor
- 获取Tensor数据规模
- 将tensor转成其他数据类型
- 改变Tensor形状
- Tensor的切片操作
- Tensor的比较
- Tensor的数据筛选
- Tensor其他常用函数
- index_select(input, dim, index)函数
- masked_select(input, mask)
- nonzero(input)操作
- gather 根据index，在dim上选取数据，输出size与index一直
- clamp截断
- 后记
简单讲讲
- 创建数据
>>> import torch
>>> from torch.autograd import Variable
>>> x = Variable(torch.ones(2, 2), requires_grad=True)
>>> x
tensor([[1., 1.],
[1., 1.]], requires_grad=True)
- Variable中有三个东西，就是下面的三个：
data，grad，grad_fn
>>> x.data
tensor([[1., 1.],
[1., 1.]])
>>> x.grad
>>> x.grad_fn
- 这时候下面两个其实是NoneType的
>>> x.grad_fn.data
Traceback (most recent call last):
File "" , line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'data'
>>> x.grad_fn.data
Traceback (most recent call last):
File "" , line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'data'
- 尝试做反向传播
>>> x.backward()
Traceback (most recent call last):
File "" , line 1, in <module>
File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\autograd\__init__.py", line 84, in backward
grad_tensors = _make_grads(tensors, grad_tensors)
File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\autograd\__init__.py", line 28, in _make_grads
raise RuntimeError("grad can be implicitly created only for scalar outputs")
RuntimeError: grad can be implicitly created only for scalar outputs
意思很简单，就是说，这里如果是做反向传播，东西必须是标量的输出。
- 对x求和
>>> y = x.sum()
>>> y
tensor(4., grad_fn=<SumBackward0>)
- 关于y做反向传播
有反向传播的函数。
>>> y.backward()
>>> y
tensor(4., grad_fn=<SumBackward0>)
>>> y.grad
>>> y.grad_fn
<SumBackward0 object at 0x0000026BADBFF278>
>>> a = y.backward()
>>> a
>>> a == None
True
- 这时候再去检查x
>>> x
tensor([[1., 1.],
[1., 1.]], requires_grad=True)
>>> x.grad
tensor([[2., 2.],
[2., 2.]])
>>> x.grad_fn
x的梯度不再是None
这里的话，其实x的梯度本来应该全是1的，但是做了两次之后，有了累积。
- 再做一次y的反向传播，验证猜想
>>> y.backward()
>>> x.grad
tensor([[3., 3.],
[3., 3.]])
验证正确。所以这告诉我们每次计算前都需要把梯度归零。这里不归零，是因为有些算法需要用到这个操作。
- 梯度清0
>>> x.grad.data.zero_()
tensor([[0., 0.],
[0., 0.]])
>>> x
tensor([[1., 1.],
[1., 1.]], requires_grad=True)
>>> x.grad
tensor([[0., 0.],
[0., 0.]])
- 再算一次y的反向传播
>>> y.backward()
>>> x.grad
tensor([[1., 1.],
[1., 1.]])
- 关于之前的求梯度结果全是1的解释
y = ∑ X i y = \sum{X_i} y=∑Xi
y y y关于 X i X_i Xi来求导，得到的结果都是1。所以每一个数值都是1。
- 验证无关a的数值
由于我们看到下面的数值当中，a的数值都变成了2。但是 求导的结果任然是这个。
>>> a = Variable(torch.ones(2,2)*2, requires_grad=True)
>>> a
tensor([[2., 2.],
[2., 2.]], requires_grad=True)
>>> y = a.sum()
>>> a
tensor([[2., 2.],
[2., 2.]], requires_grad=True)
>>> y
tensor(8., grad_fn=<SumBackward0>)
>>> y.backward()
>>> a.grad
tensor([[1., 1.],
[1., 1.]])
- 验证和算法有关
y = ∑ cos X i y = \sum{\cos{X_i}} y=∑cosXi
关于X_i求导有
d y d X i = sin X i \frac{dy}{dX_i} =\sin{X_i} dXidy=sinXi
>>> a = Variable(torch.ones(2,2)*2, requires_grad=True)
>>> a
tensor([[2., 2.],
[2., 2.]], requires_grad=True)
>>> a.grad
>>> b = a.cos()
>>> b
tensor([[-0.4161, -0.4161],
[-0.4161, -0.4161]], grad_fn=<CosBackward>)
>>> c = b.sum()
>>> b.grad
>>> c.backward()
>>> b.grad
>>> a.grad
tensor([[-0.9093, -0.9093],
[-0.9093, -0.9093]])
通过数值来验算一下：
>>> import numpy as np
>>> np.sin(2)
0.9092974268256817
>>>
发现，保留4位有效数值的结果是一样的。
Tensor
介绍Tensor
Tensor，又名张量。从工程角度不妨认为是一个数组。
和numpy中的ndarray类似，但是tensor支持GPU加速–部分摘取于《深度学习框架Pytorch入门与实践》
创建Tensor
只有使用torch.Tensor(*size)的方式创建的数组，才是创建数组的时候并不会分配内存空间，只有当使用到的时候，才会分配。
- 直接Tensor创建
>>> a = torch.Tensor(1)
>>> a
tensor([5.6052e-45])
发现数值其实是接近0的（准确说，在计算机里这个跟0也没太多区别了，一般来说）
- 直接Tensor创建（二）
这个是将另外的类似于数组的结构转变成Tensor
>>> a = torch.Tensor((2, 3))
>>> a
tensor([2., 3.])
类似的操作
>>> a = torch.Tensor((2, 3, 4, 5))
>>> a
tensor([2., 3., 4., 5.])
>>> a = torch.Tensor([2, 3, 4, 5])
>>> a
tensor([2., 3., 4., 5.])
不得不说，这其实跟numpy的array构造法类似的
>>> a = torch.Tensor(np.array([2, 3, 4, 5]))
>>> a
tensor([2., 3., 4., 5.])
- 直接Tensor创建（三）
用size的维度来做输入
>>> a = torch.Tensor(1,2)
>>> a
tensor([[0.0000, 0.0000]])
>>> a = torch.Tensor(3,2)
>>> a
tensor([[0.0000, 0.0000],
[0.0000, 0.0000],
[0.0000, 0.0000]])
- 构造全1的张量
>>> a = torch.ones(3,2)
>>> a
tensor([[1., 1.],
[1., 1.],
[1., 1.]])
>>> a = torch.ones((3,2))
>>> a
tensor([[1., 1.],
[1., 1.],
[1., 1.]])
>>> a = torch.ones([3,2])
>>> a
tensor([[1., 1.],
[1., 1.],
[1., 1.]])
>>> a = torch.ones(np.array((3,2)))
Traceback (most recent call last):
File "" , line 1, in <module>
TypeError: ones(): argument 'size' (position 1) must be tuple of ints, not numpy.ndarray
>>>
除了numpy不能使用的，其他的上面提到的，都可以作为size
- 构造全0的张量
跟全1的一模一样。
>>> a = torch.zeros(3,2)
>>> a
tensor([[0., 0.],
[0., 0.],
[0., 0.]])
>>> a = torch.zeros((3,2))
>>> a
tensor([[0., 0.],
[0., 0.],
[0., 0.]])
>>> a = torch.zeros([3,2])
>>> a
tensor([[0., 0.],
[0., 0.],
[0., 0.]])
>>> a = torch.zeros(np.array([3,2]))
Traceback (most recent call last):
File "" , line 1, in <module>
TypeError: zeros(): argument 'size' (position 1) must be tuple of ints, not numpy.ndarray
>>> a = torch.zeros(1)
>>> a
tensor([0.])
>>> a = torch.zeros(2)
>>> a
tensor([0., 0.])
>>>
- 对角元全是1，其他都是0的矩阵
只能是方阵
>>> a = torch.eye(2)
>>> a
tensor([[1., 0.],
[0., 1.]])
>>> a = torch.eye(1)
>>> a
tensor([[1.]])
>>> a = torch.eye(3)
>>> a
tensor([[1., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]])
>>> a = torch.eye(4)
>>> a
tensor([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]])
>>> a = torch.eye((1,2))
Traceback (most recent call last):
File "" , line 1, in <module>
TypeError: eye(): argument 'n' (position 1) must be int, not tuple
- arange(s,e, step)
- step：步长
>>> a = torch.arange(0,1,10)
>>> 1
>>> a = torch.arange(0,10,1)
>>> a
tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
- linspace(s, e, steps)
- steps：步数
- steps：大于等于2
>>> a = torch.linspace(0,1,10)
>>> a
tensor([0.0000, 0.1111, 0.2222, 0.3333, 0.4444, 0.5556, 0.6667, 0.7778, 0.8889,
1.0000])
>>> a = torch.linspace(0,10,1)
Traceback (most recent call last):
File "" , line 1, in <module>
RuntimeError: invalid argument 3: invalid number of points at d:\build\pytorch\pytorch-0.4.1\aten\src\th\generic\thtensormath.cpp:4408
>>> a = torch.linspace(0,10,2)
>>> a
tensor([ 0., 10.])
- rand(*size) 均匀分布 (类似的有randn标准正态分布)
>>> a = torch.rand(1)
>>> a
tensor([0.5391])
>>> a = torch.rand(1)
>>> a
tensor([0.7884])
>>> a = torch.rand(1,2)
>>> a
tensor([[0.6491, 0.8738]])
>>> a = torch.rand(2,1)
>>> a
tensor([[0.1860],
[0.4988]])
>>> a = torch.rand(2)
>>> a
tensor([0.9654, 0.6263])
>>> a = torch.randn(1)
>>> a
tensor([-1.2497])
>>> a = torch.randn(1)
>>> a
tensor([-2.0466])
>>> a = torch.randn(1)
>>> a
tensor([0.4620])
>>> a = torch.randn(2)
>>> a
tensor([0.3337, 0.5260])
>>> a = torch.randn(1,2)
>>> a
tensor([[-0.0197, 0.1966]])
>>> a = torch.randn(2,1)
>>> a
tensor([[-0.3927],
[-0.4279]])
- normal(mean, std)正态分布 / (uniform(s, e))均匀分布
下面这个写的normal()还行
https://blog.csdn.net/qq_34690929/article/details/80029974
- 固定mean，但不固定std
>>> torch.normal(mean=0,std=torch.rand(5),out=a)
tensor([-0.1707, 0.6314, 0.2634, -0.2100, -1.3774])
>>> torch.normal(mean=0,std=torch.rand(5))
tensor([-0.1613, -0.0957, 0.2675, 0.3178, -0.6548])
- 固定std，但不固定mean
>>> torch.normal(mean=torch.rand(5), std=1)
tensor([-0.7600, 1.1998, 0.9734, 3.5803, 0.8375])
>>> torch.normal(mean=torch.rand(5), std=1,out=a)
tensor([ 0.5822, 2.9761, -0.3904, 0.7321, 1.2330])
- 两个都不固定
>>> torch.normal(mean=torch.rand(5), std=torch.linspace(1,2,5),out=a)
tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729])
>>> torch.normal(mean=torch.rand(5), std=torch.linspace(1,2,5))
tensor([ 0.7096, 1.0136, -2.5004, -0.2733, 3.6389])
- 如果想要两个都固定，只能重复来做
- 下面的构造函数只有三个。
>>> torch.normal(mean=0,std=1,out=a)
Traceback (most recent call last):
File "" , line 1, in <module>
TypeError: normal() received an invalid combination of arguments - got (mean=int, std=int, out=Tensor, ), but expected one of:
* (Tensor mean, Tensor std, torch.Generator generator, Tensor out)
* (Tensor mean, float std, torch.Generator generator, Tensor out)
* (float mean, Tensor std, torch.Generator generator, Tensor out)
比如：
下面的就是标准正态分布。
>>> torch.normal(mean=torch.ones(5), std=torch.ones(5)+1)
tensor([ 0.6148, -3.3352, 3.0508, 0.4238, 1.4016])
uniform就不一样了。必须另外操作
我怀疑是改了版本了，我看有些书都是没这么写的。。感觉蛮奇怪的。
但是在pytorch的官方文档中有这样的解释：
Fills
selftensor with numbers sampled from the continuous uniform distribution:
用采样自下面的连续均匀分布的数字来填充self tensor 。
所以，虽然文档中没有写具体的实例代码，也能知道如何操作（如下：）
>>> torch.Tensor(10).uniform_(0,1)
tensor([0.3394, 0.8679, 0.3344, 0.6168, 0.6465, 0.7618, 0.3902, 0.5174, 0.0205, 0.6731])
- randperm(m)随机排列
生成0到m的随机排列
>>> torch.randperm(10)
tensor([1, 7, 4, 8, 9, 3, 0, 6, 2, 5])
获取Tensor数据规模
- 其实只有下面的两种方法
>>> a
tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729])
>>> a.size()
torch.Size([5])
>>> a.shape
torch.Size([5])
>>> a
tensor([[-0.2679],
[ 0.1669],
[ 3.7573],
[-1.9038],
[-0.2729]])
>>> a.size()
torch.Size([5, 1])
>>> a.shape
torch.Size([5, 1])
将tensor转成其他数据类型
- 转成list
很吃惊的是，居然精度有这么高！！
>>> a
tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729])
>>> a.tolist()
[-0.2678675651550293, 0.16691356897354126, 3.757300853729248, -1.9038498401641846, -0.27292633056640625]
- 转成numpy
>>> a
tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729])
>>> a.numpy()
array([-0.26786757, 0.16691357, 3.7573009 , -1.9038498 , -0.27292633],
dtype=float32)
改变Tensor形状
先创建一个数据
>>> a
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426],
[ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130, 1.0243],
[ 1.2052, -1.0993, 2.5415, 0.9572, -0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258],
[ 0.7262, -2.5908, 0.5556, 0.6691, -0.0912, 2.1089],
[ 1.5669, -0.6453, 0.8954, 0.4817, -0.6550, 0.9734]])
>>> a.size()
torch.Size([6, 6])
- reshape操作
>>> a.reshape(2, -1)
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,
-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,
-0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,
0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,
-0.6550, 0.9734]])
>>> a.reshape(2, -1).shape
torch.Size([2, 18])
可以看出，其实是扁平压缩的。应该说，这个是类似于C++存放数组的方式，将高维数据转成一维数据来处理。之后，获取的时候，只需要用这种方式来进行获取就好了。
但是注意，replace不会覆盖原来的版本
>>> a
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426],
[ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130, 1.0243],
[ 1.2052, -1.0993, 2.5415, 0.9572, -0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258],
[ 0.7262, -2.5908, 0.5556, 0.6691, -0.0912, 2.1089],
[ 1.5669, -0.6453, 0.8954, 0.4817, -0.6550, 0.9734]])
>>> a.shape
torch.Size([6, 6])
- view方法
>>> a.view(2, -1)
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,
-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,
-0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,
0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,
-0.6550, 0.9734]])
>>> a.reshape(2,-1)
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,
-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,
-0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,
0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,
-0.6550, 0.9734]])
但是a也不会覆盖掉
resize_方法
>>> a.resize_(2,18)
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,
-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,
-0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,
0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,
-0.6550, 0.9734]])
>>> a
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,
-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,
-0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,
0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,
-0.6550, 0.9734]])
这次就会发生改变了。
但是注意，这次不允许使用负数（来进行默认计算）
>>> a.resize_(2,-1)
Traceback (most recent call last):
File "" , line 1, in <module>
RuntimeError: sizes must be non-negative
再变回去，内容也不会发生变化
>>> a.resize_(6,6)
tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426],
[ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130, 1.0243],
[ 1.2052, -1.0993, 2.5415, 0.9572, -0.9123, 0.6194],
[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258],
[ 0.7262, -2.5908, 0.5556, 0.6691, -0.0912, 2.1089],
[ 1.5669, -0.6453, 0.8954, 0.4817, -0.6550, 0.9734]])
- unsqueeze()方法
- 压缩
- 其实就是扩充一个维度
- dim这个参数很关键。（从0开始，由于一开始没有dim=1，使用后多出这个维度来）
- 不发生覆盖
>>> a = torch.rand(10)
>>> a
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718])
>>> a.unsqueeze(dim=1)
tensor([[0.5478],
[0.4366],
[0.2502],
[0.5778],
[0.7834],
[0.8406],
[0.3881],
[0.8908],
[0.0255],
[0.4718]])
>>> a
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718])
但是如果维度只有1的话，我们dim=2就会出问题
>>> a.unsqueeze(dim=2)
Traceback (most recent call last):
File "" , line 1, in <module>
RuntimeError: Dimension out of range (expected to be in range of [-2, 1], but got 2)
但是这个报错很有趣，所以，我都试了下
>>> a
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718])
>>> a.unsqueeze(dim=0)
tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718]])
>>> a.unsqueeze(dim=-1)
tensor([[0.5478],
[0.4366],
[0.2502],
[0.5778],
[0.7834],
[0.8406],
[0.3881],
[0.8908],
[0.0255],
[0.4718]])
>>> a.unsqueeze(dim=-2)
tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718]])
- squeeze()反向操作
>>> a
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718])
>>> a.squeeze()
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718])
把数据做一下简单的调整
>>> a
tensor([[0.5478],
[0.4366],
[0.2502],
[0.5778],
[0.7834],
[0.8406],
[0.3881],
[0.8908],
[0.0255],
[0.4718]])
>>> a.squeeze()
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
0.4718])
先压缩一下：
>>> a = a.unsqueeze(dim=1)
>>> a
tensor([[[0.5478]],
[[0.4366]],
[[0.2502]],
[[0.5778]],
[[0.7834]],
[[0.8406]],
[[0.3881]],
[[0.8908]],
[[0.0255]],
[[0.4718]]])
>>> a.shape
torch.Size([10, 1, 1])
>>> a.squeeze()
tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718])
直接就压缩到最简单的版本了。
但是，通过下面的代码结果可以看出来，其实不是压缩到最简单的版本。估计只是考虑了中间只有1的操作
>>> a.resize_(2,5)
tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])
>>> a
tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])
>>> a.squeeze()
tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])
只是把中间的维度为1的给全去掉了。
>>> a.resize_(2, 1, 1, 5, 1, 1)
tensor([[[[[[0.5478]],
[[0.4366]],
[[0.2502]],
[[0.5778]],
[[0.7834]]]]],
[[[[[0.8406]],
[[0.3881]],
[[0.8908]],
[[0.0255]],
[[0.4718]]]]]])
>>> a.squeeze()
tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])
Tensor的切片操作
创建数据
>>> a = torch.randn(6,6)
>>> a
tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],
[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],
[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],
[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],
[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])
- 简单提取和切片的区别
- 后面两个为切片，第一个是直接index提取
>>> a[0]
tensor([-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964])
>>> a[:1]
tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964]])
>>> a[0,:]
tensor([-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964])
- 取第一行到第二行（数值从0开始取）
- 看来默认是取行了。
>>> a[1:3]
tensor([[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100]])
- 取列的话操作类似（取第一列到第二列）
>>> a[:,1:3]
tensor([[ 1.4903, -1.4979],
[-0.8761, -1.9706],
[ 1.3184, 0.2521],
[ 0.3335, -0.1813],
[-0.5142, 0.2758],
[ 0.7929, -0.8389]])
- 最后一行和最后一列
>>> a[-1]
tensor([ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701])
>>> a[:,-1]
tensor([-0.6964, 0.0181, 1.0100, -0.0776, -1.2722, -0.2701])
- 取不连续的行列
- 注意后面的逗号不能省去
- 列是类似的。
>>> a
tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],
[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],
[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],
[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],
[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])
>>> a[(1,3),]
tensor([[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776]])
Tensor的比较
- Tensor > int
>>> a
tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],
[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],
[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],
[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],
[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])
>>> a > 0
tensor([[0, 1, 0, 1, 1, 0],
[0, 0, 0, 1, 0, 1],
[1, 1, 1, 0, 0, 1],
[0, 1, 0, 0, 1, 0],
[0, 0, 1, 1, 1, 0],
[1, 1, 0, 1, 0, 0]], dtype=torch.uint8)
得到的是一个非负整数的矩阵，同等规模。估计是pytorch为了更方便计算，直接用的是uint8类型。
- 显示的时候是有精度的，所以等号不太管用
- 如果你还记得上面的tolist的操作的话，就理解了
>>> a
tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],
[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],
[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],
[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],
[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])
>>> a == -0.9391
tensor([[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]], dtype=torch.uint8)
Tensor的数据筛选
- 比如Tensor大于某个数值的才要之类的操作
>>> a[a>0]
tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,
0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])
这个跟numpy的结果类似
>>> a.numpy()[a.numpy() > 0]
array([1.4902998 , 1.4666233 , 0.18153903, 0.8805863 , 0.01809631,
0.11573527, 1.3184009 , 0.25213283, 1.0099975 , 0.3334596 ,
2.2849436 , 0.2758326 , 0.46773553, 2.0181057 , 1.8403844 ,
0.7928698 , 1.0610487 ], dtype=float32)
Tensor其他常用函数
index_select(input, dim, index)函数
在指定的维度上选取
>>> indices = torch.tensor([0,1])
>>> torch.index_select(a, 1, indices)
tensor([[-0.9391, 1.4903],
[-1.2770, -0.8761],
[ 0.1157, 1.3184],
[-0.2843, 0.3335],
[-0.6731, -0.5142],
[ 1.8404, 0.7929]])
>>> a
tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],
[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],
[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],
[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],
[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],
[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])
记住，这里的tensor不同于Tensor
>>> indices
tensor([0, 1])
>>> torch.Tensor([0,1])
tensor([0., 1.])
发现了吧，必须要是正整数。
而且也必须是Tensor
>>> torch.index_select(a, 1, [0,1])
Traceback (most recent call last):
File "" , line 1, in <module>
TypeError: index_select(): argument 'index' (position 3) must be Tensor, not list
masked_select(input, mask)
- 效果跟a[a>0]类似。
- 只不过可以使用手动创建的更复杂的masked_select操作。
>>> torch.masked_select(a, a>0)
tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,
0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])
>>> a[a>0]
tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,
0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])
nonzero(input)操作
- 返回非0点坐标
>>> a
tensor([[1., 0., 0., 0., 0.],
[0., 1., 0., 0., 0.],
[0., 0., 1., 0., 0.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.]])
>>> torch.nonzero(a)
tensor([[0, 0],
[1, 1],
[2, 2],
[3, 3],
[4, 4]])
gather 根据index，在dim上选取数据，输出size与index一直
- 其实就是一个多重映射
>>> t = torch.tensor([[1,2],[3,4]])
>>> torch.gather(t, 1, torch.tensor([[0,0],[1,0]]))
tensor([[1, 1],
[4, 3]])
>>> torch.gather(t, 0, torch.tensor([[0,0],[1,0]]))
tensor([[1, 2],
[3, 2]])
- 用下面的例子来区分dim=1和dim=0的区别
>>> t
tensor([[1, 2],
[3, 4]])
>>> torch.gather(t, 0, torch.tensor([[0,1],[1,0]]))
tensor([[1, 4],
[3, 2]])
>>> torch.gather(t, 1, torch.tensor([[0,1],[1,0]]))
tensor([[1, 2],
[4, 3]])
看到有这么一段公式
3D-Tensor下的公式。（n维都是类似的）
out[i][j][k] = input[index[i][j][k]][j][k] # if dim == 0
out[i][j][k] = input[i][index[i][j][k]][k] # if dim == 1
out[i][j][k] = input[i][j][index[i][j][k]] # if dim == 2
clamp截断
这个函数就是用来做截断的
>>> t
tensor([[1, 2],
[3, 4]])
>>> torch.clamp(t, 2, 3)
tensor([[2, 2],
[3, 3]])
后记
大概就这么多吧？以后再遇到再补充吧？大家有想法也可以提出来呢~