Torch与TensorFlow卷积对比

1. 一维卷积对比:

一维卷积tensorflow2版本的Conv1D以及Pytroch的nn.Conv1d用法 - 知乎

简单总结:

1.1 torch的1d卷积核在最后一维上滑动,tf的1d卷积核在倒数第2维上滑动。torch的in_channel是倒数第二维,tf的in_channel是倒数第一维。

1.2 torch需要指定in_channel和out_channel,前者是倒数第二维的维度,后者实际就是filter的数量,对应mlp中的输出维度。

torch.nn.Conv1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)

in_channels:在文本应用中,即为词向量的维度(文本经过embedding之后的维度)
out_channels:卷积产生的通道数,相当于是将词向量的维度从in_channels变成了out_channels
kernel_size:卷积核的尺寸;卷积核的第二个维度由in_channels决定,所以实际上卷积核的大小为kernel_size * in_channels
padding:对输入的每一条边,补充0的层数

1.3 tf不需要指定in_channel,in_channel默认就是最后一维(也就是通常H*W*C中的C),只需要指定out_channel和kernel_size。这样看来tf的设计更好一些,因为in_channle已经通过固定维度指定了,不会发生变化,这一点上来说torch可以向tf学习。

conv1D = tf.keras.layers.Conv1D(1, 3, padding='valid')  #1是输出通道数,3是卷积核大小,不使用边界填充

2. 二维卷积对比:

2.1 torch版本:

torch.nn.Conv2d(in_channelsout_channelskernel_sizestride=1padding=0dilation=1groups=1bias=Truepadding_mode='zeros'device=Nonedtype=None)

torch卷积输入数据格式是 (N,Cin​,H,W) , 输出是 (N,Cout​,Hout​,Wout​) 

其中kernel_sizestridepaddingdilation 三个参数有两种设置方法:

  • a single int – in which case the same value is used for the height and width dimension

  • tuple of two ints – in which case, the first int is used for the height dimension, and the second int for the width dimension

2.2 tf.layer版本

tf卷积输入数据格式是 (N,H,W,Cin​) , 输出是 (N​,Hout​,Wout​,Cout) 

tf.layers.Conv2D(inputs, filters, kernel_size, strides=(1, 1), padding='valid')

其中kernel_size和strides都需要设置为一个list或者integer,integer会对所有维度生效相同的值。

Arguments

inputs Tensor input.
filters Integer, the dimensionality of the output space (i.e. the number of filters in the convolution).
kernel_size An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.
strides An integer or tuple/list of 2 integers, specifying the strides of the convolution along the height and width. Can be a single integer to specify the same value for all spatial dimensions. Specifying any stride value != 1 is incompatible with specifying any dilation_rate value != 1.
padding One of "valid" or "same" (case-insensitive).

2.3 tf.nn版本

tf.nn.conv2d(
    input, filter=None, strides=None, padding=None, use_cudnn_on_gpu=True,
    data_format='NHWC', dilations=[1, 1, 1, 1], name=None, filters=None
)

Args

input Tensor. Must be one of the following types: halfbfloat16float32float64. A 4-D tensor. The dimension order is interpreted according to the value of data_format, see below for details.
filter Tensor. Must have the same type as input. A 4-D tensor of shape [filter_height, filter_width, in_channels, out_channels]
strides An int or list of ints that has length 12 or 4. The stride of the sliding window for each dimension of input. If a single value is given it is replicated in the H and W dimension. By default the N and C dimensions are set to 1. The dimension order is determined by the value of data_format, see below for details.
padding Either the string "SAME" or "VALID" indicating the type of padding algorithm to use, or a list indicating the explicit paddings at the start and end of each dimension. When explicit padding is used and data_format is "NHWC", this should be in the form [[0, 0], [pad_top,pad_bottom], [pad_left, pad_right], [0, 0]]. When explicit padding used and data_format is "NCHW", this should be in the form [[0, 0], [0, 0],[pad_top, pad_bottom], [pad_left, pad_right]].

3. 用Conv实现Mlp

(1)torch中对于数据【N,C,H,W】这种四维数据可以用Conv2d实现mlp,只需要将kernal_size和stride_size都设置成【1,1】,filter_size设置成mlp的out_channel即可实现第二维从C变成filter_size的mlp,对于tf来说【N,H,W,C】的数据同理。

(2)torch中对于数据【N,C,H】这种四维数据可以用Conv1d实现mlp,只需要将kernal_size和stride_size都设置成【1】,filter_size设置成mlp的out_channel即可实现第二维从C变成filter_size的mlp,对于tf来说【N,H,C】的数据同理。

你可能感兴趣的:(torch与tf使用笔记,cnn,深度学习,pytorch)