Caffe layers

Caffe layer 按首字母排序


title: Absolute Value Layer

Absolute Value Layer

  • Layer type: AbsVal

  • Doxygen Documentation

  • Header: ./include/caffe/layers/absval_layer.hpp

  • CPU implementation: ./src/caffe/layers/absval_layer.cpp

  • CUDA GPU implementation: ./src/caffe/layers/absval_layer.cu

  • Sample

    layer {
      name: "layer"
      bottom: "in"
      top: "out"
      type: "AbsVal"
    }
    

The AbsVal layer computes the output as abs(x) for each input element x.


title: Accuracy and Top-k

Accuracy and Top-k

Accuracy scores the output as the accuracy of output with respect to target – it is not actually a loss and has no backward step.

  • Layer type: Accuracy
  • Doxygen Documentation
  • Header: ./include/caffe/layers/accuracy_layer.hpp
  • CPU implementation: ./src/caffe/layers/accuracy_layer.cpp

Parameters

  • Parameters (AccuracyParameter accuracy_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/AccuracyParameter.txt %}
{% endhighlight %}


title: ArgMax Layer

ArgMax Layer

  • Layer type: ArgMax
  • Doxygen Documentation
  • Header: ./include/caffe/layers/argmax_layer.hpp
  • CPU implementation: ./src/caffe/layers/argmax_layer.cpp

Parameters

  • Parameters (ArgMaxParameter argmax_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/ArgMaxParameter.txt %}
{% endhighlight %}


title: Batch Norm Layer

Batch Norm Layer

  • Layer type: BatchNorm
  • Doxygen Documentation
  • Header: ./include/caffe/layers/batch_norm_layer.hpp
  • CPU implementation: ./src/caffe/layers/batch_norm_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/batch_norm_layer.cu

Parameters

  • Parameters (BatchNormParameter batch_norm_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/BatchNormParameter.txt %}
{% endhighlight %}


title: Batch Reindex Layer

Batch Reindex Layer

  • Layer type: BatchReindex
  • Doxygen Documentation
  • Header: ./include/caffe/layers/batch_reindex_layer.hpp
  • CPU implementation: ./src/caffe/layers/batch_reindex_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/batch_reindex_layer.cu

Parameters

No parameters.


title: Bias Layer

Bias Layer

  • Layer type: Bias
  • Doxygen Documentation
  • Header: ./include/caffe/layers/bias_layer.hpp
  • CPU implementation: ./src/caffe/layers/bias_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/bias_layer.cu

Parameters

  • Parameters (BiasParameter bias_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/BiasParameter.txt %}
{% endhighlight %}


title: BNLL Layer

BNLL Layer

  • Layer type: BNLL
  • Doxygen Documentation
  • Header: ./include/caffe/layers/bnll_layer.hpp
  • CPU implementation: ./src/caffe/layers/bnll_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/bnll_layer.cu

The BNLL (binomial normal log likelihood) layer computes the output as log(1 + exp(x)) for each input element x.

Parameters

No parameters.

Sample

  layer {
    name: "layer"
    bottom: "in"
    top: "out"
    type: BNLL
  }

title: Clip Layer

Clip Layer

  • Layer type: Clip
  • Doxygen Documentation
  • Header: ./include/caffe/layers/clip_layer.hpp
  • CPU implementation: ./src/caffe/layers/clip_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/clip_layer.cu

Parameters

  • Parameters (ClipParameter clip_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ClipParameter.txt %}
{% endhighlight %}


title: Concat Layer

Concat Layer

  • Layer type: Concat

  • Doxygen Documentation

  • Header: ./include/caffe/layers/concat_layer.hpp

  • CPU implementation: ./src/caffe/layers/concat_layer.cpp

  • CUDA GPU implementation: ./src/caffe/layers/concat_layer.cu

  • Input

    • n_i * c_i * h * w for each input blob i from 1 to K.
  • Output

    • if axis = 0: (n_1 + n_2 + ... + n_K) * c_1 * h * w, and all input c_i should be the same.
    • if axis = 1: n_1 * (c_1 + c_2 + ... + c_K) * h * w, and all input n_i should be the same.
  • Sample

    layer {
      name: "concat"
      bottom: "in1"
      bottom: "in2"
      top: "out"
      type: "Concat"
      concat_param {
        axis: 1
      }
    }
    

The Concat layer is a utility layer that concatenates its multiple input blobs to one single output blob.

Parameters

  • Parameters (ConcatParameter concat_param)
    • Optional
      • axis [default 1]: 0 for concatenation along num and 1 for channels.
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/ConcatParameter.txt %}
{% endhighlight %}


title: Contrastive Loss Layer

Contrastive Loss Layer

  • Layer type: ContrastiveLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/contrastive_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/contrastive_loss_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/contrastive_loss_layer.cu

Parameters

  • Parameters (ContrastiveLossParameter contrastive_loss_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/ContrastiveLossParameter.txt %}
{% endhighlight %}


title: Convolution Layer

Convolution Layer

  • Layer type: Convolution
  • Doxygen Documentation
  • Header: ./include/caffe/layers/conv_layer.hpp
  • CPU implementation: ./src/caffe/layers/conv_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/conv_layer.cu
  • Input
    • n * c_i * h_i * w_i
  • Output
    • n * c_o * h_o * w_o, where h_o = (h_i + 2 * pad_h - kernel_h) / stride_h + 1 and w_o likewise.

The Convolution layer convolves the input image with a set of learnable filters, each producing one feature map in the output image.

Sample

Sample (as seen in ./models/bvlc_reference_caffenet/train_val.prototxt):

  layer {
    name: "conv1"
    type: "Convolution"
    bottom: "data"
    top: "conv1"
    # learning rate and decay multipliers for the filters
    param { lr_mult: 1 decay_mult: 1 }
    # learning rate and decay multipliers for the biases
    param { lr_mult: 2 decay_mult: 0 }
    convolution_param {
      num_output: 96     # learn 96 filters
      kernel_size: 11    # each filter is 11x11
      stride: 4          # step 4 pixels between each filter application
      weight_filler {
        type: "gaussian" # initialize the filters from a Gaussian
        std: 0.01        # distribution with stdev 0.01 (default mean: 0)
      }
      bias_filler {
        type: "constant" # initialize the biases to zero (0)
        value: 0
      }
    }
  }

Parameters

  • Parameters (ConvolutionParameter convolution_param)
    • Required
      • num_output (c_o): the number of filters
      • kernel_size (or kernel_h and kernel_w): specifies height and width of each filter
    • Strongly Recommended
      • weight_filler [default type: 'constant' value: 0]
    • Optional
      • bias_term [default true]: specifies whether to learn and apply a set of additive biases to the filter outputs
      • pad (or pad_h and pad_w) [default 0]: specifies the number of pixels to (implicitly) add to each side of the input
      • stride (or stride_h and stride_w) [default 1]: specifies the intervals at which to apply the filters to the input
      • group (g) [default 1]: If g > 1, we restrict the connectivity of each filter to a subset of the input. Specifically, the input and output channels are separated into g groups, and the i i ith output group channels will be only connected to the i i ith input group channels.
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/ConvolutionParameter.txt %}
{% endhighlight %}


title: Crop Layer

Crop Layer

  • Layer type: Crop
  • Doxygen Documentation
  • Header: ./include/caffe/layers/crop_layer.hpp
  • CPU implementation: ./src/caffe/layers/crop_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/crop_layer.cu

Parameters

  • Parameters (CropParameter crop_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/CropParameter.txt %}
{% endhighlight %}


title: Database Layer

Database Layer

  • Layer type: Data
  • Doxygen Documentation
  • Header: ./include/caffe/layers/data_layer.hpp
  • CPU implementation: ./src/caffe/layers/data_layer.cpp

Parameters

  • Parameters (DataParameter data_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/DataParameter.txt %}
{% endhighlight %}

  • Parameters
    • Required
      • source: the name of the directory containing the database
      • batch_size: the number of inputs to process at one time
    • Optional
      • rand_skip: skip up to this number of inputs at the beginning; useful for asynchronous sgd
      • backend [default LEVELDB]: choose whether to use a LEVELDB or LMDB

title: Deconvolution Layer

Deconvolution Layer

  • Layer type: Deconvolution
  • Doxygen Documentation
  • Header: ./include/caffe/layers/deconv_layer.hpp
  • CPU implementation: ./src/caffe/layers/deconv_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/deconv_layer.cu

Parameters

Uses the same parameters as the Convolution layer.

  • Parameters (ConvolutionParameter convolution_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/ConvolutionParameter.txt %}
{% endhighlight %}


title: Dropout Layer

Dropout Layer

  • Layer type: Dropout
  • Doxygen Documentation
  • Header: ./include/caffe/layers/dropout_layer.hpp
  • CPU implementation: ./src/caffe/layers/dropout_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/dropout_layer.cu

Parameters

  • Parameters (DropoutParameter dropout_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/DropoutParameter.txt %}
{% endhighlight %}


title: Dummy Data Layer

Dummy Data Layer

  • Layer type: DummyData
  • Doxygen Documentation
  • Header: ./include/caffe/layers/dummy_data_layer.hpp
  • CPU implementation: ./src/caffe/layers/dummy_data_layer.cpp

Parameters

  • Parameters (DummyDataParameter dummy_data_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/DummyDataParameter.txt %}
{% endhighlight %}


title: Eltwise Layer

Eltwise Layer

  • Layer type: Eltwise
  • Doxygen Documentation
  • Header: ./include/caffe/layers/eltwise_layer.hpp
  • CPU implementation: ./src/caffe/layers/eltwise_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/eltwise_layer.cu

Parameters

  • Parameters (EltwiseParameter eltwise_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/EltwiseParameter.txt %}
{% endhighlight %}


title: ELU Layer

ELU Layer

  • Layer type: ELU
  • Doxygen Documentation
  • Header: ./include/caffe/layers/elu_layer.hpp
  • CPU implementation: ./src/caffe/layers/elu_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/elu_layer.cu

References

  • Clevert, Djork-Arne, Thomas Unterthiner, and Sepp Hochreiter.
    “Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)” arXiv:1511.07289. (2015).

Parameters

  • Parameters (ELUParameter elu_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ELUParameter.txt %}
{% endhighlight %}


title: Embed Layer

Embed Layer

  • Layer type: Embed
  • Doxygen Documentation
  • Header: ./include/caffe/layers/embed_layer.hpp
  • CPU implementation: ./src/caffe/layers/embed_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/embed_layer.cu

Parameters

  • Parameters (EmbedParameter embed_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/EmbedParameter.txt %}
{% endhighlight %}


title: Euclidean Loss Layer

Sum-of-Squares / Euclidean Loss Layer

  • Layer type: EuclideanLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/euclidean_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/euclidean_loss_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/euclidean_loss_layer.cu

The Euclidean loss layer computes the sum of squares of differences of its two inputs, 1 2 N ∑ i = 1 N ∥ x i 1 − x i 2 ∥ 2 2 \frac 1 {2N} \sum_{i=1}^N \| x^1_i - x^2_i \|_2^2 2N1i=1Nxi1xi222.

Parameters

Does not take any parameters.


title: Exponential Layer

Exponential Layer

  • Layer type: Exp
  • Doxygen Documentation
  • Header: ./include/caffe/layers/exp_layer.hpp
  • CPU implementation: ./src/caffe/layers/exp_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/exp_layer.cu

Parameters

  • Parameters (Parameter exp_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ExpParameter.txt %}
{% endhighlight %}

See also

  • Power layer

title: Filter Layer

Filter Layer

  • Layer type: Filter
  • Doxygen Documentation
  • Header: ./include/caffe/layers/filter_layer.hpp
  • CPU implementation: ./src/caffe/layers/filter_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/filter_layer.cu

Parameters

Does not take any parameters.


title: Flatten Layer

Flatten Layer

  • Layer type: Flatten
  • Doxygen Documentation
  • Header: ./include/caffe/layers/flatten_layer.hpp
  • CPU implementation: ./src/caffe/layers/flatten_layer.cpp

The Flatten layer is a utility layer that flattens an input of shape n * c * h * w to a simple vector output of shape n * (c*h*w).

Parameters

  • Parameters (FlattenParameter flatten_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/FlattenParameter.txt %}
{% endhighlight %}


title: HDF5 Data Layer

HDF5 Data Layer

  • Layer type: HDF5Data
  • Doxygen Documentation
  • Header: ./include/caffe/layers/hdf5_data_layer.hpp
  • CPU implementation: ./src/caffe/layers/hdf5_data_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/hdf5_data_layer.cu

Parameters

  • Parameters (HDF5DataParameter hdf5_data_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/HDF5DataParameter.txt %}
{% endhighlight %}


title: HDF5 Output Layer

HDF5 Output Layer

  • Layer type: HDF5Output
  • Doxygen Documentation
  • Header: ./include/caffe/layers/hdf5_output_layer.hpp
  • CPU implementation: ./src/caffe/layers/hdf5_output_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/hdf5_output_layer.cu

The HDF5 output layer performs the opposite function of the other layers in this section: it writes its input blobs to disk.

Parameters

  • Parameters (HDF5OutputParameter hdf5_output_param)

    • Required
      • file_name: name of file to write to
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/HDF5OutputParameter.txt %}
{% endhighlight %}


title: Hinge Loss Layer

Hinge (L1, L2) Loss Layer

  • Layer type: HingeLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/hinge_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/hinge_loss_layer.cpp

Parameters

  • Parameters (HingeLossParameter hinge_loss_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/HingeLossParameter.txt %}
{% endhighlight %}


title: Im2col Layer

im2col

  • File type: Im2col
  • Header: ./include/caffe/layers/im2col_layer.hpp
  • CPU implementation: ./src/caffe/layers/im2col_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/im2col_layer.cu

Im2col is a helper for doing the image-to-column transformation that you most
likely do not need to know about. This is used in Caffe’s original convolution
to do matrix multiplication by laying out all patches into a matrix.


title: ImageData Layer

ImageData Layer

  • Layer type: ImageData
  • Doxygen Documentation
  • Header: ./include/caffe/layers/image_data_layer.hpp
  • CPU implementation: ./src/caffe/layers/image_data_layer.cpp

Parameters

  • Parameters (ImageDataParameter image_data_parameter)

    • Required
      • source: name of a text file, with each line giving an image filename and label
      • batch_size: number of images to batch together
    • Optional
      • rand_skip
      • shuffle [default false]
      • new_height, new_width: if provided, resize all images to this size
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ImageDataParameter.txt %}
{% endhighlight %}


title: Infogain Loss Layer

Infogain Loss Layer

  • Layer type: InfogainLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/infogain_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/infogain_loss_layer.cpp

A generalization of MultinomialLogisticLossLayer that takes an “information gain” (infogain) matrix specifying the “value” of all label pairs.

Equivalent to the MultinomialLogisticLossLayer if the infogain matrix is the identity.

Parameters

  • Parameters (Parameter infogain_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/InfogainLossParameter.txt %}
{% endhighlight %}


title: Inner Product / Fully Connected Layer

Inner Product / Fully Connected Layer

  • Layer type: InnerProduct

  • Doxygen Documentation

  • Header: ./include/caffe/layers/inner_product_layer.hpp

  • CPU implementation: ./src/caffe/layers/inner_product_layer.cpp

  • CUDA GPU implementation: ./src/caffe/layers/inner_product_layer.cu

  • Input

    • n * c_i * h_i * w_i
  • Output

    • n * c_o * 1 * 1
  • Sample

    layer {
      name: "fc8"
      type: "InnerProduct"
      # learning rate and decay multipliers for the weights
      param { lr_mult: 1 decay_mult: 1 }
      # learning rate and decay multipliers for the biases
      param { lr_mult: 2 decay_mult: 0 }
      inner_product_param {
        num_output: 1000
        weight_filler {
          type: "gaussian"
          std: 0.01
        }
        bias_filler {
          type: "constant"
          value: 0
        }
      }
      bottom: "fc7"
      top: "fc8"
    }
    

The InnerProduct layer (also usually referred to as the fully connected layer) treats the input as a simple vector and produces an output in the form of a single vector (with the blob’s height and width set to 1).

Parameters

  • Parameters (InnerProductParameter inner_product_param)
    • Required
      • num_output (c_o): the number of filters
    • Strongly recommended
      • weight_filler [default type: 'constant' value: 0]
    • Optional
      • bias_filler [default type: 'constant' value: 0]
      • bias_term [default true]: specifies whether to learn and apply a set of additive biases to the filter outputs
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/InnerProductParameter.txt %}
{% endhighlight %}


title: Input Layer

Input Layer

  • Layer type: Input
  • Doxygen Documentation
  • Header: ./include/caffe/layers/input_layer.hpp
  • CPU implementation: ./src/caffe/layers/input_layer.cpp

Parameters

  • Parameters (InputParameter input_param)
  • From ./src/caffe/proto/caffe.proto):

{% highlight Protobuf %}
{% include proto/InputParameter.txt %}
{% endhighlight %}


title: Log Layer

Log Layer

  • Layer type: Log
  • Doxygen Documentation
  • Header: ./include/caffe/layers/log_layer.hpp
  • CPU implementation: ./src/caffe/layers/log_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/log_layer.cu

Parameters

  • Parameters (Parameter log_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/LogParameter.txt %}
{% endhighlight %}


title: Local Response Normalization (LRN)

Local Response Normalization (LRN)

  • Layer type: LRN
  • Doxygen Documentation
  • Header: ./include/caffe/layers/lrn_layer.hpp
  • CPU Implementation: ./src/caffe/layers/lrn_layer.cpp
  • CUDA GPU Implementation: ./src/caffe/layers/lrn_layer.cu
  • Parameters (LRNParameter lrn_param)
    • Optional
      • local_size [default 5]: the number of channels to sum over (for cross channel LRN) or the side length of the square region to sum over (for within channel LRN)
      • alpha [default 1]: the scaling parameter (see below)
      • beta [default 5]: the exponent (see below)
      • norm_region [default ACROSS_CHANNELS]: whether to sum over adjacent channels (ACROSS_CHANNELS) or nearby spatial locations (WITHIN_CHANNEL)

The local response normalization layer performs a kind of “lateral inhibition” by normalizing over local input regions. In ACROSS_CHANNELS mode, the local regions extend across nearby channels, but have no spatial extent (i.e., they have shape local_size x 1 x 1). In WITHIN_CHANNEL mode, the local regions extend spatially, but are in separate channels (i.e., they have shape 1 x local_size x local_size). Each input value is divided by ( 1 + ( α / n ) ∑ i x i 2 ) β (1 + (\alpha/n) \sum_i x_i^2)^\beta (1+(α/n)ixi2)β, where n n n is the size of each local region, and the sum is taken over the region centered at that value (zero padding is added where necessary).

Parameters

  • Parameters (LRNParameter lrn_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/LRNParameter.txt %}
{% endhighlight %}


title: LSTM Layer

LSTM Layer

  • Layer type: LSTM
  • Doxygen Documentation
  • Header: ./include/caffe/layers/lstm_layer.hpp
  • CPU implementation: ./src/caffe/layers/lstm_layer.cpp
  • CPU implementation (helper): ./src/caffe/layers/lstm_unit_layer.cpp
  • CUDA GPU implementation (helper): ./src/caffe/layers/lstm_unit_layer.cu

Parameters

  • Parameters (Parameter recurrent_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/RecurrentParameter.txt %}
{% endhighlight %}


title: Memory Data Layer

Memory Data Layer

  • Layer type: MemoryData
  • Doxygen Documentation
  • Header: ./include/caffe/layers/memory_data_layer.hpp
  • CPU implementation: ./src/caffe/layers/memory_data_layer.cpp

The memory data layer reads data directly from memory, without copying it. In order to use it, one must call MemoryDataLayer::Reset (from C++) or Net.set_input_arrays (from Python) in order to specify a source of contiguous data (as 4D row major array), which is read one batch-sized chunk at a time.

Parameters

  • Parameters (MemoryDataParameter memory_data_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/MemoryDataParameter.txt %}
{% endhighlight %}

  • Parameters
    • Required
      • batch_size, channels, height, width: specify the size of input chunks to read from memory

title: Multinomial Logistic Loss Layer

Multinomial Logistic Loss Layer

  • Layer type: MultinomialLogisticLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/multinomial_logistic_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/multinomial_logistic_loss_layer.cpp

Parameters

  • Parameters (LossParameter loss_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/LossParameter.txt %}
{% endhighlight %}


title: Mean-Variance Normalization (MVN) Layer

Mean-Variance Normalization (MVN) Layer

  • Layer type: MVN
  • Doxygen Documentation
  • Header: ./include/caffe/layers/mvn_layer.hpp
  • CPU implementation: ./src/caffe/layers/mvn_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/mvn_layer.cu

Parameters

  • Parameters (MVNParameter mvn_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/MVNParameter.txt %}
{% endhighlight %}


title: Parameter Layer

Parameter Layer

  • Layer type: Parameter
  • Doxygen Documentation
  • Header: ./include/caffe/layers/parameter_layer.hpp
  • CPU implementation: ./src/caffe/layers/parameter_layer.cpp

See https://github.com/BVLC/caffe/pull/2079.

Parameters

  • Parameters (ParameterParameter parameter_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ParameterParameter.txt %}
{% endhighlight %}


title: Pooling Layer

Pooling

  • Layer type: Pooling

  • Doxygen Documentation

  • Header: ./include/caffe/layers/pooling_layer.hpp

  • CPU implementation: ./src/caffe/layers/pooling_layer.cpp

  • CUDA GPU implementation: ./src/caffe/layers/pooling_layer.cu

  • Input

    • n * c * h_i * w_i
  • Output

    • n * c * h_o * w_o, where h_o and w_o are computed in the same way as convolution.

Parameters

  • Parameters (PoolingParameter pooling_param)

    • Required
      • kernel_size (or kernel_h and kernel_w): specifies height and width of each filter
    • Optional
      • pool [default MAX]: the pooling method. Currently MAX, AVE, or STOCHASTIC
      • pad (or pad_h and pad_w) [default 0]: specifies the number of pixels to (implicitly) add to each side of the input
      • stride (or stride_h and stride_w) [default 1]: specifies the intervals at which to apply the filters to the input
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/PoolingParameter.txt %}
{% endhighlight %}

Sample

  • Sample (as seen in ./models/bvlc_reference_caffenet/train_val.prototxt)

    layer {
      name: "pool1"
      type: "Pooling"
      bottom: "conv1"
      top: "pool1"
      pooling_param {
        pool: MAX
        kernel_size: 3 # pool over a 3x3 region
        stride: 2      # step two pixels (in the bottom blob) between pooling regions
      }
    }
    

title: Power Layer

Power Layer

  • Layer type: Power
  • Doxygen Documentation
  • Header: ./include/caffe/layers/power_layer.hpp
  • CPU implementation: ./src/caffe/layers/power_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/power_layer.cu

The Power layer computes the output as (shift + scale * x) ^ power for each input element x.

Parameters

  • Parameters (PowerParameter power_param)

    • Optional
      • power [default 1]
      • scale [default 1]
      • shift [default 0]
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/PowerParameter.txt %}
{% endhighlight %}

Sample

  layer {
    name: "layer"
    bottom: "in"
    top: "out"
    type: "Power"
    power_param {
      power: 1
      scale: 1
      shift: 0
    }
  }

See also

  • Exponential layer

title: PReLU Layer

PReLU Layer

  • Layer type: PReLU
  • Doxygen Documentation
  • Header: ./include/caffe/layers/prelu_layer.hpp
  • CPU implementation: ./src/caffe/layers/prelu_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/prelu_layer.cu

Parameters

  • Parameters (PReLUParameter prelu_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/PReLUParameter.txt %}
{% endhighlight %}


title: Python Layer

Python Layer

  • Layer type: Python
  • Doxygen Documentation
  • Header: ./include/caffe/layers/python_layer.hpp

The Python layer allows users to add customized layers without modifying the Caffe core code.

Parameters

  • Parameters (PythonParameter python_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/PythonParameter.txt %}
{% endhighlight %}

Examples and tutorials

  • Simple Euclidean loss example
    ** Python code
    ** Prototxt
  • Tutorial for writing Python layers with DIGITS

title: Recurrent Layer

Recurrent Layer

  • Layer type: Recurrent
  • Doxygen Documentation
  • Header: ./include/caffe/layers/recurrent_layer.hpp
  • CPU implementation: ./src/caffe/layers/recurrent_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/recurrent_layer.cu

Parameters

  • Parameters (RecurrentParameter recurrent_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/RecurrentParameter.txt %}
{% endhighlight %}


title: Reduction Layer

Reduction Layer

  • Layer type: Reduction
  • Doxygen Documentation
  • Header: ./include/caffe/layers/reduction_layer.hpp
  • CPU implementation: ./src/caffe/layers/reduction_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/reduction_layer.cu

Parameters

  • Parameters (ReductionParameter reduction_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ReductionParameter.txt %}
{% endhighlight %}


title: ReLU / Rectified-Linear and Leaky-ReLU Layer

ReLU / Rectified-Linear and Leaky-ReLU Layer

  • Layer type: ReLU

  • Doxygen Documentation

  • Header: ./include/caffe/layers/relu_layer.hpp

  • CPU implementation: ./src/caffe/layers/relu_layer.cpp

  • CUDA GPU implementation: ./src/caffe/layers/relu_layer.cu

  • Sample (as seen in ./models/bvlc_reference_caffenet/train_val.prototxt)

    layer {
      name: "relu1"
      type: "ReLU"
      bottom: "conv1"
      top: "conv1"
    }
    

Given an input value x, The ReLU layer computes the output as x if x > 0 and negative_slope * x if x <= 0. When the negative slope parameter is not set, it is equivalent to the standard ReLU function of taking max(x, 0). It also supports in-place computation, meaning that the bottom and the top blob could be the same to preserve memory consumption.

Parameters

  • Parameters (ReLUParameter relu_param)
    • Optional
      • negative_slope [default 0]: specifies whether to leak the negative part by multiplying it with the slope value rather than setting it to 0.
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ReLUParameter.txt %}
{% endhighlight %}


title: Reshape Layer

Reshape Layer

  • Layer type: Reshape

  • Doxygen Documentation

  • Header: ./include/caffe/layers/reshape_layer.hpp

  • Implementation: ./src/caffe/layers/reshape_layer.cpp

  • Input

    • a single blob with arbitrary dimensions
  • Output

    • the same blob, with modified dimensions, as specified by reshape_param
  • Sample

      layer {
        name: "reshape"
        type: "Reshape"
        bottom: "input"
        top: "output"
        reshape_param {
          shape {
            dim: 0  # copy the dimension from below
            dim: 2
            dim: 3
            dim: -1 # infer it from the other dimensions
          }
        }
      }
    

The Reshape layer can be used to change the dimensions of its input, without changing its data. Just like the Flatten layer, only the dimensions are changed; no data is copied in the process.

Output dimensions are specified by the ReshapeParam proto. Positive numbers are used directly, setting the corresponding dimension of the output blob. In addition, two special values are accepted for any of the target dimension values:

  • 0 means “copy the respective dimension of the bottom layer”. That is, if the bottom has 2 as its 1st dimension, the top will have 2 as its 1st dimension as well, given dim: 0 as the 1st target dimension.
  • -1 stands for “infer this from the other dimensions”. This behavior is similar to that of -1 in numpy’s or [] for MATLAB’s reshape: this dimension is calculated to keep the overall element count the same as in the bottom layer. At most one -1 can be used in a reshape operation.

As another example, specifying reshape_param { shape { dim: 0 dim: -1 } } makes the layer behave in exactly the same way as the Flatten layer.

Parameters

  • Parameters (ReshapeParameter reshape_param)
    • Optional: (also see detailed description below)
      • shape
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ReshapeParameter.txt %}
{% endhighlight %}


title: RNN Layer

RNN Layer

  • Layer type: RNN
  • Doxygen Documentation
  • Header: ./include/caffe/layers/rnn_layer.hpp
  • CPU implementation: ./src/caffe/layers/rnn_layer.cpp

Parameters

  • Parameters (RecurrentParameter recurrent_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/RecurrentParameter.txt %}
{% endhighlight %}


title: Scale Layer

Scale Layer

  • Layer type: Scale
  • Doxygen Documentation
  • Header: ./include/caffe/layers/scale_layer.hpp
  • CPU implementation: ./src/caffe/layers/scale_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/scale_layer.cu

Parameters

  • Parameters (ScaleParameter scale_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ScaleParameter.txt %}
{% endhighlight %}


title: Sigmoid Layer

Sigmoid Layer

  • Layer type: Sigmoid

  • Doxygen Documentation

  • Header: ./include/caffe/layers/sigmoid_layer.hpp

  • CPU implementation: ./src/caffe/layers/sigmoid_layer.cpp

  • CUDA GPU implementation: ./src/caffe/layers/sigmoid_layer.cu

  • Example (from ./examples/mnist/mnist_autoencoder.prototxt):

    layer {
      name: "encode1neuron"
      bottom: "encode1"
      top: "encode1neuron"
      type: "Sigmoid"
    }
    

The Sigmoid layer computes sigmoid(x) for each element x in the bottom blob.

Parameters

  • Parameters (SigmoidParameter sigmoid_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/SigmoidParameter.txt %}
{% endhighlight %}


title: Sigmoid Cross-Entropy Loss Layer

Sigmoid Cross-Entropy Loss Layer

  • Layer type: SigmoidCrossEntropyLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/sigmoid_cross_entropy_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/sigmoid_cross_entropy_loss_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/sigmoid_cross_entropy_loss_layer.cu

To-do.


title: Silence Layer

Silence Layer

  • Layer type: Silence
  • Doxygen Documentation
  • Header: ./include/caffe/layers/silence_layer.hpp
  • CPU implementation: ./src/caffe/layers/silence_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/silence_layer.cu

Silences a blob, so that it is not printed.

Parameters

No parameters.


title: Slice Layer

Slice Layer

  • Layer type: Slice
  • Doxygen Documentation
  • Header: ./include/caffe/layers/slice_layer.hpp
  • CPU implementation: ./src/caffe/layers/slice_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/slice_layer.cu

The Slice layer is a utility layer that slices an input layer to multiple output layers along a given dimension (currently num or channel only) with given slice indices.

  • Sample

    layer {
      name: "slicer_label"
      type: "Slice"
      bottom: "label"
      ## Example of label with a shape N x 3 x 1 x 1
      top: "label1"
      top: "label2"
      top: "label3"
      slice_param {
        axis: 1
        slice_point: 1
        slice_point: 2
      }
    }
    

axis indicates the target axis; slice_point indicates indexes in the selected dimension (the number of indices must be equal to the number of top blobs minus one).

Parameters

  • Parameters (SliceParameter slice_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/SliceParameter.txt %}
{% endhighlight %}


title: Softmax Layer

Softmax Layer

  • Layer type: Softmax
  • Doxygen Documentation
  • Header: ./include/caffe/layers/softmax_layer.hpp
  • CPU implementation: ./src/caffe/layers/softmax_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/softmax_layer.cu

Parameters

  • Parameters (SoftmaxParameter softmax_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/SoftmaxParameter.txt %}
{% endhighlight %}

See also

  • Softmax loss layer

title: Softmax with Loss Layer

Softmax with Loss Layer

  • Layer type: SoftmaxWithLoss
  • Doxygen Documentation
  • Header: ./include/caffe/layers/softmax_loss_layer.hpp
  • CPU implementation: ./src/caffe/layers/softmax_loss_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/softmax_loss_layer.cu

The softmax loss layer computes the multinomial logistic loss of the softmax of its inputs. It’s conceptually identical to a softmax layer followed by a multinomial logistic loss layer, but provides a more numerically stable gradient.

Parameters

  • Parameters (SoftmaxParameter softmax_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/SoftmaxParameter.txt %}
{% endhighlight %}

  • Parameters (LossParameter loss_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/LossParameter.txt %}
{% endhighlight %}

See also

  • Softmax layer

title: Split Layer

Split Layer

  • Layer type: Split
  • Doxygen Documentation
  • Header: ./include/caffe/layers/split_layer.hpp
  • CPU implementation: ./src/caffe/layers/split_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/split_layer.cu

The Split layer is a utility layer that splits an input blob to multiple output blobs. This is used when a blob is fed into multiple output layers.

Parameters

Does not take any parameters.


title: Spatial Pyramid Pooling Layer

Spatial Pyramid Pooling Layer

  • Layer type: SPP
  • Doxygen Documentation
  • Header: ./include/caffe/layers/spp_layer.hpp
  • CPU implementation: ./src/caffe/layers/spp_layer.cpp

Parameters

  • Parameters (SPPParameter spp_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/SPPParameter.txt %}
{% endhighlight %}


title: TanH Layer

TanH Layer

  • Header: ./include/caffe/layers/tanh_layer.hpp
  • CPU implementation: ./src/caffe/layers/tanh_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/tanh_layer.cu

Parameters

  • Parameters (TanHParameter tanh_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/TanHParameter.txt %}
{% endhighlight %}


title: Threshold Layer

Threshold Layer

  • Header: ./include/caffe/layers/threshold_layer.hpp
  • CPU implementation: ./src/caffe/layers/threshold_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/threshold_layer.cu

Parameters

  • Parameters (ThresholdParameter threshold_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/ThresholdParameter.txt %}
{% endhighlight %}


title: Tile Layer

Tile Layer

  • Layer type: Tile
  • Doxygen Documentation
  • Header: ./include/caffe/layers/tile_layer.hpp
  • CPU implementation: ./src/caffe/layers/tile_layer.cpp
  • CUDA GPU implementation: ./src/caffe/layers/tile_layer.cu

Parameters

  • Parameters (TileParameter tile_param)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/TileParameter.txt %}
{% endhighlight %}


title: WindowData Layer

WindowData Layer

  • Layer type: WindowData
  • Doxygen Documentation
  • Header: ./include/caffe/layers/window_data_layer.hpp
  • CPU implementation: ./src/caffe/layers/window_data_layer.cpp

Parameters

  • Parameters (WindowDataParameter)
  • From ./src/caffe/proto/caffe.proto:

{% highlight Protobuf %}
{% include proto/WindowDataParameter.txt %}
{% endhighlight %}

你可能感兴趣的:(#,Caffe,深度学习)