Python 装饰器，@property 以及 Pycaffe.py

Python装饰器的知识请参考：12步轻松搞定python装饰器
@property函数的知识请参考：Python进阶之“属性（property）”详解

下面贴一段pycaffe.py中的代码

@property
def _Net_blobs(self):
    """ An OrderedDict (bottom to top, i.e., input to output) of network blobs indexed by name """
    return OrderedDict(zip(self._blob_names, self._blobs))


@property
def _Net_blob_loss_weights(self):
    """ An OrderedDict (bottom to top, i.e., input to output) of network blob loss weights indexed by name """
    return OrderedDict(zip(self._blob_names, self._blob_loss_weights))


@property
def _Net_params(self):
    """ An OrderedDict (bottom to top, i.e., input to output) of network parameters indexed by name; each is a list of multiple blobs (e.g., weights and biases) """
    return OrderedDict([(name, lr.blobs)
                        for name, lr in zip(self._layer_names, self.layers)
                        if len(lr.blobs) > 0])


@property
def _Net_inputs(self):
    return [list(self.blobs.keys())[i] for i in self._inputs]


@property
def _Net_outputs(self):
    return [list(self.blobs.keys())[i] for i in self._outputs]


def _Net_forward(self, blobs=None, start=None, end=None, **kwargs):
    """ Forward pass: prepare inputs and run the net forward. Parameters ---------- blobs : list of blobs to return in addition to output blobs. kwargs : Keys are input blob names and values are blob ndarrays. For formatting inputs for Caffe, see Net.preprocess(). If None, input is taken from data layers. start : optional name of layer at which to begin the forward pass end : optional name of layer at which to finish the forward pass (inclusive) Returns ------- outs : {blob name: blob ndarray} dict. """
    if blobs is None:
        blobs = []

    if start is not None:
        start_ind = list(self._layer_names).index(start)
    else:
        start_ind = 0

    if end is not None:
        end_ind = list(self._layer_names).index(end)
        outputs = set([end] + blobs)
    else:
        end_ind = len(self.layers) - 1
        outputs = set(self.outputs + blobs)

    if kwargs:
        if set(kwargs.keys()) != set(self.inputs):
            raise Exception('Input blob arguments do not match net inputs.')
        # Set input according to defined shapes and make arrays single and
        # C-contiguous as Caffe expects.
        for in_, blob in kwargs.iteritems():
            if blob.shape[0] != self.blobs[in_].num:
                raise Exception('Input is not batch sized')
            self.blobs[in_].data[...] = blob

    self._forward(start_ind, end_ind)

    # Unpack blobs to extract
    return {out: self.blobs[out].data for out in outputs}


def _Net_backward(self, diffs=None, start=None, end=None, **kwargs):
    """ Backward pass: prepare diffs and run the net backward. Parameters ---------- diffs : list of diffs to return in addition to bottom diffs. kwargs : Keys are output blob names and values are diff ndarrays. If None, top diffs are taken from forward loss. start : optional name of layer at which to begin the backward pass end : optional name of layer at which to finish the backward pass (inclusive) Returns ------- outs: {blob name: diff ndarray} dict. """
    if diffs is None:
        diffs = []

    if start is not None:
        start_ind = list(self._layer_names).index(start)
    else:
        start_ind = len(self.layers) - 1

    if end is not None:
        end_ind = list(self._layer_names).index(end)
        outputs = set([end] + diffs)
    else:
        end_ind = 0
        outputs = set(self.inputs + diffs)

    if kwargs:
        if set(kwargs.keys()) != set(self.outputs):
            raise Exception('Top diff arguments do not match net outputs.')
        # Set top diffs according to defined shapes and make arrays single and
        # C-contiguous as Caffe expects.
        for top, diff in kwargs.iteritems():
            if diff.shape[0] != self.blobs[top].num:
                raise Exception('Diff is not batch sized')
            self.blobs[top].diff[...] = diff

    self._backward(start_ind, end_ind)

    # Unpack diffs to extract
    return {out: self.blobs[out].diff for out in outputs}

初次看的时候，也看不懂，没办法，只能靠自己研究。以下纯属个人理解，学识有限，求指正！
此处分割线

在Python进阶之“属性（property）”详解中有提到：
Python中有一个被称为属性函数(property)的小概念，它可以做一些有用的事情, 比如：1. 将类方法转换为只读属性 2. 重新实现一个属性的setter和getter方法。所以：
1. 很自然的一个问题就是：在pycaffe.py中，并没有声明一个类，而且其中的一些函数的声明都包含了self参数，self是跟具体某个类的实例对象绑定的，那么这个类的定义在哪里呢？？？
首先，Python中可以在类外定义方法（我觉得这应该没有问题，毕竟python是一门非常强大的语言）找到一篇文章Python class 入门，可以参考一下：下面是我自己定义的一个类：

#! /usr/bin/env python

@property
def upperName(self):
    return self.name.upper()

class Person:
    Year = 2016 # 类属性，而不是实例属性
    def __init__(self, name):
        self.name = name
        self.age = 23
    @property
    def name(self):
        return self.name

在解释器中创建实例对象：p = Person("sam")，然后查看一些属性, 如下:

>>> Person.__dict__
{'__module__': '__main__', '__doc__': None, 'name': <property object at 0x7f094fa965d0>, '__init__': <function __init__ at 0x7f094fa9f6e0>, 'Year': 2016}

可以看到此时的类对象Person并没有upperName这个属性

>>> p.__dict__
{'age': 23, 'name': 'sam'}

这是实例对象p的属性

然后，将upperName这个对象赋值给Person.upper，此行代码同时给类Person创建了一个新的属性upper。这就是python的强大之处：能够在“运行时”创建属性。
注意在python中，一切皆为对象，类是对象，类的实例也是对象，函数是对象，变量是对象等等。。。

Person.upper = upperName

在运行这行代码之前，运行p.upperName 或者Person.upperName 都会报错：

>>> p.upperName
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Person instance has no attribute 'upperName'
>>> Person.upperName
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: class Person has no attribute 'upperName'

但，如果运行了 Person.upper = upperName 呢？我们先看Person的属性：

>>> Person.__dict__
{'upper': <property object at 0x7f094fa96578>, '__module__': '__main__', 'name': <property object at 0x7f094fa965d0>, 'Year': 2016, '__doc__': None, '__init__': <function __init__ at 0x7f094fa9f6e0>}

显然多了一个属性：'upper': <property object at 0x7f094fa96578>, 而实例对象p就没有变化：

>>> p.__dict__
{'age': 23, 'name': 'sam'}

接下来，调用：

>>> p.upper
'SAM'

OK，可见python的强大之处。
注意这是利用实例对象调用的upper，如用类来调用，并不会运行def upperName(self), 只是输出一些信息：

>>> Person.upper
<property object at 0x7f094fa96578>

那么，在pycaffe.py中，情况如何呢？？？请看下面的代码：

# Attach methods to Net.
Net.blobs = _Net_blobs
Net.blob_loss_weights = _Net_blob_loss_weights
Net.params = _Net_params
Net.forward = _Net_forward
Net.backward = _Net_backward
Net.forward_all = _Net_forward_all
Net.forward_backward_all = _Net_forward_backward_all
Net.set_input_arrays = _Net_set_input_arrays
Net._batch = _Net_batch
Net.inputs = _Net_inputs
Net.outputs = _Net_outputs

Net是从_caffe中导入的一个类（个人理解）：from ._caffe import Net，那么上述代码就是将在pycaffe.py中定义的属性“绑定”到Net上，也就是Person.upper = upperName 这个操作。

最后，用pacaffe.py中的注释作为总结： Wrap the internal caffe C++ module (_caffe.so) with a clean, Pythonic
interface.

以_Net_blobs说下这些属性

@property
def _Net_blobs(self):
    """ An OrderedDict (bottom to top, i.e., input to output) of network blobs indexed by name """
    return OrderedDict(zip(self._blob_names, self._blobs))

a) self._blob_names 调用Caffe.Net的blob_names属性（底层调用Net::blob_names方法），返回blob_names ， 即整个网络中所有非参数blob的name
b) self._blobs 调用Caffe.Net的blobs属性（底层调用Net::blobs方法）返回blobs, 即整个网络中所有非参数blob
c) return OrderedDict(zip(self._blob_names, self._blobs)) 以字典的方式返回

还有一个类：

class _Net_IdNameWrapper:
    """ A simple wrapper that allows the ids propery to be accessed as a dict indexed by names. Used for top and bottom names """
    def __init__(self, net, func):
        self.net, self.func = net, func

    def __getitem__(self, name):
        # Map the layer name to id
        ids = self.func(self.net, list(self.net._layer_names).index(name))
        # Map the blob id to name
        id_to_name = list(self.net.blobs)
        return [id_to_name[i] for i in ids]

对应的有：

# property为内建属性函数
Net.top_names = property(lambda n: _Net_IdNameWrapper(n, Net._top_ids))
Net.bottom_names = property(lambda n: _Net_IdNameWrapper(n, Net._bottom_ids))

在底层，top blob， bottom blob 是按id来索引访问的，所以在底层就相应的有top_id_vecs_ 和 bottom_id_vecs_ 这两个变量。而class _Net_IdNameWrapper 的功能就是将这种按id访问，改为按blob的name来访问，在训练的时候带来方便性。
下面以 Net.top_names = property(lambda n: _Net_IdNameWrapper(n, Net._top_ids)) 为例，解释一下。
a) _Net_IdNameWrapper(n, Net._top_ids) 创建一个_Net_IdNameWrapper对象， 那么它是如何实现利用blob name来访问blob的呢？
b) 其实是通过def getitem(self, name)来定制的: ids = self.func(self.net, list(self.net._layer_names).index(name)) 这里self，表示Net_IdNameWrapper实例对象。self.net._layer_names调用Caffe.Net的_layer_names属性，返回网络中所有layer的名字，index(name)找到对应name的layer的id，而self.func即为Net._top_ids方法（定义在_caffe.cpp中），它调用底层的Net::top_ids方法，返回Net::top_id_vecs, 其类型为：vector<vector<int> >，所以综合一下可知，ids为名字为name的layer的所以top blob 的id。 这里需要注意的是list(dict) 返回的是dict的键的列表
c) id_to_name = list(self.net.blobs) ：self.net.blobs调用caffe.Net的blobs属性返回网络中所有的非参数blob所组成的字典，这是因为caffe.Net的blobs属性已经经过wraped .[参见def _Net_blobs(self)]。即id_to_name是一个blob name 的列表，里面存储了网络的所有非参数blob的名字。
d) return [id_to_name[i] for i in ids] 将所有需要的blob name以列表的形式返回。

总之Net.top_names这个属性的功能就是返回给定名字为name的层的所有top blob。