从论文到代码完成 RoIPooling

从论文到代码完成 RoIPooling

RoI Pooling
得到特征图和候选框,就会将候选框投影在特征图,然后停止一次缩放得到大小分歧的特征图,在 Faster RCNN 中,区域候选框用来预测对象是前景还是背景,这是 class head 要做的工作,而 regression 是学习到基于 anchor 的差分,也就是中心的偏移量和宽高的缩放。

在投影过程中候选框的尺寸和位置是相关于输入图像,而不是相关于特征图,首先需求将其停止转换到候选框在特征图上详细位置,然后在对提取候选框停止尺寸的缩放。

给定一个特征图和一组提议,返回汇合的特征表示。区域提议网络被用来预测对象性和回归盒的偏向(对锚点)。这些偏移量与 anchor 分离起来生成候选框。这些倡议通常是输入图像的大小而不是特征层的大小。因而,这些倡议需求按比例减少到特征图层,之所以这样做,以便下游的CNN层可以提取特征。

我们在原图上有一个尺寸,也就是候选框中心点的坐标以及宽度,首先我们投影在原图上坐标点除以下采样的倍数,也就是 32 倍下采样,假如坐标无法整除则停止取整操作。

import numpy as np
import torch
import torch.nn as nn
floattype = torch.cuda.FloatTensor
class TorchROIPool(object):
    def __init__(self, output_size, scaling_factor):
        #输出特征图的尺寸
        self.output_size = output_size
        #缩放比率
        self.scaling_factor = scaling_factor
    def _roi_pool(self, features):
        """
        在给定的缩放提取特征图根底,返回固定大小的特征图
        Args:
            features (np.Array): 
        """
        # 特征图的通道数、高 和 宽
        num_channels, h, w = features.shape
        
        # 计算步长
        w_stride = w/self.output_size
        h_stride = h/self.output_size
        # 
        res = torch.zeros((num_channels, self.output_size, self.output_size))
        res_idx = torch.zeros((num_channels, self.output_size, self.output_size))
        
        for i in range(self.output_size):
            for j in range(self.output_size):
                
                # important to round the start and end, and then conver to int
                # 
                w_start = int(np.floor(j*w_stride))
                w_end = int(np.ceil((j+1)*w_stride))
                h_start = int(np.floor(i*h_stride))
                h_end = int(np.ceil((i+1)*h_stride))
                # limiting start and end based on feature limits
                # 
                w_start = min(max(w_start, 0), w)
                w_end = min(max(w_end, 0), w)
                h_start = min(max(h_start, 0), h)
                h_end = min(max(h_end, 0), h)
                patch = features[:, h_start: h_end, w_start: w_end]
                max_val, max_idx = torch.max(patch.reshape(num_channels, -1), dim=1)
                res[:, i, j] = max_val
                res_idx[:, i, j] = max_idx
        return res, res_idx
    def __call__(self, feature_layer, proposals):
        """Given feature layers and a list of proposals, it returns pooled
        respresentations of the proposals. Proposals are scaled by scaling factor
        before pooling.
        Args:
            feature_layer (np.Array): 特征层尺寸
            proposals (list of np.Array): 列表中每一个元素 Each element of the list represents a bounding
            box as (w,y,w,h)
        Returns:
            np.Array: proposal 数量,通道数,输出特征图高度, self.output_size
        """
        batch_size, num_channels, _, _ = feature_layer.shape
        # first scale proposals based on self.scaling factor 
        scaled_proposals = torch.zeros_like(proposals)
        # the rounding by torch.ceil is important for ROI pool
        scaled_proposals[:, 0] = torch.ceil(proposals[:, 0] * self.scaling_factor)
        scaled_proposals[:, 1] = torch.ceil(proposals[:, 1] * self.scaling_factor)
        scaled_proposals[:, 2] = torch.ceil(proposals[:, 2] * self.scaling_factor)
        scaled_proposals[:, 3] = torch.ceil(proposals[:, 3] * self.scaling_factor)
        res = torch.zeros((len(proposals), num_channels, self.output_size,
                        self.output_size))
        res_idx = torch.zeros((len(proposals), num_channels, self.output_size,
                        self.output_size))
        
        # 遍历候选框
        for idx in range(len(proposals)):
            #
            proposal = scaled_proposals[idx]
            # adding 1 to include the end indices from proposal
            extracted_feat = feature_layer[0, :, proposal[1].to(dtype=torch.int8):proposal[3].to(dtype=torch.int8)+1, proposal[0].to(dtype=torch.int8):proposal[2].to(dtype=torch.int8)+1]
            res[idx], res_idx[idx] = self._roi_pool(extracted_feat)
        return res

你可能感兴趣的:(ruby)