在JM8.6解码端提取DCT系数和残差并模拟验证itrans函数的功能
如何在JM8.6的解码端提取DCT系数呢？自然而然会想到的问题是：DCT系数从哪里来，要到哪里去，所以，要提取DCT系数，可以看它是哪里产生的，也可以看它要到哪里去，然后在任意一条路上"截击"它，必然能找到DCT系数. 下面找DCT系数的思路是要看DCT系数到哪里去. 很显然，在解码端，DCT系数的下一个动作是进行反DCT变换，所以自然而言找到反DCT变换函数，找了一下，发现itans函数正是反DCT变换的函数.
先把JM8.6中的itrans函数的原型列出来：
void itrans(struct img_par *img,
int ioff,
int joff,
int i0,
int j0);
仍然以foreman视频第一帧第一宏块第一个4*4块为例：
在itrans函数中，我们分别打印出img->cof和img->mpr,得到的结果分别为：
2304 -3840 -2816 -1600
960 -1200 320 0
768 -320 -512 320
0 0 0 0
128 128 128 128
128 128 128 128
128 128 128 128
128 128 128 128
利用H.264visa分析得：
====================== Y Data ====================== （解码端的DCT系数）
+------------------------+------------------------+------------------------+------------------------+
| 2304,-3840,-2816,-1600,| 768, 640, -256, 640,| 1280, 320, 256, -640,| 768, -320, -768, 0,|
| 960,-1200, 320, 0,| 0, 0, 320, 0,| -640, -800, 0, 0,| 960, -800, 320, 0,|
| 768, -320, -512, 320,| 512, -640, 0, -320,| -768, 320, -512, 320,| 768, 0, 256, 0,|
| 0, 0, 0, 0,| 0, 0, 0, 0,| 0, 400, 0, 0,| -320, 0, 0, 0,|
+------------------------+------------------------+------------------------+------------------------+
|-1024,-5120,-1792, -640,| 2560, -640, -256, 0,| 512, 0, 0, 0,| 0, 0, 0, 0,|
| 0, 1200, -640, -400,| -320, 400, 0, 0,| 640, 0, 0, 0,| 320, 0, 0, 0,|
| 512, 0, -256, 0,| 0, 0, 0, 0,| 0, 0, 0, 0,| 0, 0, 0, 0,|
| 320, 0, 0, 0,| -320, 0, 0, 0,| 0, 0, 0, 0,| 0, 0, 0, 0,|
+------------------------+------------------------+------------------------+------------------------+
| 0, 320, 0, 0,| 0, 0, 0, 0,| -768, 640, 0, 0,| 512, 0, -256, 0,|
| 0, 0, 0, 0,| 0, 0, 0, 0,| 640, -800, 0, 0,| -640, -400, 320, 0,|
| 0, 0, 0, 0,| 0, 0, 0, 0,| -256, 320, 0, 0,| 0, 320, 0, 0,|
| 0, 0, 0, 0,| 0, 0, 0, 0,| 320, -400, 0, 0,| -320, 0, 0, 0,|
+------------------------+------------------------+------------------------+------------------------+
| -512, 640, -256, 0,| 0, 0, 0, 0,| 1024, -320, -256, 0,| 1024, -320, 0, 0,|
| 960,-1200, 320, 0,| 960, -800, 320, 0,|-1280, 800, 0, 400,| -640, 400, 0, 0,|
| -512, 320, 0, 0,| 0, 0, -256, 0,| 0, 0, -256, 0,| 512, 640, 0, 0,|
| 0, 0, 0, 0,| -320, 0, 0, 0,| -640, 800, 0, 0,| 0, 0, 0, 0,|
+------------------------+------------------------+------------------------+------------------------+
====================== Y Data ====================== （预测值）
+----------------+----------------+----------------+----------------+
|128,128,128,128,|238,238,238,238,|255,255,255,255,|245,228,210,192,|
|128,128,128,128,|205,205,205,205,|180,180,180,180,|210,192,174,170,|
|128,128,128,128,|167,167,167,167,|137,137,137,137,|174,170,166,166,|
|128,128,128,128,|160,160,160,160,|169,169,169,169,|166,166,166,166,|
+----------------+----------------+----------------+----------------+
|161,161,161,161,|179,189,182,169,|201,201,201,201,|219,219,219,219,|
|161,161,161,161,|184,185,175,169,|214,214,214,214,|227,227,227,227,|
|161,161,161,161,|189,182,169,169,|216,216,216,216,|219,219,219,219,|
|161,161,161,161,|185,175,169,169,|229,229,229,229,|227,227,227,227,|
+----------------+----------------+----------------+----------------+
| 28,127,218,229,|224,224,224,224,|227,227,228,227,|224,224,224,224,|
| 28,127,218,229,|224,224,224,224,|224,225,227,227,|214,214,214,214,|
| 28,127,218,229,|224,224,224,224,|224,224,224,225,|212,212,212,212,|
| 28,127,218,229,|224,224,224,224,|224,224,224,224,|165,165,165,165,|
+----------------+----------------+----------------+----------------+
| 33,130,216,224,|231,223,215,194,|204,182,160,142,|127,133,139,147,|
| 33,130,216,224,|215,194,173,162,|160,142,125,120,|139,147,154,165,|
| 33,130,216,224,|173,162,150,150,|125,120,116,116,|154,165,176,176,|
| 33,130,216,224,|150,150,150,150,|116,116,116,116,|176,176,176,176,|
+----------------+----------------+----------------+----------------+
可见，img->cof存的正是解码端的DCT系数，而img->mpr存得正是预测值.
在之前的博文中已经介绍了根据公式生硬地实现反DCT的过程，并给出了相应的代码。在JM8.6解码器中虽然遵照了同样的公式，但是实现方式却有所不同，采用了蝶形运算法，结果相同，但效果更好。可以打这样一个简单的比方，根据公式直接实现类似于DFT, 而采用蝶形运算则类似于FFT.
（题外话：分析问题，解决问题要抓住问题的本质和关键. 在此，不得不承认，蝶形算法跟我所研究的方向关系不大，所以不需要具体管蝶形运算是如何实现的，我要的是运算的结果; 同理，运动估计在H.264中是最重要的，但是依然与我的研究方向关系不大，所以，不需要清楚地知道运动矢量到底是怎么估计出来的，我需要知道的是，估计出来的运动矢量在哪里，怎么提取.）
下面来模拟itrans函数的功能，同时提取残差，代码如下：
#include<iostream>
#define DQ_BITS 6
#define DQ_ROUND 32
#define BLOCK_SIZE 4
using namespace std;
// 解码端的DCT系数
int cof[BLOCK_SIZE][BLOCK_SIZE] =
{
2304, -3840, -2816, -1600,
960, -1200, 320, 0,
768, -320, -512, 320,
0, 0, 0, 0
};
// 预测值
int mpr[BLOCK_SIZE][BLOCK_SIZE] =
{
128, 128, 128, 128,
128, 128, 128, 128,
128, 128, 128, 128,
128, 128, 128, 128
};
int m5[BLOCK_SIZE]; //中间变量
int m6[BLOCK_SIZE]; //中间变量
int m7[BLOCK_SIZE][BLOCK_SIZE]; // 进行了[0, 255]化的值
int myMat[BLOCK_SIZE][BLOCK_SIZE]; // 未进行[0, 255]化的值
int min(int x, int y)
{
return x < y ? x : y;
}
int max(int x, int y)
{
return x > y ? x : y;
}
//矩阵显示
void matrixShow(int a[][BLOCK_SIZE])
{
int i, j;
cout << "*****************************" << endl;
for(i = 0; i < BLOCK_SIZE; i++)
{
for(j = 0; j < BLOCK_SIZE; j++)
{
cout << a[i][j] << "\t";
}
cout << endl;
}
cout << "*****************************" << endl << endl;
}
//矩阵相减
void matrixSubtract(int a[][BLOCK_SIZE],int b[][BLOCK_SIZE])
{
int i, j;
for(i = 0;i < BLOCK_SIZE; i++)
{
for(j = 0; j < BLOCK_SIZE; j++)
{
a[i][j] -= b[i][j];
}
}
}
//反DCT变换
void itrans()
{
int i, j, i1, j1;
//蝶形算法（不用蝶形算法也可以，但要慢一些）
for (j = 0; j < BLOCK_SIZE; j++)
{
for (i = 0; i < BLOCK_SIZE; i++)
m5[i] = cof[i][j];
m6[0] = m5[0] + m5[2];
m6[1] = m5[0] - m5[2];
m6[2] = (m5[1] >> 1) - m5[3];
m6[3] = m5[1] + (m5[3] >> 1);
for (i = 0; i < 2; i++)
{
i1 = 3 - i;
m7[i][j] = m6[i] + m6[i1];
m7[i1][j] = m6[i] - m6[i1];
}
}// end for (j = 0; j < BLOCK_SIZE; j++)
for (i = 0; i < BLOCK_SIZE; i++)
{
for (j = 0; j < BLOCK_SIZE; j++)
m5[j] = m7[i][j];
m6[0] = m5[0] + m5[2];
m6[1] = m5[0] - m5[2];
m6[2] = (m5[1] >> 1) - m5[3];
m6[3] = m5[1] + (m5[3] >> 1);
for (j = 0; j < 2; j++)
{
j1 = 3 - j;
myMat[i][j] = (m6[j] + m6[j1] + (mpr[i][j] << DQ_BITS) + DQ_ROUND) >> DQ_BITS;
myMat[i][j1] = (m6[j] - m6[j1] + (mpr[i][j1] << DQ_BITS) + DQ_ROUND) >> DQ_BITS;
m7[i][j] = max(0, min(255, (m6[j] + m6[j1] + (mpr[i][j] << DQ_BITS) + DQ_ROUND) >> DQ_BITS));
m7[i][j1] = max(0, min(255, (m6[j] - m6[j1] + (mpr[i][j1] << DQ_BITS) + DQ_ROUND) >> DQ_BITS));
}
}// end for (i = 0; i < BLOCK_SIZE; i++)
}
int main()
{
cout << "解码端的DCT矩阵为:" << endl;
matrixShow(cof);
cout << "预测矩阵为:" << endl;
matrixShow(mpr);
itrans();
cout <<"未进行[0, 255]化的矩阵为:" << endl;
matrixShow(myMat);
cout <<"进行了[0, 255]化的矩阵为:" << endl;
matrixShow(m7);
//下面要用myMat, 而不能用m7
cout <<"残差矩阵为:" << endl;
matrixSubtract(myMat, mpr);
matrixShow(myMat);
return 0;
}
解码端的DCT矩阵为:
*****************************
2304 -3840 -2816 -1600
960 -1200 320 0
768 -320 -512 320
0 0 0 0
*****************************
预测矩阵为:
*****************************
128 128 128 128
128 128 128 128
128 128 128 128
128 128 128 128
*****************************
未进行[0, 255]化的矩阵为:
*****************************
50 216 260 238
47 191 195 205
45 190 176 167
48 215 221 160
*****************************
进行了[0, 255]化的矩阵为:
*****************************
50 216 255 238
47 191 195 205
45 190 176 167
48 215 221 160
*****************************
残差矩阵为:
*****************************
-78 88 132 110
-81 63 67 77
-83 62 48 39
-80 87 93 32
*****************************
而用H.264visa观察的结果为：
====================== Y Data ====================== （进行了[0, 255]化的值）
+----------------+----------------+----------------+----------------+
| 50,216,255,238,|255,252,255,255,|247,255,252,251,|255,255,255,243,|
| 47,191,195,205,|235,206,216,180,|199,204,186,239,|198,212,206,180,|
| 45,190,176,167,|192,173,183,137,|191,183,141,181,|144,177,171,134,|
| 48,215,221,160,|174,184,194,169,|200,206,176,166,|172,194,187,157,|
+----------------+----------------+----------------+----------------+
| 44,183,212,183,|204,224,220,201,|219,219,219,219,|224,224,224,224,|
| 26,139,183,180,|216,228,225,214,|227,227,227,227,|230,230,230,230,|
| 30,123,199,216,|209,217,217,216,|219,219,219,219,|217,217,217,217,|
| 28,127,218,229,|212,218,229,229,|227,227,227,227,|222,222,222,222,|
+----------------+----------------+----------------+----------------+
| 33,130,216,224,|224,224,224,224,|223,223,225,224,|219,218,219,222,|
| 33,130,216,224,|224,224,224,224,|221,220,217,214,|212,219,228,229,|
| 33,130,216,224,|224,224,224,224,|221,219,214,212,|212,226,227,215,|
| 33,130,216,224,|224,224,224,224,|226,211,180,165,|188,200,189,165,|
+----------------+----------------+----------------+----------------+
| 29,126,216,228,|232,228,233,220,|204,182,158,108,|152,153,147,150,|
| 35,137,227,234,|228,197,182,187,|166,149,150,146,|130,144,163,180,|
| 33,136,208,196,|168,151,133,133,|141,140,128,132,|149,169,198,207,|
| 27,125,177,150,|141,153,140,116,|122,159,171,176,|209,209,211,211,|
+----------------+----------------+----------------+----------------+
====================== Y Data ====================== （残差值）
+------------------------+------------------------+------------------------+------------------------+
| -78, 88, 132, 110,| 24, 14, 24, 19,| -8, 7, -3, -4,| 16, 31, 48, 51,|
| -81, 63, 67, 77,| 30, 1, 11, -25,| 19, 24, 6, 59,| -12, 20, 32, 10,|
| -83, 62, 48, 39,| 25, 6, 16, -30,| 54, 46, 4, 44,| -30, 7, 5, -32,|
| -80, 87, 93, 32,| 14, 24, 34, 9,| 31, 37, 7, -3,| 6, 28, 21, -9,|
+------------------------+------------------------+------------------------+------------------------+
| -117, 22, 51, 22,| 25, 35, 38, 32,| 18, 18, 18, 18,| 5, 5, 5, 5,|
| -135, -22, 22, 19,| 32, 43, 50, 45,| 13, 13, 13, 13,| 3, 3, 3, 3,|
| -131, -38, 38, 55,| 20, 35, 48, 47,| 3, 3, 3, 3,| -2, -2, -2, -2,|
| -133, -34, 57, 68,| 27, 43, 60, 60,| -2, -2, -2, -2,| -5, -5, -5, -5,|
+------------------------+------------------------+------------------------+------------------------+
| 5, 3, -2, -5,| 0, 0, 0, 0,| -4, -4, -3, -3,| -5, -6, -5, -2,|
| 5, 3, -2, -5,| 0, 0, 0, 0,| -3, -5, -10, -13,| -2, 5, 14, 15,|
| 5, 3, -2, -5,| 0, 0, 0, 0,| -3, -5, -10, -13,| 0, 14, 15, 3,|
| 5, 3, -2, -5,| 0, 0, 0, 0,| 2, -13, -44, -59,| 23, 35, 24, 0,|
+------------------------+------------------------+------------------------+------------------------+
| -4, -4, 0, 4,| 1, 5, 18, 26,| 0, 0, -2, -34,| 25, 20, 8, 3,|
| 2, 7, 11, 10,| 13, 3, 9, 25,| 6, 7, 25, 26,| -9, -3, 9, 15,|
| 0, 6, -8, -28,| -5, -11, -17, -17,| 16, 20, 12, 16,| -5, 4, 22, 31,|
| -6, -5, -39, -74,| -9, 3, -10, -34,| 6, 43, 55, 60,| 33, 33, 35, 35,|
+------------------------+------------------------+------------------------+------------------------+
数据高度一致，结果不言而喻.
最后值得一提的是，在JM8.6的解码器中，共有3处调用到了itrans函数，其中有一处我目前还没有发现程序执行时候真正调用到它. 另外两处分别是进行亮度反DCT变化和色度反DCT变换，具体是这样的：对于一个宏块而言，先进行16次亮度y的变换，再进行4次色度u的变换，再进行4次色度v的变换. 所以，如果只需要提取亮度y的相关数据时，可以在itrans函数里面提取数据时屏蔽掉色度数据，这样就更好.之前我的实验把色度数据也提取出来了，然后又用matlab把色度数据分割掉，这无疑多了两次不必要的操作.