IOS opengl es2.0显示YUV RGB

简介

opengl es (OpenGL for Embedded Systems)是OpenGL针对嵌入式系统的图形API. Android, IOS, 以及PC, 都支持这个规范.
opengl es 2.0 API文档
https://www.khronos.org/registry/OpenGL-Refpages/es2.0/
opengl es 2.0 规范
https://www.khronos.org/registry/OpenGL/specs/es/2.0/es_full_spec_2.0.pdf
The OpenGL ES Shading Language
https://www.khronos.org/files/opengles_shading_language.pdf
这是我写的一个简单的Shadinge Language教程
https://blog.csdn.net/wzj_whut/article/details/85225179

opengl es能做什么? 使用硬件来进行3D图形处理.
3D图形的表现形式, 应该在一些科幻电影里都见过, 在三个维坐标系统中, 用一堆三角形, 来表达一个立体物体, 再给每个三角形贴上贴图, 就可以表现出具有细节的事物.

在进行视频开发时, 主要用于yuv数据的显示, 因为H.264解码后的数据是yuv.

#编写shader

static const char gVertexShader[] = {
    "attribute vec4 vertexIn;    \n"
    "attribute vec2 textureIn;   \n"
    "varying vec2 textureOut;    \n"
    "void main(void)             \n"
    "{                           \n"
    "    gl_Position = vertexIn; \n"
    "    textureOut = textureIn; \n"
    "}                           \n"
};


static const char gFragmentShader[] = {
    "#ifdef GL_ES\n"
    "precision mediump int;\n"
    "precision mediump float;\n"
    "#endif\n"
    "varying vec2 textureOut;\n"
    "uniform sampler2D tex_y;\n"
    "uniform sampler2D tex_u;\n"
    "uniform sampler2D tex_v;\n"
    "void main(void) {\n"
    "    vec3 yuv;\n"
    "    vec3 rgb;\n"
    "    yuv.x = texture2D(tex_y, textureOut).r;\n"
    "    yuv.y = texture2D(tex_u, textureOut).r - 0.5;\n"
    "    yuv.z = texture2D(tex_v, textureOut).r - 0.5;\n"
    "    rgb = mat3( 1,       1,         1,\n"
    "                0,       -0.39465,  2.03211,\n"
    "                1.13983, -0.58060,  0) * yuv;\n"
    "    		gl_FragColor = vec4(rgb, 1.0);\n"
    "}\n"
};

上面这个yuv转rgb的算法, 我也不知道当初从哪来拷过来的, 看起来很诡异, 但是能正常工作, 也没有出现偏色, 我估摸着应该是求的近似解. 正常的写法应该是这样的

static const char gFragmentShader[] = {
    "#ifdef GL_ES\n"
    "precision mediump int;\n"
    "precision mediump float;\n"
    "#endif\n"
    "varying vec2 textureOut;\n"
    "uniform sampler2D tex_y;\n"
    "uniform sampler2D tex_u;\n"
    "uniform sampler2D tex_v;\n"
    "void main(void) {\n"
    "    float y,u,v,r,g,b;\n"
    "    y = texture2D(tex_y, textureOut).r;\n"
    "    u = texture2D(tex_u, textureOut).r - 0.5;\n"
    "    v = texture2D(tex_v, textureOut).r - 0.5;\n"
    "    y=1.1643*(y-0.0625);\n"
    "    r=y+1.5958*v;\n"
    "    g=y-0.39173*u-0.81290*v;\n"
    "    b=y+2.017*u;\n"
    "  gl_FragColor=vec4(r,g,b,1.0);\n"
"}\n"
};

遵循的公式为

Y  =      (0.257 * R) + (0.504 * G) + (0.098 * B) + 16
Cb = U = -(0.148 * R) - (0.291 * G) + (0.439 * B) + 128
Cr = V =  (0.439 * R) - (0.368 * G) - (0.071 * B) + 128

R = 1.164*(Y - 16) + 1.596*(V - 128)
B = 1.164*(Y - 16)  + 2.018*(U - 128)
G = 1.164*(Y - 16) - 0.813*(V - 128) - 0.391*(U - 128)

完整示例
https://github.com/wzjwhut/ios-opengles