Google VR开发-Cardboard VR SDK反畸变实现
上一篇文章分析了Cardboard SDK的生命周期设计。
这里我们看下畸变部分的实现。
Cardboard中将畸变这部分封装成了一个Distortion类和DistortionRenderer类。
我们看下Distortion这个类：
private static final float[] DEFAULT_COEFFICIENTS = { 250.0F, 50000.0F };
private float[] mCoefficients;
里面定义了一组默认系数
DEFAULT_COEFFICIENTS，并提供了一个变量mCoefficients
保存实际使用的系数
提供了函数distortionFactor和distort和来计算畸变因子，做畸变及反畸变：
public float distortionFactor(float radius)
{
float rSq = radius * radius;
return 1.0F + mCoefficients[0] * rSq + mCoefficients[1] * rSq * rSq;
}
public float distort(float radius)
{
return radius * distortionFactor(radius);
}
public float distortInverse(float radius)
{
float r0 = radius / 0.9F;
float r1 = radius * 0.9F;
float dr0 = radius - distort(r0);
while (Math.abs(r1 - r0) > 0.0001D) {
float dr1 = radius - distort(r1);
float r2 = r1 - dr1 * ((r1 - r0) / (dr1 - dr0));
r0 = r1;
r1 = r2;
dr0 = dr1;
}
return r1;
}
通过查看对getDistortion（）的调用可以看到哪些地方用到了Distortion类。
一个是DistortionRenderer.onProjectionChanged->DistortionRenderer. createDistortionMesh->CardboardDeviceParams.getDistortion()
另一个是
RendererHelper构造和触发onDrawFrame的时候会调用updateFieldOfView()进而调用CardboardDeviceParams.getDistortion()
前者取得Distortion这个类后用于构造DistortionMesh对象，在其中调用了distortInverse这个反畸变计算函数！
后者会调用Distortion类的distort函数来计算outerAngle，innerAngle，bottomAngle，topAngle等值。！
好了，现在来看下
DistortionMesh对象中具体干了些什么
（1）首先了解下dpi的概念，即每英寸的像素数（Dots Per Inch）。这个值可以通过获取DisplayMetrics这个类来得到：
WindowManager windowManager = (WindowManager)context.getSystemService("window");
DisplayMetrics metrics = new DisplayMetrics();
windowManager.getDefaultDisplay().getRealMetrics(metrics);
比如在Nexus6 上,metrics.xdpi=494.27metrics.ydpi=492.606
（2）根据dpi，可以计算出每个像素多少米,进而与屏幕宽高的像素值相乘（
比如mWidth=2560，mHeight=1440
）计算出
屏幕的宽和高分别是多少米
mXMetersPerPixel = (0.0254F / metrics.xdpi);
mYMetersPerPixel = (0.0254F / metrics.ydpi);
public float getWidthMeters()
{
return mWidth * mXMetersPerPixel;
}
public float getHeightMeters()
{
return mHeight * mYMetersPerPixel;
}
在Nexus6上
screen.getWidthMeters()=0.13155563
screen.getHeightMeters()=0.07425001
（3）有了屏幕的宽高信息，我们再计算一下视场角相关的值
private EyeViewport initViewportForEye(EyeParams eye, float xOffsetM)
{
//获取屏幕属性
ScreenParams screen = mHmd.getScreen();
//获取Cardboard设备属性
CardboardDeviceParams cdp = mHmd.getCardboard();
//计算出眼睛到屏幕的距离0.011+0.037
float eyeToScreenDistanceM = cdp.getEyeToLensDistance() + cdp.getScreenToLensDistance();
//根据视场角算出人眼可见的屏幕区域
float leftM = (float)Math.tan(Math.toRadians(eye.getFov().getLeft())) * eyeToScreenDistanceM;
float rightM = (float)Math.tan(Math.toRadians(eye.getFov().getRight())) * eyeToScreenDistanceM;
float bottomM = (float)Math.tan(Math.toRadians(eye.getFov().getBottom())) * eyeToScreenDistanceM;
float topM = (float)Math.tan(Math.toRadians(eye.getFov().getTop())) * eyeToScreenDistanceM;
EyeViewport vp = new EyeViewport();
//视场偏移量
vp.x = xOffsetM;
vp.y = 0.0F;
//视场的宽
vp.width = (leftM + rightM);
//视场高
vp.height = (bottomM + topM);
//视场左上角坐标
vp.eyeX = (leftM + xOffsetM);
vp.eyeY = bottomM;
//屏幕横向像素数2560/以米为单位的屏幕宽度0.13155563，得到每米像素数19459.447
float xPxPerM = screen.getWidth() / screen.getWidthMeters();
//屏幕纵向像素数1440/以米为单位的屏幕高度0.07425001，得到每米像素数19393.936
float yPxPerM = screen.getHeight() / screen.getHeightMeters();
//最终算出视场左上角的像素坐标和像素宽高
eye.getViewport().x = Math.round(vp.x * xPxPerM);
eye.getViewport().y = Math.round(vp.y * xPxPerM);
eye.getViewport().width = Math.round(vp.width * xPxPerM);
eye.getViewport().height = Math.round(vp.height * xPxPerM);
return vp;
}
上面计算出来：
左右眼视场的上下左右宽度
leftM=0.05015834rightM=0.037965bottomM=0.041869722topM=0.054545455
leftM=0.037965rightM=0.05015834bottomM=0.041869722topM=0.054545455
并计算出视场左上角坐标，偏移量，以及宽高
vp.eyeX=0.05015834vp.eyeY=0.041869722vp.x=0.0vp.y=0.0vp.width=0.08812334vp.height=0.09641518
vp.eyeX=0.12608834vp.eyeY=0.041869722vp.x=0.08812334vp.y=0.0vp.width=0.08812334vp.height=0.09641518
这里数据有问题的地方是，先通过视场角，计算出左眼的视场宽度为leftM+rightM=0.08812334 ，然后右眼的偏移以左眼的最右边为边界。
但这里算出的右眼宽度也为0.08812334，加起来的值大于屏幕宽度0.17624668，大于前面计算出来的screen.getWidthMeters()=0.13155563了。
说明这个算法没有根据屏幕的宽高来调整FOV。理想情况下，应该参数根据根据屏幕宽高来做调整。
然后用这些值来构造DistortionMesh这个类。
好了现在，我们看DistortionMesh构造函数传了些什么参数进来：
EyeParams eye:眼睛相关参数，其中包含mViewport，mFov，还有视角矩阵mEyeTransform
Distortion distortion：畸变相关类，用于计算反畸变参数
screenWidthM ：屏幕宽（米为单位，由getWidthMeters（）计算得到）
screenHeightM :屏幕高（米为单位，由getHeightMeters()计算得到）
xEyeOffsetMScreen ：眼镜中心点离屏幕边沿的横向距离（横屏模式下）
yEyeOffsetMScreen ：眼镜中心点离屏幕边沿的纵向距离（横屏模式下，并出去3cm的空余边界）
textureWidthM ：纹理宽度（其实是左眼视场宽度+右眼视场宽度）
textureHeightM ：纹理高度（左右眼视场高度中较大的那个值）
xEyeOffsetMTexture ：（以米为单位的视场角中心点横向坐标，vp.eyeX ）
yEyeOffsetMTexture ：（以米为单位的视场角中心点纵向坐标，vp.eyeY ）
viewportXMTexture ：（屏幕坐标里，视场左上角，也就是原点的横向像素坐标，vp.x ）
viewportYMTexture ：（屏幕坐标里，视场左上角，也就是原点的纵向像素坐标 ，vp.y）
viewportWidthMTexture ：（视场像素宽，vp.width ）
viewportHeightMTexture ：（视场像素高，vp.height）
现在看下这个类的构造函数：
public DistortionMesh(EyeParams eye, Distortion distortion, float screenWidthM, float screenHeightM, float xEyeOffsetMScreen, float yEyeOffsetMScreen, float textureWidthM, float textureHeightM, float xEyeOffsetMTexture, float yEyeOffsetMTexture, float viewportXMTexture, float viewportYMTexture, float viewportWidthMTexture, float viewportHeightMTexture)
{
float mPerUScreen = screenWidthM;
float mPerVScreen = screenHeightM;
float mPerUTexture = textureWidthM;
float mPerVTexture = textureHeightM;
float[] vertexData = new float[8000];
int vertexOffset = 0;
Log.d(TAG,"screenWidthM="+screenWidthM+"\nscreenHeightM="+screenHeightM+"\nxEyeOffsetMScreen="+xEyeOffsetMScreen+"\nyEyeOffsetMScreen="+yEyeOffsetMScreen+"\ntextureWidthM="+textureWidthM+"\ntextureHeightM="+textureHeightM+"\nxEyeOffsetMTexture="+xEyeOffsetMTexture+"\nyEyeOffsetMTexture="+yEyeOffsetMTexture+"\nviewportXMTexture="+viewportXMTexture+"\nviewportYMTexture="+viewportYMTexture+"\nviewportWidthMTexture="+viewportWidthMTexture+"\nviewportHeightMTexture="+viewportHeightMTexture);
for (int row = 0; row < 40; row++) {
for (int col = 0; col < 40; col++)
{
float uTexture = col / 39.0F * (viewportWidthMTexture / textureWidthM) + viewportXMTexture / textureWidthM;
float vTexture = row / 39.0F * (viewportHeightMTexture / textureHeightM) + viewportYMTexture / textureHeightM;
float xTexture = uTexture * mPerUTexture;
float yTexture = vTexture * mPerVTexture;
float xTextureEye = xTexture - xEyeOffsetMTexture;
float yTextureEye = yTexture - yEyeOffsetMTexture;
float rTexture = (float)Math.sqrt(xTextureEye * xTextureEye + yTextureEye * yTextureEye);
float textureToScreen = rTexture > 0.0F ? distortion.distortInverse(rTexture) / rTexture : 1.0F;
float xScreen = xTextureEye * textureToScreen + xEyeOffsetMScreen;
float yScreen = yTextureEye * textureToScreen + yEyeOffsetMScreen;
float uScreen = xScreen / mPerUScreen;
float vScreen = yScreen / mPerVScreen;
float vignetteSizeMTexture = 0.002F / textureToScreen;
float dxTexture = xTexture - DistortionRenderer.clamp(xTexture, viewportXMTexture + vignetteSizeMTexture, viewportXMTexture + viewportWidthMTexture - vignetteSizeMTexture);
float dyTexture = yTexture - DistortionRenderer.clamp(yTexture, viewportYMTexture + vignetteSizeMTexture, viewportYMTexture + viewportHeightMTexture - vignetteSizeMTexture);
float drTexture = (float)Math.sqrt(dxTexture * dxTexture + dyTexture * dyTexture);
float vignette = 1.0F - DistortionRenderer.clamp(drTexture / vignetteSizeMTexture, 0.0F, 1.0F);
vertexData[(vertexOffset + 0)] = (2.0F * uScreen - 1.0F);
vertexData[(vertexOffset + 1)] = (2.0F * vScreen - 1.0F);
vertexData[(vertexOffset + 2)] = vignette;
vertexData[(vertexOffset + 3)] = uTexture;
vertexData[(vertexOffset + 4)] = vTexture;
vertexOffset += 5;
}
}
nIndices = 3158;
int[] indexData = new int[nIndices];
int indexOffset = 0;
vertexOffset = 0;
for (int row = 0; row < 39; row++) {
if (row > 0) {
indexData[indexOffset] = indexData[(indexOffset - 1)];
indexOffset++;
}
for (int col = 0; col < 40; col++) {
if (col > 0) {
if (row % 2 == 0)
{
vertexOffset++;
}
else {
vertexOffset--;
}
}
indexData[(indexOffset++)] = vertexOffset;
indexData[(indexOffset++)] = (vertexOffset + 40);
}
vertexOffset += 40;
}
FloatBuffer vertexBuffer = ByteBuffer.allocateDirect(vertexData.length * 4).order(ByteOrder.nativeOrder()).asFloatBuffer();
vertexBuffer.put(vertexData).position(0);
IntBuffer indexBuffer = ByteBuffer.allocateDirect(indexData.length * 4).order(ByteOrder.nativeOrder()).asIntBuffer();
indexBuffer.put(indexData).position(0);
int[] bufferIds = new int[2];
GLES20.glGenBuffers(2, bufferIds, 0);
mArrayBufferId = bufferIds[0];
mElementBufferId = bufferIds[1];
GLES20.glBindBuffer(34962, mArrayBufferId);
GLES20.glBufferData(34962, vertexData.length * 4, vertexBuffer, 35044);
GLES20.glBindBuffer(34963, mElementBufferId);
GLES20.glBufferData(34963, indexData.length * 4, indexBuffer, 35044);
GLES20.glBindBuffer(34962, 0);
GLES20.glBindBuffer(34963, 0);
}
构建完这个类之后，还需要创建纹理：
private int createTexture(int width, int height) {
int[] textureIds = new int[1];
GLES20.glGenTextures(1, textureIds, 0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureIds[0]);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGB, width, height, 0,GLES20.GL_RGB, GLES20.GL_UNSIGNED_SHORT_5_6_5, null);
return textureIds[0];
}
private int setupRenderTextureAndRenderbuffer(int width, int height) {
if (mTextureId != -1) {
GLES20.glDeleteTextures(1, new int[] { mTextureId }, 0);
}
if (mRenderbufferId != -1) {
GLES20.glDeleteRenderbuffers(1, new int[] { mRenderbufferId }, 0);
}
if (mFramebufferId != -1) {
GLES20.glDeleteFramebuffers(1, new int[] { mFramebufferId }, 0);
}
mTextureId = createTexture(width, height);
checkGlError("setupRenderTextureAndRenderbuffer: create texture");
int[] renderbufferIds = new int[1];
GLES20.glGenRenderbuffers(1, renderbufferIds, 0);
GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, renderbufferIds[0]);
GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16, width, height);
mRenderbufferId = renderbufferIds[0];
checkGlError("setupRenderTextureAndRenderbuffer: create renderbuffer");
int[] framebufferIds = new int[1];
GLES20.glGenFramebuffers(1, framebufferIds, 0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, framebufferIds[0]);
mFramebufferId = framebufferIds[0];
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, mTextureId, 0);
GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER,
renderbufferIds[0]);
int status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
if (status != GLES20.GL_FRAMEBUFFER_COMPLETE) {
throw new RuntimeException("Framebuffer is not complete: "
+ Integer.toHexString(status));
}
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
return framebufferIds[0];
}
我们看下这个类的函数实现
（1）在构造RendererHelper的时候初始化DistortionRenderer对象
（2）在每次调用RendererHelper.onDrawFrame的时候都会检查
mProjectionChanged，来判定是否要在VR模式下调用
DistortionRenderer.onProjectionChanged
然后在每一帧绘制前后调用beforeDrawFrame和afterDrawFrame
DistortionRenderer.beforeDrawFrame()
public void beforeDrawFrame()
{
GLES20.glGetIntegerv(GLES20.GL_FRAMEBUFFER_BINDING, mOriginalFramebufferId);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mFramebufferId);
}
DistortionRenderer.afterDrawFrame()
public void afterDrawFrame()
{
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, mOriginalFramebufferId.array()[0]);
GLES20.glViewport(0, 0, mHmd.getScreen().getWidth(), mHmd.getScreen().getHeight());
GLES20.glGetIntegerv(GLES20.GL_VIEWPORT, mViewport);
GLES20.glGetIntegerv(GLES20.GL_CULL_FACE, mCullFaceEnabled);
GLES20.glGetIntegerv(GLES20.GL_SCISSOR_TEST, mScissorTestEnabled);
GLES20.glDisable(GLES20.GL_SCISSOR_TEST);
GLES20.glDisable(GLES20.GL_CULL_FACE);
GLES20.glClearColor(0.0F, 0.0F, 0.0F, 1.0F);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
GLES20.glUseProgram(mProgramHolder.program);
GLES20.glEnable(GLES20.GL_SCISSOR_TEST);
GLES20.glScissor(0, 0, mHmd.getScreen().getWidth() / 2, mHmd.getScreen().getHeight());
renderDistortionMesh(mLeftEyeDistortionMesh);
GLES20.glScissor(mHmd.getScreen().getWidth() / 2, 0, mHmd.getScreen().getWidth() / 2, mHmd.getScreen().getHeight());
renderDistortionMesh(mRightEyeDistortionMesh);
GLES20.glDisableVertexAttribArray(mProgramHolder.aPosition);
GLES20.glDisableVertexAttribArray(mProgramHolder.aVignette);
GLES20.glDisableVertexAttribArray(mProgramHolder.aTextureCoord);
GLES20.glUseProgram(0);
GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, 0);
GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, 0);
GLES20.glDisable(GLES20.GL_SCISSOR_TEST);
if (mCullFaceEnabled.array()[0] == 1) {
GLES20.glEnable(GLES20.GL_CULL_FACE);
}
if (mScissorTestEnabled.array()[0] == 1) {
GLES20.glEnable(GLES20.GL_SCISSOR_TEST);
}
GLES20.glViewport(mViewport.array()[0], mViewport.array()[1], mViewport.array()[2], mViewport.array()[3]);
}
DistortionRenderer.onProjectionChanged()
public void onProjectionChanged(HeadMountedDisplay hmd, EyeParams leftEye, EyeParams rightEye, float zNear, float zFar)
{
mHmd = new HeadMountedDisplay(hmd);
mLeftEyeFov = new FieldOfView(leftEye.getFov());
mRightEyeFov = new FieldOfView(rightEye.getFov());
ScreenParams screen = mHmd.getScreen();
CardboardDeviceParams cdp = mHmd.getCardboard();
if (mProgramHolder == null) {
mProgramHolder = createProgramHolder();
}
EyeViewport leftEyeViewport = initViewportForEye(leftEye, 0.0F);
EyeViewport rightEyeViewport = initViewportForEye(rightEye, leftEyeViewport.width);
leftEye.getFov().toPerspectiveMatrix(zNear, zFar, leftEye.getTransform().getPerspective(), 0);
rightEye.getFov().toPerspectiveMatrix(zNear, zFar, rightEye.getTransform().getPerspective(), 0);
float textureWidthM = leftEyeViewport.width + rightEyeViewport.width;
float textureHeightM = Math.max(leftEyeViewport.height, rightEyeViewport.height);
float xPxPerM = screen.getWidth() / screen.getWidthMeters();
float yPxPerM = screen.getHeight() / screen.getHeightMeters();
int textureWidthPx = Math.round(textureWidthM * xPxPerM);
int textureHeightPx = Math.round(textureHeightM * yPxPerM);
float xEyeOffsetMScreen = screen.getWidthMeters() / 2.0F - cdp.getInterpupillaryDistance() / 2.0F;
float yEyeOffsetMScreen = cdp.getVerticalDistanceToLensCenter() - screen.getBorderSizeMeters();
mLeftEyeDistortionMesh = createDistortionMesh(leftEye, leftEyeViewport, textureWidthM, textureHeightM, xEyeOffsetMScreen, yEyeOffsetMScreen);
xEyeOffsetMScreen = screen.getWidthMeters() - xEyeOffsetMScreen;
mRightEyeDistortionMesh = createDistortionMesh(rightEye, rightEyeViewport, textureWidthM, textureHeightM, xEyeOffsetMScreen, yEyeOffsetMScreen);
setupRenderTextureAndRenderbuffer(textureWidthPx, textureHeightPx);
}