Android4.0 SurfaceTexture

source link: http://developer.android.com/reference/android/graphics/SurfaceTexture.html


public class SurfaceTexture


Class Overview：
Captures frames from an image stream as an OpenGL ES texture.
从一个图像流中捕获图像帧作为OpenGL ES纹理。


The image stream may come from either camera preview or video decode. A SurfaceTexture may be used in place of a SurfaceHolder when specifying the output destination of a Camera or MediaPlayer object. Doing so will cause all the frames from the image stream to be sent to the SurfaceTexture object rather than to the device's display. When updateTexImage() is called, the contents of the texture object specified when the SurfaceTexture was created are updated to contain the most recent image from the image stream. This may cause some frames of the stream to be skipped.
图像流可以来自照相机预览或视频解码。当指定Camera或MediaPlayer对象的输出目标时，SurfaceTexture可以取代SurfaceHolder，这样将使得从图像流中得到的所有帧都输出到SurfaceTexture对象，而不是用于在设备上显示。当调用updateTexImage()时，用来创建SurfaceTexture的纹理对象内容被更新为包含图像流中最近的图片。这可能会使得流中的某些帧被跳过。


When sampling from the texture one should first transform the texture coordinates using the matrix queried via getTransformMatrix(float[]). The transform matrix may change each time updateTexImage() is called, so it should be re-queried each time the texture image is updated. This matrix transforms traditional 2D OpenGL ES texture coordinate column vectors of the form (s, t, 0, 1) where s and t are on the inclusive interval [0, 1] to the proper sampling location in the streamed texture. This transform compensates for any properties of the image stream source that cause it to appear different from a traditional OpenGL ES texture. For example, sampling from the bottom left corner of the image can be accomplished by transforming the column vector (0, 0, 0, 1) using the queried matrix, while sampling from the top right corner of the image can be done by transforming (1, 1, 0, 1).
当对纹理进行采样的时候，应该首先使用getTransformMatrix(float[])查询得到的矩阵来变换纹理坐标。每次调用updateTexImage()的时候，可能会导致变换矩阵发生变化，因此在纹理图像更新时需要重新查询。该矩阵将传统的2D OpenGL ES纹理坐标列向量(s,t,0,1)，其中s，t∈[0,1]，变换为纹理中对应的采样位置。该变换补偿了图像流中任何可能导致与传统OpenGL ES纹理有差异的属性。例如，从图像的左下角开始采样，可以通过使用查询得到的矩阵来变换列向量(0,0,0,1)，而从右上角采样可以通过变换(1,1,0,1)来得到。


The texture object uses the GL_TEXTURE_EXTERNAL_OES texture target, which is defined by the GL_OES_EGL_image_external OpenGL ES extension. This limits how the texture may be used. Each time the texture is bound it must be bound to the GL_TEXTURE_EXTERNAL_OES target rather than the GL_TEXTURE_2D target. Additionally, any OpenGL ES 2.0 shader that samples from the texture must declare its use of this extension using, for example, an "#extension GL_OES_EGL_image_external : require" directive. Such shaders must also access the texture using the samplerExternalOES GLSL sampler type.
纹理对象使用GL_TEXTURE_EXTERNAL_OES作为纹理目标，其是OpenGL ES扩展GL_OES_EGL_image_external定义的。这种纹理目标会对纹理的使用方式造成一些限制。每次纹理绑定的时候，都要绑定到GL_TEXTURE_EXTERNAL_OES，而不是GL_TEXTURE_2D。而且，任何需要从纹理中采样的OpenGL ES 2.0 shader都需要声明其对此扩展的使用，例如，使用指令”#extension GL_OES_EGL_image_external:require”。这些shader也必须使用samplerExternalOES采样方式来访问纹理。


SurfaceTexture objects may be created on any thread. updateTexImage() may only be called on the thread with the OpenGL ES context that contains the texture object. The frame-available callback is called on an arbitrary thread, so unless special care is taken updateTexImage() should not be called directly from the callback.
SurfaceTexture对象可以在任何线程里创建。updateTexImage()只能在包含纹理对象的OpenGL ES上下文所在的线程里创建。可以得到帧信息的回调可以在任何线程被调用，因此在没有做必要的保护的情况下，updateTexImage()不应该直接从回调函数中调用。


Summary


Nested Classes
interface SurfaceTexture.OnFrameAvailableListener Callback interface for being notified that a new stream frame is available. 当新的帧流可用时通知该回调接口。
class SurfaceTexture.OutOfResourcesException Exception thrown when a surface couldn't be created or resized 当无法创建surface或调整surface尺寸时，抛出异常。


Public Constructors
SurfaceTexture(int texName)
Construct a new SurfaceTexture to stream images to a given OpenGL texture.
构造新的源自图像流的SurfaceTexture到给定的OpenGL纹理。


Public Methods
long getTimestamp()
Retrieve the timestamp associated with the texture image set by the most recent call to updateTexImage.
提取最近调用的updateTexImage()为纹理图像设置的时间戳。


void getTransformMatrix(float[] mtx)
Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by the most recent call to updateTexImage.
提取最近调用的updateTexImage()为纹理图像设置的4×4的纹理坐标变换矩阵。


void release()
release() frees all the buffers and puts the SurfaceTexture into the 'abandoned' state.
释放所有的缓冲区，将SurfaceTexture置为abandoned状态。


void setDefaultBufferSize(int width, int height)
Set the default size of the image buffers.
设置默认的图像缓冲区大小。


void setOnFrameAvailableListener(SurfaceTexture.OnFrameAvailableListener l)
Register a callback to be invoked when a new image frame becomes available to the SurfaceTexture.
注册一个回调函数，当新一帧图像对SurfaceTexture可用时调用。


void updateTexImage()
Update the texture image to the most recent frame from the image stream.
更新纹理图像为从图像流中提取的最近一帧。


Protected Methods
void finalize()
Invoked when the garbage collector has detected that this instance is no longer reachable.


[Expand]
Inherited Methods
From class java.lang.Object
Public Constructors


public SurfaceTexture (int texName)


Since: API Level 11
Construct a new SurfaceTexture to stream images to a given OpenGL texture.
构造新的源自图像流的SurfaceTexture到给定的OpenGL纹理。


Parameters


texName the OpenGL texture object name (e.g. generated via glGenTextures)
texName：OpenGL纹理对象名字，如通过glGenTextures创建的。
Public Methods


public long getTimestamp ()


Since: API Level 14
Retrieve the timestamp associated with the texture image set by the most recent call to updateTexImage. This timestamp is in nanoseconds, and is normally monotonically increasing. The timestamp should be unaffected by time-of-day adjustments, and for a camera should be strictly monotonic but for a MediaPlayer may be reset when the position is set. The specific meaning and zero point of the timestamp depends on the source providing images to the SurfaceTexture. Unless otherwise specified by the image source, timestamps cannot generally be compared across SurfaceTexture instances, or across multiple program invocations. It is mostly useful for determining time offsets between subsequent frames.
提取最近调用的updateTexImage()为纹理图像设置的时间戳。该时间戳以纳秒为单位，通常是单调递增的。该时间戳不应受日立时间影响，对camera应该是严格单调的，但是对MediaPlayer来说，设置播放位置时可能会重置。时间戳的特殊含义以及零点位置取决于为SurfaceTexture提供图像的源。除非图像源有特殊说明，时间戳不应该在不同的SurfaceTexture实例之间，或多个程序调用之间进行比较。它主要应该用于在后续帧之间决定时间间隔。


public void getTransformMatrix (float[] mtx)


Since: API Level 11
Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by the most recent call to updateTexImage. This transform matrix maps 2D homogeneous texture coordinates of the form (s, t, 0, 1) with s and t in the inclusive range [0, 1] to the texture coordinate that should be used to sample that location from the texture. Sampling the texture outside of the range of this transform is undefined. The matrix is stored in column-major order so that it may be passed directly to OpenGL ES via the glLoadMatrixf or glUniformMatrix4fv functions.
提取最近调用的updateTexImage()为纹理图像设置的4×4的纹理坐标变换矩阵。该变换矩阵将2D的其次纹理坐标(s,t,0,1)，s，t∈[0,1]变换为对应的用于从纹理中采样的纹理坐标。在本变换的范围之外对纹理进行采样时未定义的行为。矩阵列主序存储，可以通过glLoadMatrixf()或glUniformMatrix4fv()函数直接传递给OpenGL ES。


Parameters


mtx the array into which the 4x4 matrix will be stored. The array must have exactly 16 elements.
mtx用于存储4×4矩阵。该矩阵必须是16个元素。


public void release ()
Since: API Level 14
release() frees all the buffers and puts the SurfaceTexture into the 'abandoned' state. Once put in this state the SurfaceTexture can never leave it. When in the 'abandoned' state, all methods of the ISurfaceTexture interface will fail with the NO_INIT error. Note that while calling this method causes all the buffers to be freed from the perspective of the the SurfaceTexture, if there are additional references on the buffers (e.g. if a buffer is referenced by a client or by OpenGL ES as a texture) then those buffer will remain allocated. Always call this method when you are done with SurfaceTexture. Failing to do so may delay resource deallocation for a significant amount of time.
释放所有的缓冲区，将SurfaceTexture置为abandoned状态。一旦进入此状态，SurfaceTexture将不会离开此状态。一旦进入abandoned状态，ISurfaceTexture接口的所有方法都将返回NO_INIT错误。注意：调用此方法时，从SurfaceTexture的角度看，所有的缓冲区都被释放，如果仍然有其他的到缓冲区的引用(如，一缓冲区被客户端引用，或者被OpenGL ES作为纹理引用)，那么该缓冲区将仍存在。当使用完SurfaceTexture之后，一定要调用此方法，否则将会导致资源重新分配被延误很长时间。


public void setDefaultBufferSize (int width, int height)


Since: API Level 15
Set the default size of the image buffers. The image producer may override the buffer size, in which case the producer-set buffer size will be used, not the default size set by this method. Both video and camera based image producers do override the size. This method may be used to set the image size when producing images with Canvas (via lockCanvas(Rect)), or OpenGL ES (via an EGLSurface). The new default buffer size will take effect the next time the image producer requests a buffer to fill. For Canvas this will be the next time lockCanvas(Rect) is called. For OpenGL ES, the EGLSurface should be destroyed (via eglDestroySurface), made not-current (via eglMakeCurrent), and then recreated (via eglCreateWindowSurface) to ensure that the new default size has taken effect. The width and height parameters must be no greater than the minimum of GL_MAX_VIEWPORT_DIMS and GL_MAX_TEXTURE_SIZE (see glGetIntegerv). An error due to invalid dimensions might not be reported until updateTexImage() is called.
设置默认的图像缓冲区大小。图像提供者可能会重写缓冲区大小，这时，将使用新的缓冲区大小，而不是默认大小。基于视频和camera的图像生产者都会重写缓冲区大小。该方法可以用于设置使用Canvas(通过lockCanvas(Rect))来产生图像,或通过OpenGL ES(通过EGLSurface)来产生图像时图像的大小。新的默认缓冲区大小将会在下一次图像生产者请求填充一块缓冲区时生效。对Canvas来说，是下一次调用lockCanvas(Rect)；对OpenGL ES，EGLSurface应该被销毁(通过eglDestroySurface)，变成非当前的(通过eglMakeCurrent)，然后在重新创建(通过eglCreateWindowSurface)，以保证新的默认大小生效。width和height参数值应该不大于GL_MAX_VIEWPORT_DIMS和GL_MAX_TEXTURE_SIZE(参见glGetIntegerv)。不合理的尺寸所引起的错误在updateTexImage()被调用之前不会被报告。


public void setOnFrameAvailableListener (SurfaceTexture.OnFrameAvailableListener l)


Since: API Level 11
Register a callback to be invoked when a new image frame becomes available to the SurfaceTexture. Note that this callback may be called on an arbitrary thread, so it is not safe to call updateTexImage() without first binding the OpenGL ES context to the thread invoking the callback.
注册一个回调函数，当新一帧图像对SurfaceTexture可用时调用。注意：该回调可能在任何线程被调用，因此在事先不绑定OpenGL ES上下文到调用该回调的线程就调用updateTexImage()是不安全的。


public void updateTexImage ()


Since: API Level 11
Update the texture image to the most recent frame from the image stream. This may only be called while the OpenGL ES context that owns the texture is bound to the thread. It will implicitly bind its texture to the GL_TEXTURE_EXTERNAL_OES texture target.
更新纹理图像为从图像流中提取的最近一帧。只有在拥有该纹理的OpenGL ES上下文被绑定到线程之后调用。隐式绑定该纹理到GL_TEXTURE_EXTERNAL_OES纹理目标。
Protected Methods


protected void finalize ()


Since: API Level 11
Invoked when the garbage collector has detected that this instance is no longer reachable. The default implementation does nothing, but this method can be overridden to free resources.


Note that objects that override finalize are significantly more expensive than objects that don't. Finalizers may be run a long time after the object is no longer reachable, depending on memory pressure, so it's a bad idea to rely on them for cleanup. Note also that finalizers are run on a single VM-wide finalizer thread, so doing blocking work in a finalizer is a bad idea. A finalizer is usually only necessary for a class that has a native peer and needs to call a native method to destroy that peer. Even then, it's better to provide an explicit close method (and implement Closeable), and insist that callers manually dispose of instances. This works well for something like files, but less well for something like a BigInteger where typical calling code would have to deal with lots of temporaries. Unfortunately, code that creates lots of temporaries is the worst kind of code from the point of view of the single finalizer thread.


If you must use finalizers, consider at least providing your own ReferenceQueue and having your own thread process that queue.


Unlike constructors, finalizers are not automatically chained. You are responsible for calling super.finalize() yourself.


Uncaught exceptions thrown by finalizers are ignored and do not terminate the finalizer thread. See Effective Java Item 7, "Avoid finalizers" for more.


Throws


Throwable

 

int SurfaceTextureClient::dequeueBuffer(android_native_buffer_t** buffer)

2012-07-31 09:29:08

138int SurfaceTextureClient::dequeueBuffer(android_native_buffer_t** buffer) {
139 LOGV("SurfaceTextureClient::dequeueBuffer");
140 Mutex::Autolock lock(mMutex);
141 int buf = -1;
142 status_t result = mSurfaceTexture->dequeueBuffer(&buf, mReqWidth, mReqHeight,
143 mReqFormat, mReqUsage);

//////从slot里边找一个空闲的.

144 if (result < 0) {
145 LOGV("dequeueBuffer: ISurfaceTexture::dequeueBuffer(%d, %d, %d, %d)"
146 "failed: %d", mReqWidth, mReqHeight, mReqFormat, mReqUsage,
147 result);
148 return result;
149 }
150 sp<GraphicBuffer>& gbuf(mSlots[buf]);
151 if (result & ISurfaceTexture::RELEASE_ALL_BUFFERS) {
152 freeAllBuffers();
153 }
154
155 if ((result & ISurfaceTexture::BUFFER_NEEDS_REALLOCATION) || gbuf == 0) {

///如果size/formate不一样导致需要重新分配或者找到的空闲位置处的为NULL则requestBuffer
////requestBuffer的实现
259status_t SurfaceTexture::requestBuffer(int slot, sp<GraphicBuffer>* buf) {
260 ST_LOGV("requestBuffer: slot=%d", slot);
261 Mutex::Autolock lock(mMutex);
262 if (mAbandoned) {
263 ST_LOGE("requestBuffer: SurfaceTexture has been abandoned!");
264 return NO_INIT;
265 }
266 if (slot < 0 || mBufferCount <= slot) {
267 ST_LOGE("requestBuffer: slot index out of range [0, %d]: %d",
268 mBufferCount, slot);
269 return BAD_VALUE;
270 }
271 mSlots[slot].mRequestBufferCalled = true;
272 *buf = mSlots[slot].mGraphicBuffer;
273 return NO_ERROR;
274}
/////////////////////////////////////

156 result = mSurfaceTexture->requestBuffer(buf, &gbuf);
157 if (result != NO_ERROR) {
158 LOGE("dequeueBuffer: ISurfaceTexture::requestBuffer failed: %d",
159 result);
160 return result;
161 }
162 }
163 *buffer = gbuf.get();
164 return OK;
165}