UNITY SPRITES: SPRITERENDERER VS. CANVASRENDERER (UI IMAGE)

本文讲述的是 2Dsprite 和 UI Image 的对比，仅翻译个人认为值得注意部分，并适当加入个人理解，其他内容可自行解读。

While working in a project for one of our clients, I was asked about the difference between sprites (SpriteRenderer) and UI images (CanvasRenderer) in Unity. I didn’t find much information about it so I decided to prepare a presentation in my company to help making it clear. In this post you will find a more detailed version of the original slides I prepared for it. This blog entry will be based on Unity 5.3.4f1.

Sprites are basically semi-transparent textures that are imported as sprites in its texture import settings. They are not directly applied to meshes like textures, but rather on rectangles/polygons (at the end, they are meshes too, so not such of a big difference). Sprites are images that will be rendered in either your 2d/3d scene or in your interface.

1. Usage

It is straightforward to use sprites in Unity. Just drop the desired image (in PNG preferably) in the assets folder and click on it to access the inspector settings. Mark it as sprite (2D and UI) as shown in the screenshot below.

Image import settings

Now it is time for you to decide between using it as a sprite or as an UI Image. But if you are reading this, you might not be sure which method you want. We will describe the differences in the next section; for now we will just quickly outline how to create both of them.

If you want to use a SpriteRenderer, just drop the sprite from the project view into the hierarchy or scene view. The inspector of the new GameObject will look like:

If you wanted to create a UI Image instead, just right click in the hierarchy and create new UI -> Image. That component requires a canvas, so it will be created if you don’t have one yet. At the end, it will look like:

UI Image相比2DSprite，需要一个Canvas父物体，不过创建UI时系统会自动创建一个。

2. Comparison: SpriteRenderer vs CanvasRenderer

When it comes to the hierarchy, you can place sprites wherever you want in your scene. UI Images, on the other hand, have to be inside a canvas (a GameObject having a Canvas component). You can position sprites just like all other objects through its transform, but images will use a RectTransform instead so as to help positioning the image in your interface system.

Sprites are rendered in the transparent geometry queue (unless you use another material than the default). UI Images are also rendered in the transparent geometry queue (Render.TransparentGeometry), unless you use the overlay rendering mode in which case it will be rendered through Canvas.RenderOverlay. As you might have guessed, it is relatively expensive to draw them on mobile. We will explain later why.

2DSprite的默认材质在透明几何体队列渲染。UI Image也是，除非overlayer渲染模式，就会通过Canvas.RenderOverlay渲染，这在手机来说会相对费时。

One of the key differences between sprites and images is that sprites support the automatic creation of meshes out of it, where UI images consist always of rectangles. The reason for creating meshes will be explained in the next section; we will see how important it is, as it has an important performance impact.

最关键的不同点是sprite的会根据图形创建多个的网格用于裁剪透明部分，而UI Image只会是矩形。

Lastly, both can be used with sprite atlases in order to reduce draw calls. That’s good.

It might help seeing the differences with concrete examples.

Example 1

Check the differences between both methods in example 1. UI Image created a tight rectangle that envelops the sprite, whereas the SpriteRenderer created a mesh that better fits the sprite we are rendering. Let’s check another example:

Example 2

Likewise, the same happened in example 2. But the mesh looks much more complicated now, why is that? Unity tries to fit the sprite the best it cans without introducing too many polygons, so that is the result we get. One could argue if the trade-off is beneficial or not.

以多个网格来表示裁剪透明后的2Dsprite，需要根据自身实际情况权衡对性能的消耗是提升还是降低，这对最终的决策很重要。

And what happens now, if we import a PNG with islands, e.g. an image containing different figures separated by transparent areas?

Example 3

Very interesting results in the third example. SpriteRenderer creates two submeshes, one per island; however, UI Image extends the rect so it covers the whole image. Anyway, is this important?

3. Performance

You guessed it, it’s closely related to performance. It does make a huge difference if you are rendering many of them (like grass in a terrain, or particles). So let us analyse the reasons for it.

Whenever you render a texture, you need to send a command to the GPU driver. You set some information, like the vertices, indices, uv coordinates, texture data and shader parameters and then you make the famous draw call. Afterwards, some nasty things happen in the GPU before the final image is displayed in your screen. A (really) simplified rendering pipeline consists of the following:

1. CPU sends a draw command to the GPU.
2. The GPU gets all the information it needs for drawing (like copying the textures to its VRAM, if it’s got one).
3. The geometries sent are transformed into pixels through vertex shaders and the rasterizer.
4. Every pixel is transformed through fragment shaders and written one or several times into the frame buffer.
5. When the frame is completed, the image is displayed in your screen so you can have fun playing.

So back to the topic, what is the difference between SpriteRenderers and UI Images? It might seem that sprites are more expensive since its geometry is more complex. But what if I told you that vertex operations are normally cheaper than fragment operations? That is especially true for mobile and semi-transparent objects. But why?

虽然2Dsprite三角形多，但是在GPU中顶点操作比片元操作消耗小很多。

In many engines, including Unity, semi-transparent materials are rendered back-to-front. That means, the farthest object (from the camera) are always drawn first so the alpha blending operations work as expected. For opaque materials it is just the opposite so we can cull objects that are not visible.

包括unity的大多数引擎中，半透明材质的渲染都是从后往前，透明混合处理才会正确。而不透明材质是从前往后渲染，于是可以剔除被遮挡的物体。

Pixel shaders will be executed for every (visible) pixel of the sprite you are rendering, so if you have a huge sprite (in relation to the screen size), then you will execute that fragment shader on many pixels. The problem with transparent objects is that you can not effectively cull them if they lie in your camera frustrum, so basically you are rendering ALL semi-transparent objects even if many of them will not be visible at the end. So you find yourself rendering many times the same pixels and overwriting them the whole time in the frame buffer. This issue is commonly known as overdraw. And yes, it is a problem since it wastes memory bandwith and you will quickly reach the pixel fillrate limit of your GPU, something you can’t allow if you are targeting mobile. That is the key point to be understood.

如果整个精灵很大尺寸（相对于屏幕），片元着色器要对很多像素执行，而像素着色器根据渲染的sprite有多少像素就要执行多少次的。对于透明物体的问题是如果canvas依赖于视觉平截体（绑定相机），那就不可以有效剔除物体。精灵会绘制自身所有的像素，即使会被遮挡住的，然后从后往前，一个个地，所有其他精灵在已经绘制好的精灵上也绘制自身所有的像素，导致某些像素被前面覆盖的物体多次重写，这就是常说的overdraw。这个问题就会浪费内存带宽和容易触及GPU的像素填充率上限，手机端游戏有时会限制更大，这是一个需要理解的关键点。

Now, if you did understand it, you will also figure out why SpriteRenderer and CanvasRenderer are so different. The former creates meshes that eliminates unneeded transparent pixels (therefore avoiding executing expensive fragment shaders and reducing overdraw), while UI Image creates a simple mesh that probably provokes a lot of overdraw. There’s always a balance you have to achieve between having complex geometry or living with more fragment operations.

所以SpriteRenderer和CanvasRenderer的区别在于，spriterenderer是生成更复杂的几何体来排除透明像素的绘制，canvasrenderer是利用简单的几何体并带有更多的片元操作，这就是使用者衡量两者如何取决的地方。

You should always consider using sprite atlases, since sprites are normally large in number but small in size. That leads to having so many draw calls as sprites, and that is not good. Also, image compression works better if you have larger images.

Reducing draw calls through sprite atlas

You can easily create sprite atlas through the Atlas Packer. Still, sometimes the automatically created meshes are not optimal and you have no control over it, so you might consider using more advanced software like ShoeBox or TexturePacker.

图片可以考虑打包在一起来降低drawcall，如果很大还可以考虑压缩图片。这些方法简单但是未必令人满意，可以使用ShoeBox或者TexturePacker等软件。

4. Conclusion

The next time you work with sprites, think about the following:

1. If you only have a few sprites, use whatever you prefer. If you have hundreds of them, reread this post.
2. Use the profiler and the frame debugger to know what is going on.
3. Avoid using transparency. Try replacing sprites with opaque alternatives if your budget allows it.
4. Avoid rendering big sprites on screen, as more overdraw will happen. You can easily check for overdraw issues in the scene view by selecting Overdraw as rendering mode. This is critical for particle systems.
5. Prefer having a more complicated geometry than a lot of pixels, especially on mobile. Check the result in the scene view by rendering it as shaded wireframe.
6. If you need the interface positioning helpers (like content fitter, vertical groups, etc.) go for UI Images.
7. To know if you are hitting the pixel fillrate limits, check if performance substantially improves by reducing the resolution of your rendered area.

1. 如果只有一些精灵，随便你用。如果有好几百，重读本文。
2. 使用profiler和frame debugger来了解发生了什么事。
3. 避免使用透明，尝试使用不透明物体代替。
4. 避免在屏幕上渲染大尺寸精灵，这会产生很多overdraw。可以在scene视图点overdraw渲染模式查看（视图左上角，默认shaded），这对粒子系统来说很关键。
5. 个人更倾向于更复杂的几何体代替更多的像素点，特别是在手机上。Scene视图Shaded WireFrame模式查看。
6. 如果需要界面布置助手（比如内容尺寸适配、垂直布局、等等），选择UI Image。
7. 了解是否触及像素填充率上限，检查通过降低渲染区域的分辨率后性能是否整体有提升。

Let me know if you have questions or would like to point out a mistake. Any other appreciation is also welcome.

原文：https://rubentorresbonet.wordpress.com/2016/05/26/unity-sprites-spriterenderer-vs-canvasrenderer-ui-image/