由于项目需要,研究了一下xfreerdp的图像更新流程。freerdp中有各种各样的函数指针,还有很多宏定义,具体的函数调用,通过阅读源码很难知道!这时候就需要祭出大杀器:systemtap了。
最最基础我知道每次图像更新调用了xf_client.c的函数:xf_hw_begin_paint,xf_hw_end_paint。
通过侦测libfreerdp/core/rdp.c和xf_client.c的两个函数,获得如下信息:
可以看到程序中有两个线程在读取socket中的数据:31762和6454。
0 gokuapp(6454): << freerdp_check_event_handles
7 gokuapp(6454): << freerdp_check_fds
12 gokuapp(6454): << rdp_check_fds
//rdp_check_fds中会调用transport_check_fds(transport.c),在transport.c中负责数据的接收。
17 gokuapp(6454): << freerdp_shall_disconnect
22 gokuapp(6454): >> freerdp_shall_disconnect
//transport接收完数据后,回调rdp_recv_callback,这个函数负责数据的解析。
70 gokuapp(6454): << rdp_recv_callback
75 gokuapp(6454): << rdp_recv_pdu
//解析到一个fastpath pdu!
79 gokuapp(6454): << rdp_recv_fastpath_pdu
//我们推测具体画的动作在这两个函数之间:<<<<<<<
//begin和end之间又做了些什么呢?
85 gokuapp(6454): << xf_hw_begin_paint
88 gokuapp(6454): >> xf_hw_begin_paint
107 gokuapp(6454): << xf_hw_end_paint
132 gokuapp(6454): >> xf_hw_end_paint
//我们推测具体画的动作在这两个函数之间:>>>>>>
134 gokuapp(6454): >> rdp_recv_fastpath_pdu
135 gokuapp(6454): >> rdp_recv_pdu
137 gokuapp(6454): >> rdp_recv_callback
141 gokuapp(6454): << freerdp_shall_disconnect
144 gokuapp(6454): >> freerdp_shall_disconnect
150 gokuapp(6454): >> rdp_check_fds
152 gokuapp(6454): >> freerdp_check_fds
156 gokuapp(6454): << checkChannelErrorEvent
159 gokuapp(6454): >> checkChannelErrorEvent
161 gokuapp(6454): >> freerdp_check_event_handles
0 gokuapp(31762): << freerdp_check_event_handles
6 gokuapp(31762): << freerdp_check_fds
12 gokuapp(31762): << rdp_check_fds
17 gokuapp(31762): << freerdp_shall_disconnect
23 gokuapp(31762): >> freerdp_shall_disconnect
40 gokuapp(31762): >> rdp_check_fds
41 gokuapp(31762): >> freerdp_check_fds
46 gokuapp(31762): << checkChannelErrorEvent
49 gokuapp(31762): >> checkChannelErrorEvent
50 gokuapp(31762): >> freerdp_check_event_handles
0 gokuapp(31762): << freerdp_shall_disconnect
3 gokuapp(31762): >> freerdp_shall_disconnect
0 gokuapp(31762): << freerdp_focus_required
2 gokuapp(31762): >> freerdp_focus_required
0 gokuapp(31762): << freerdp_get_event_handles
4 gokuapp(31762): << getChannelErrorEventHandle
6 gokuapp(31762): >> getChannelErrorEventHandle
8 gokuapp(31762): >> freerdp_get_event_handles
//上面留下了一个疑问,begin和end之间又做了些什么呢:
int fastpath_recv_updates(rdpFastPath* fastpath, wStream* s)
{
rdpUpdate* update = fastpath->rdp->update;
IFCALL(update->BeginPaint, update->context);
while (Stream_GetRemainingLength(s) >= 3)
{
if (fastpath_recv_update_data(fastpath, s) < 0)
{
WLog_ERR(TAG, "fastpath_recv_update_data() fail");
return -1;
}
}
IFCALL(update->EndPaint, update->context);
return 0;
}
这个函数的调用信息如下:
0 gokuapp(6454): <>fastpath_read_update_header
//然后,根据头的信息,解压数据!
16 gokuapp(6454): <>bulk_compression_level
27 gokuapp(6454): >>bulk_compression_max_size
56 gokuapp(6454): <>metrics_write_bytes
59 gokuapp(6454): >>bulk_decompress
//解析解压后的数据!
63 gokuapp(6454): <>update_read_frame_marker_order
94 gokuapp(6454): >>update_recv_altsec_order
96 gokuapp(6454): >>update_recv_order
98 gokuapp(6454): <>update_read_field_flags
110 gokuapp(6454): <>update_read_coord
118 gokuapp(6454): <>update_read_coord
121 gokuapp(6454): >>update_read_memblt_order
127 gokuapp(6454): >>update_recv_primary_order
128 gokuapp(6454): >>update_recv_order
130 gokuapp(6454): <>update_read_field_flags
139 gokuapp(6454): <>update_read_coord
145 gokuapp(6454): >>update_read_memblt_order
147 gokuapp(6454): >>update_recv_primary_order
148 gokuapp(6454): >>update_recv_order
150 gokuapp(6454): <>update_read_field_flags
160 gokuapp(6454): <>update_read_coord
165 gokuapp(6454): >>update_read_memblt_order
166 gokuapp(6454): >>update_recv_primary_order
168 gokuapp(6454): >>update_recv_order
170 gokuapp(6454): <>update_read_field_flags
179 gokuapp(6454): <>update_read_coord
184 gokuapp(6454): >>update_read_memblt_order
186 gokuapp(6454): >>update_recv_primary_order
187 gokuapp(6454): >>update_recv_order
189 gokuapp(6454): <>update_read_field_flags
198 gokuapp(6454): <>update_read_coord
205 gokuapp(6454): <>update_read_coord
207 gokuapp(6454): >>update_read_memblt_order
209 gokuapp(6454): >>update_recv_primary_order
210 gokuapp(6454): >>update_recv_order
212 gokuapp(6454): <>update_read_field_flags
221 gokuapp(6454): <>update_read_coord
227 gokuapp(6454): >>update_read_memblt_order
228 gokuapp(6454): >>update_recv_primary_order
229 gokuapp(6454): >>update_recv_order
231 gokuapp(6454): <>update_read_field_flags
241 gokuapp(6454): <>update_read_coord
246 gokuapp(6454): >>update_read_memblt_order
247 gokuapp(6454): >>update_recv_primary_order
248 gokuapp(6454): >>update_recv_order
250 gokuapp(6454): <>update_read_field_flags
260 gokuapp(6454): <>update_read_coord
265 gokuapp(6454): >>update_read_memblt_order
267 gokuapp(6454): >>update_recv_primary_order
268 gokuapp(6454): >>update_recv_order
270 gokuapp(6454): <>update_read_field_flags
279 gokuapp(6454): <>update_read_coord
286 gokuapp(6454): <>update_read_coord
288 gokuapp(6454): >>update_read_memblt_order
290 gokuapp(6454): >>update_recv_primary_order
291 gokuapp(6454): >>update_recv_order
293 gokuapp(6454): <>update_read_field_flags
302 gokuapp(6454): <>update_read_coord
308 gokuapp(6454): >>update_read_memblt_order
309 gokuapp(6454): >>update_recv_primary_order
310 gokuapp(6454): >>update_recv_order
312 gokuapp(6454): <>update_read_field_flags
321 gokuapp(6454): <>update_read_coord
327 gokuapp(6454): >>update_read_memblt_order
328 gokuapp(6454): >>update_recv_primary_order
329 gokuapp(6454): >>update_recv_order
331 gokuapp(6454): <>update_read_field_flags
340 gokuapp(6454): <>update_read_coord
346 gokuapp(6454): >>update_read_memblt_order
347 gokuapp(6454): >>update_recv_primary_order
348 gokuapp(6454): >>update_recv_order
350 gokuapp(6454): <>update_read_frame_marker_order
359 gokuapp(6454): >>update_recv_altsec_order
360 gokuapp(6454): >>update_recv_order
361 gokuapp(6454): >>fastpath_recv_orders
363 gokuapp(6454): >>fastpath_recv_update
364 gokuapp(6454): >>fastpath_recv_update_data
385 gokuapp(6454): >>fastpath_recv_updates
上面的内容是典型的rdp gdi协议!
下面我们直接看rdp gdi协议文档内容:
The Remote Desktop Protocol: Graphics Devices Interfaces (GDI) Acceleration Extension is an extension to the Remote Desktop Protocol: Basic Connectivity and Graphics Remoting (as specified in [MS-DPBCGR]). The aim of the Remote Desktop Protocol: GDI Acceleration Extension is to reduce the bandwidth associated with graphics remoting by encoding the drawing operations that produce an image instead of encoding the actual image.
可以看到gdi协议的目的是减少网络带宽的占用,如何减少,用传输drawing operations替代传输编码的图像。所以,gdi协议的大部分内容是传输drawing operations!
一些文档中用到的术语!列出了我认为比较重要的几个:
ARGB: A color space wherein each color is represented as a quad (A, R, G, B), where A represents the alpha (transparency) component, R represents the red component, G represents the green component, and B represents the blue component. The ARGB value is typically stored as a 32-bit integer, wherein the alpha channel is stored in the highest 8 bits and the blue value is storedin the lowest 8 bits.
brush: An 8-by-8-pixel bitmap that is repeated horizontally and vertically to fill an area.
color plane: A two-dimensional surface containing a collection of values that represent a single component of the ARGB or AYCoCg color space.
color space: Any method of representing colors for printing or electronic display.
Packed Encoding Rules (PER): A set of encoding rules for ASN.1 notation, specified in [ITUX691]. These rules enable the identification, extraction, and decoding of data structures.
protocol data unit (PDU): Information that is delivered as a unit among peer entities of a network and that may contain control information, address information, or data. For more information on remote procedure call (RPC)-specific PDUs, see [C706] section 12.
Reverse Polish Notation (RPN): A mathematical notation wherein each operator follows all of its operands. Also known as postfix notation.
screen tearing: A phenomenon in video rendering where a newly rendered frame partially overlaps with a previously rendered frame, creating a torn look as graphical objects do not line up.
具体GDI协议是如何降低带宽的使用,同时又能保证终端的体验效果呢?文档有如下解释:
For example, instead of sending the bitmap image of a filled rectangle from server to client, an orderto render a rectangle at coordinate (X, Y) with a given width, height, and fill color is sent to the client. The client then executes the drawing order to produce the intended graphics result.
In addition to defining how to encode common drawing operations, the Remote Desktop Protocol: GDI Acceleration Extension also facilitates the use of caches to store drawing primitives such as bitmaps, color tables, and characters. The effective use of caching techniques helps to reduce wire traffic by ensuring that items used in multiple drawing operations are sent only once from server to client (retransmission of these items for use in conjunction with future drawing operations is not required after the item has been cached on the client).
GDI协议是发送的编码的drawing order,同时还会发送bitmaps, color tables, characters的cache。所以GDI协议的cache机制很关键!
GDI协议的几个关键部分:
Caches
The Remote Desktop Protocol: GDI Acceleration Extension defines a number of caches that may be
leveraged by clients and servers:
Bitmap Cache: Stores bitmap images.
Color Table Cache: Stores color palettes.
Glyph Cache: Stores character images.
Fragment Cache: Stores collections of glyphs.
Brush Cache: Stores 8-by-8-pixel bitmaps used to fill regions.
Offscreen Bitmap Cache: Stores writable bitmaps.
GDI+ Caches: Used to cache GDI+ 1.1 primitives:
Graphics Cache
Brush Cache
Pen Cache
Image Attributes Cache
Image Cache
NineGrid Bitmap Cache: Stores NineGrid-compliant bitmaps. For more information about nine-grid
bitmaps, see [NINEGRID].Drawing Orders
Drawing orders are used to perform the following operations:
Transport bitmap data
Encode graphics rendering primitives
Manipulate data caches
Manage rendering surfaces
Support application remoting ([MS-RDPERP] section 1.3)
Support desktop composition ([MS-RDPEDC] section 1.3)
There are three classes of drawing orders:
Primary
Secondary
Alternate secondaryBulk Data Compression