ios屏幕录制60帧
It’s been over two years since I last published a blog, so I thought I’d give this another go in 2020 and kick it off by writing about an iOS-related project I’ve been working on over the last couple weeks – a reverse engineering task involving the iOS screen frame-buffer.
自上次发布博客以来已经过去了两年多，所以我认为我会在2020年再试一次，并写一篇关于我在过去几周内一直在从事的与iOS相关的项目的开篇-相反涉及iOS屏幕帧缓冲区的工程任务。
First of all, my inspiration to start looking at this came shortly after the release of the checkra1n jailbreak for the iPhone 5S – iPhone X. If you haven’t yet played with checkra1n, check it out here – https://checkra.in/.
首先，我的灵感开始于在iPhone 5S的checkra1n越狱发布之后– iPhoneX。如果您还没有玩过checkra1n，请在此处查看– https://checkra.in /。
One of the things checkra1n does during the jailbreaking process is display a series of debug messages on the phone’s screen to let the user know about the progress of the jailbreak, and allow them to see what went wrong in the event of the jailbreak failing.
checkra1n在越狱过程中要做的一件事是在手机屏幕上显示一系列调试消息，以使用户了解越狱的进度，并让他们查看在越狱失败时出了什么问题。
These debug messages are written to the display over the top of the standard boot screen. I was intrigued by this and started to wonder how the checkra1n team had implemented it. I heard some phrases thrown around online mentioning the use of a “framebuffer” and that the jailbreak tool was somehow “writing” to it.
这些调试消息将被写入标准引导屏幕顶部的显示中。 我对此很感兴趣，并开始想知道Checkra1n团队是如何实施的。 我在网上听到一些短语，提到使用“帧缓冲区”，而越狱工具正在以某种方式“编写”它。
So I set myself the challenge of figuring out how this worked and how I could replicate this in order render some characters on the screen by directly manipulating the pixels.
因此，我面临挑战，要弄清楚它是如何工作的，以及如何通过直接操作像素来在屏幕上渲染一些字符，从而复制它。
在内核内存中找到iOS帧缓冲区 (Locating the iOS frame-buffer in kernel memory)
The first step towards reaching my goal of writing custom characters to the screen was to find where in memory the screen pixel data was stored.
实现我将自定义字符写入屏幕的目标的第一步是找到屏幕像素数据存储在内存中的位置。
The checkra1n tool manipulates pixels in the framebuffer during the iBoot stage of the boot process, which is why we see the debug messages printed over the Apple boot-logo. I decided to take a different approach and focus on an already-booted iPhone.
checkra1n工具在引导过程的iBoot阶段处理帧缓冲区中的像素，这就是为什么我们看到在Apple引导徽标上打印调试消息的原因。 我决定采用另一种方法，并专注于已经启动的iPhone。
The only difference meant dealing with kernel memory as opposed to iBoot memory, and dealing with the kernel memory seemed like the simpler approach as I could get started right away without having to construct a fully patched boot-chain etc.
唯一的区别是处理内核内存而不是iBoot内存，并且处理内核内存似乎是更简单的方法，因为我可以立即开始而无需构建完整修补的启动链等。
I started digging around to find where the iOS kernel holds the screen pixel data in memory. Knowing this memory location would allow me to write to it (assuming I’m using a jailbroken device with a TFP0 patch) resulting in the colour of the pixels changing.
我开始四处寻找以发现iOS内核将屏幕像素数据保存在内存中的位置。 知道此内存位置后，我将可以对其进行写入(假设我使用的是带有TFP0补丁的越狱设备)，从而导致像素的颜色发生了变化。
My first thought was to see if there was some kind of framebuffer symbol within the iOS kernel.
我的第一个想法是查看iOS内核中是否存在某种framebuffer符号。
I opened a decrypted kernelcache file (for this, I’m using an iPhone3,1 iOS 7.1.2 kernel) in Hopper disassembler and searched for “framebuffer” but no luck :(
我在Hopper反汇编程序中打开了一个解密的kernelcache文件(为此，我使用的是iPhone3,1 iOS 7.1.2内核)，并搜索“ framebuffer”，但没有运气:(
Despite not finding any symbols, Hopper did find a string containing “framebuffer”:
尽管没有找到任何符号，但Hopper确实找到了一个包含“ framebuffer”的字符串：
This string appeared to be some kind of debug message coming from the function initialize_screen. I went to the XNU source code to confirm this function actually existed and found it in the file https://github.com/apple/darwin-xnu/blob/a449c6a3b8014d9406c2ddbdc81795da24aa7443/osfmk/console/video_console.c.
该字符串似乎是某种来自函数initialize_screen的调试消息。 我转到XNU源代码以确认此功能确实存在，并在文件https://github.com/apple/darwin-xnu/blob/a449c6a3b8014d9406c2ddbdc81795da24aa7443/osfmk/console/video_console.c中找到了它。
Looking at the first few lines of code in this function it is clear that this function is responsible for setting up the device’s screen and assigning values for the width, height, colour depth etc.
查看此功能的前几行代码，很明显，此功能负责设置设备的屏幕并为宽度，高度，色深等分配值。
The first argument to this function is a pointer to a PE_Video structure. I went to where this struct was defined (in https://github.com/apple/darwin-xnu/blob/master/pexpert/pexpert/pexpert.h) and found that it holds some information about the host’s screen, including the width, height, colour depth and a base address of where in memory the pixel data actually lies.
该函数的第一个参数是指向PE_Video结构的指针。 我去到了该结构的定义位置(在https://github.com/apple/darwin-xnu/blob/master/pexpert/pexpert/pexpert.h中 )，发现它包含有关主机屏幕的一些信息，包括宽度，高度，颜色深度以及像素数据实际位于内存中的基址。
struct PE_Video {
unsigned long v_baseAddr; /* Base address of video memory */
unsigned long v_rowBytes; /* Number of bytes per pixel row */
unsigned long v_width; /* Width */
unsigned long v_height; /* Height */
unsigned long v_depth; /* Pixel Depth */
unsigned long v_display; /* Text or Graphics */
char v_pixelFormat[64];
unsigned long v_offset; /* offset into video memory to start at */
unsigned long v_length; /* length of video memory (0 for v_rowBytes * v_height) */
unsigned char v_rotate; /* Rotation: 0:normal, 1:right 90, 2:left 180, 3:left 90 */
unsigned char v_scale; /* Scale Factor for both X & Y */
char reserved1[2];
#ifdef __LP64__
long reserved2;
#else
long v_baseAddrHigh;
#endif
};If I could find this structure sitting in the kernel memory I’d be able to follow the v_baseAddr pointer and find the raw pixel data.
如果可以在内核内存中找到该结构，则可以遵循v_baseAddr指针并找到原始像素数据。
I followed the first XREF from the debug string I had found in the iOS kernelcache and landed in the following code:
我遵循了在iOS内核缓存中找到的调试字符串中的第一个XREF，并使用了以下代码：
If this code was part of initialize_screen then I’d have a chance at finding the address of this PE_Video structure in here.
如果此代码是initialize_screen一部分，那么我将有机会在这里找到此PE_Video结构的地址。
There was no symbol for initialize_screen in the binary so I couldn’t tell for sure if the code I had found myself in was actually initialize_screen but a couple of observations between the disassembly and the source code left me pretty confident that it probably was.
二进制文件中没有initialize_screen符号，因此我无法确定自己所找到的代码是否实际上是initialize_screen但是在反汇编和源代码之间的一些观察使我非常有信心。
I did a bit of digging and found out that the PE_Video structure is only used when setting up the screen initially, and that the screen information was actually held somewhere else after the kernel had properly started.
我做了一些挖掘，发现PE_Video结构仅在最初设置屏幕时使用，并且在内核正确启动之后，屏幕信息实际上保留在其他地方。
The values passed in through the PE_Video structure are copied over to a global vc_info structure named vinfo further down in the code.
通过PE_Video结构传递的值将复制到代码中更下方的名为vinfo的全局vc_info结构。
Looking at the definition for vc_info we can see that it is similar to the PE_Video structure in terms of the information it holds about the screen:
查看vc_info的定义，我们可以发现它在有关屏幕的信息方面类似于PE_Video结构：
struct vc_info{
unsigned int v_height; /* pixels */
unsigned int v_width; /* pixels */
unsigned int v_depth;
unsigned int v_rowbytes;
unsigned long v_baseaddr;
unsigned int v_type;
char v_name[32];
uint64_t v_physaddr;
unsigned int v_rows; /* characters */
unsigned int v_columns; /* characters */
unsigned int v_rowscanbytes; /* Actualy number of bytes used for display per row*/
unsigned int v_scale;
unsigned int v_rotate;
unsigned int v_reserved[3];
};The vinfo structure is assigned its values between calls to simple_lock and simple_unlock in initialize_screen.
该vinfo结构调用之间分配其价值simple_lock和simple_unlock在initialize_screen 。
if (vc_progress) {
simple_lock(&vc_progress_lock);
vinfo = new_vinfo;
simple_unlock(&vc_progress_lock);
}Locating this code in the disassembly was relatively easy as we had some pretty clear ‘markers’ to look out for — the lock and unlock functions.
在我们的反汇编中找到此代码相对容易，因为我们需要寻找一些非常清晰的“标记” —锁定和解锁功能。
Here is the disassembly of the above code in the iOS kernel (note: in the kernel version I was using for this, the lock and unlock functions used were actually _lck_spin_lock and _lck_spin_unlock instead of simple_lock and simple_unlock):
这是上面在iOS内核中的代码的分解(请注意：在我为此使用的内核版本中，使用的锁定和解锁函数实际上是_lck_spin_lock和_lck_spin_unlock而不是simple_lock和simple_unlock )：
8011c688 blx _lck_spin_lock
8011c68c ldr r0, [sp, #-0x8 + 56]
8011c68e add.w r1, fp, #0x8
8011c692 str.w r0, [fp]
8011c696 ldr r0, [sp, #-0x8 + 20]
8011c698 str.w r0, [fp, #0x4]
8011c69c ldr r0, [sp, #-0x8 + 52]
8011c69e stm.w r1, {r0, r8, sl}
8011c6a2 ldr r0, [sp, #-0x8 + 24]
8011c6a4 str.w r0, [fp, #0x14]
8011c6a8 movs r0, #0x0
8011c6aa strb.w r0, [fp, #0x18]
8011c6ae ldr r0, [sp, #-0x8 + 28]
8011c6b0 vld1.8 {d16, d17}, [r0]
8011c6b4 add r0, sp, #0x54
8011c6b6 vld1.32 {d18, d19}, [r0]
8011c6ba ldr r0, [sp, #-0x8 + 36]
8011c6bc vst1.8 {d16, d17}, [r0]
8011c6c0 ldr r0, [sp, #-0x8 + 40]
8011c6c2 vst1.8 {d18, d19}, [r0]
8011c6c6 ldr r0, [sp, #-0x8 + 60]
8011c6c8 str.w r0, [fp, #0x3c]
8011c6cc str.w r5, [fp, #0x38]
8011c6d0 ldr r0, [sp, #-0x8 + 88]
8011c6d2 vldr d16, [sp, #-0x8 + 80]
8011c6d6 str.w r0, [fp, #0x48]
8011c6da ldr r0, [sp, #-0x8 + 32]
8011c6dc vstr d16, [fp, #0x40]
8011c6e0 str.w r0, [fp, #0x4c]
8011c6e4 add r0, sp, #0x38
8011c6e6 vld1.32 {d16, d17}, [r0]
8011c6ea ldr r0, [sp, #-0x8 + 44]
8011c6ec vst1.64 {d16, d17}, [r0, #0x80]
8011c6f0 mov r0, r6
8011c6f2 blx _lck_spin_unlockThe series of STR and LDR instructions in the above assembly code are copying each of the values out of the PE_Video struct over to the new vc_info struct.
上面的汇编代码中的STR和LDR指令系列将每个值从PE_Video结构复制到新的vc_info结构。
I decided to go to the same code in Ghidra, in pseudo code view, as it helped display the memory addresses that were being written to a little clearer.
我决定在伪代码视图中使用Ghidra中的相同代码，因为它有助于更清晰地显示正在写入的内存地址。
_lck_spin_lock(&DAT_8037d714);
DAT_8037f260 = uStack104;
DAT_8037f268 = uStack108;
uRam8037f278 = 0;
uRam8037f29c = uStack100;
uRam8037f2b0 = CONCAT44(uStack96,uStack96);
uRam8037f2b8 = CONCAT44(uStack92,uStack92);
DAT_8037f264 = uVar12;
DAT_8037f26c = uVar14;
DAT_8037f270 = iVar15;
uRam8037f274 = uVar11;
uRam8037f298 = uVar7;
_DAT_8037f2a0 = uVar4;
DAT_8037f2a8 = iVar5;
uRam8037f2ac = uVar10;
_lck_spin_unlock(&DAT_8037d714);The address 0x8037f260 is written to first, on the line DAT_8037f260 = uStack104;. This is actually the base address of the vc_info structure in memory. All other addresses written to in the above code are relative to this address.
首先在行DAT_8037f260 = uStack104;地址0x8037f260写入DAT_8037f260 = uStack104; 。 这实际上是内存中vc_info结构的基地址。 上面代码中写入的所有其他地址都与此地址有关。
Taking this address, adding the KASLR slide and reading some bytes from it produced the following output:
使用此地址，添加KASLR幻灯片并从中读取一些字节会产生以下输出：
Billys-N90AP:/var/mobile root# ./kernread 0x9B17F260 0x50
0x9b17f260: 000003c0 00000280 00000020 00000a00 | � �
0x9b17f270: 92e31000 00000000 00000000 ffff0000 | �� ��Nice! This definitely looks like it is the vc_info structure.
真好！ 这肯定看起来像是vc_info结构。
We can confirm that this is the case by comparing the values against the struct definition — the 3c0 and 280 (in hex) should represent the screen height and width in pixels. This checks out as I’m using an iPhone with a 3.5 inch display, so the decimal conversions of these values (960 and 640) are accurate.
通过将值与struct定义进行比较，我们可以确认是这种情况3c0和280 (以十六进制表示)应该代表屏幕的高度和宽度(以像素为单位)。 当我使用的是具有3.5英寸显示屏的iPhone时，这将进行检查，因此这些值( 960和640 )的十进制转换是准确的。
The 5th value in the memory dump is what I was looking for — the v_baseaddr pointer to the start of the raw pixel data. Reading from this memory, we see a bunch of ffffffff values.
内存转储中的第五个值是我要寻找的-指向原始像素数据开始的v_baseaddr指针。 从该内存中读取，我们看到了一堆ffffffff值。
Billys-N90AP:/var/mobile root# ./kernread 0x92e31000 0x100
0x92e31000: ffffffff ffffffff ffffffff ffffffff | ���� ���� ����
0x92e31010: ffffffff ffffffff ffffffff ffffffff | ���� ���� ����
0x92e31020: ffffffff ffffffff ffffffff ffffffff | ���� ���� ����
0x92e31030: ffffffff ffffffff ffffffff ffffffff | ���� ���� ����
0x92e31040: ffffffff ffffffff ffffffff ffffffff | ���� ���� ����This is because at the time of reading the memory, I had an app open on the iPhone’s screen with a white background, so these ffffffff values are representing those white pixels.
这是因为在读取内存时，我在iPhone屏幕上打开了一个带有白色背景的应用程序，因此这些ffffffff值表示这些白色像素。
However, when I read from the same memory when on the home screen, we get a different result:
但是，当我在主屏幕上从相同的内存中读取数据时，会得到不同的结果：
Billys-N90AP:/var/mobile root# ./kernread 0x92e31000 0x100
0x92e31000: ff162341 ff162342 ff172442 ff152241 | �#A �#B �$B �"A
0x92e31010: ff152342 ff162342 ff162341 ff162241 | �#B �#B �#A �"A
0x92e31020: ff172341 ff162241 ff172341 ff162241 | �#A �"A �#A �"A
0x92e31030: ff172240 ff172341 ff172241 ff172140 | �"@ �#A �"A �!@
0x92e31040: ff172141 ff172341 ff16203e ff152041 | �!A �#A � > � AThese values are representing the blues and greens found on the default iOS 7 wallpaper on the SpringBoard.
这些值代表在SpringBoard上默认iOS 7墙纸上找到的蓝色和绿色。
操纵像素 (Manipulating the pixels)
Now that we know exactly where the screen pixel values are held in kernel memory, we can move on to actually manipulating them.
既然我们知道了屏幕像素值在内核内存中的确切位置，我们就可以继续实际操作它们了。
I started by writing a small program that writes some black and white pixels to the screen, just to test that any changes I made to the memory actually affected what was being displayed on the screen.
我首先编写了一个向屏幕上写入一些黑白像素的小程序，目的只是测试我对内存所做的任何更改实际上是否影响了屏幕上显示的内容。
The result looked something like this:
结果看起来像这样：
So now I had confirmed that this was indeed the framebuffer and changes to the memory resulted in changes to what we see on the device’s screen almost immediately.
因此，现在我已经确认这确实是帧缓冲区，对内存的更改几乎立即导致了我们在设备屏幕上看到的内容的更改。
Cool! Now we can look at actually rendering something potentially useful.
凉！ 现在我们来看一下实际渲染潜在有用的东西。
渲染字符 (Rendering characters)
The next step was to figure out how to actually render alpha-numeric characters so we can print our own text strings over the display.
下一步是弄清楚如何实际呈现字母数字字符，以便我们可以在显示屏上打印我们自己的文本字符串。
I did some more digging around the XNU source and found some functions that appeared to be responsible for rendering characters on the screen. One of these functions was vc_render_char.
我在XNU源代码上做了一些进一步的挖掘，发现一些似乎负责在屏幕上呈现字符的函数。 这些功能之一是vc_render_char 。
This function follows an algorithm that renders a specific character on the screen. The font used here is the iso_font which is the same font that is used during the verbose boot mode on iOS and macOS. It is defined as a 256*16 byte array in the file https://github.com/apple/darwin-xnu/blob/master/osfmk/console/iso_font.c.
此功能遵循在屏幕上呈现特定字符的算法。 此处使用的字体是iso_font ，它与在iOS和macOS上的详细启动模式期间使用的字体相同。 它在文件https://github.com/apple/darwin-xnu/blob/master/osfmk/console/iso_font.c中定义为256 * 16字节数组。
The algorithm to render the characters works by taking a desired character — say ‘A’ — and using its ASCII code (65) multiplied by the desired character height in pixels and using it as an index into the iso_font array.
呈现字符的算法通过采用所需字符(例如'A')并将其ASCII码(65)乘以所需字符高度(以像素为单位)并将其用作iso_font数组的索引来iso_font 。
A series of bit-shifts and bitwise ANDs are then performed on the values to produce an output value. Depending on this output value, either a black or white pixel is written to the framebuffer.
然后对这些值执行一系列的移位和按位与运算，以产生输出值。 根据此输出值，将黑色或白色像素写入帧缓冲区。
I wrote some code to test if I could generate a character based on this algorithm, using the iso_font. The output was as follows:
我编写了一些代码，以测试是否可以使用iso_font基于此算法生成字符。 输出如下：
~/Documents/dev ./print_char
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 0 0 0 0 0 1 1
0 0 1 1 1 0 0 1
0 0 1 1 1 0 0 1
0 0 1 1 1 0 0 1
0 0 0 0 0 0 0 1
0 0 1 1 1 0 0 1
0 0 1 1 1 0 0 1
0 0 1 1 1 0 0 1
0 0 1 1 1 0 0 1
0 0 1 1 1 0 0 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
~/Documents/devLooking closely at the above output, you can see that the 0 characters create the shape of a capital A and the 1 characters act as a background.
仔细查看上面的输出，您会看到0字符创建了大写字母A的形状，而1字符则充当了背景。
To translate this to screen pixel data all we have to do is render a white pixel when the value returned is a 0 and a black pixel when the value returned is a 1.
把这种屏幕的像素数据，我们所要做的就是呈现一个白色像素时，返回的值是0和黑色像素时，返回的值是1 。
Final outcome
最终结果
As a result of the above research, I managed write a program that allows me to render arbitrary text strings to the iPhone’s screen by directly modifying the framebuffer pixels!
作为上述研究的结果，我设法编写了一个程序，该程序可以通过直接修改帧缓冲像素来将任意文本字符串渲染到iPhone的屏幕上！
I guess this research doesn’t have any real practical use, but it was a fun project to work on and the end result looks pretty cool ;)
我想这项研究没有任何实际的实际用途，但这是一个有趣的项目，最终结果看起来很酷；)
If you want to try running this code on your own jailbroken device, you can find the code for my fb_write tool here https://github.com/Billy-Ellis/framebuffer_write.
如果要尝试在自己的越狱设备上运行此代码，可以在https://github.com/Billy-Ellis/framebuffer_write上找到我的fb_write工具的代码。
Right now the offsets are hard-coded for the iPhone3,1 running iOS 7.1.2, but you can add your own for other devices you want to try this on.
目前，偏移量已针对运行iOS 7.1.2的iPhone3,1进行了硬编码，但您可以为要尝试使用此功能的其他设备添加自己的偏移量。
Hopefully you enjoyed this post and got something out of it! Feedback is welcome — you can contact me @bellis1000 on Twitter or [email protected] via e-mail.
希望您喜欢这篇文章，并从中受益！ 欢迎提供反馈-您可以在Twitter上通过电子邮件@ bellis1000或通过电子邮件[email protected]与我联系。
Thanks for reading!
谢谢阅读！
翻译自: https://medium.com/@bellis1000/exploring-the-ios-screen-frame-buffer-a-kernel-reversing-experiment-6cbf9847365
ios屏幕录制60帧