Hot Patching

Well, how frustrating is this... as far as I can tell, .Text completely ate my last post, except for a couple of sentences at the top. I've switched to writing these posts off-line (like a real blogger!), so hopefully that's the last time this will happen to me.

Some co-workers and I were discussing a change you may have noticed in recent Windows kernel binaries. Disassembling into a kernel function shows an odd-looking instruction at the top:

 

   lkd> u NtCreateFile
   nt!NtCreateFile:
   80570d48 8bff             mov     edi,edi
   80570d4a 55               push    ebp
   80570d4b 8bec             mov     ebp,esp
   ...

Notice the 'mov edi,edi' at the top - that would seem to be a creative no-op, and in fact, it is. This seemingly useless instruction is designed to enable a new capability for modern kernels: hot patching. Hot patching is designed to address the availablility requirements of modern servers (and workstations, for that matter), by enabling certain hotfixes to patch the live kernel and redirect function calls from the existing function into a replacement (or potentially a filter function, etc). Administrators need something like this to enable them to apply hotfixes in a timely manner, rather than having to wait for a maintenance window. The world is a better place when servers are patched!

Technically, this works by replacing those two extra MOV bytes at the top of a function with a jump instruction. Now, some of you may have noticed that, on x86, two bytes will only get you a short jump, good for 127 bytes in either direction. Clearly this isn't going to do it - you'd have to write the new function into memory that is probably occupied by another function, which is no good. However, a little extra searching reveals this:

 

   lkd> u NtCreateFile-0x10
   nt!NtOpenFile+0x55:
   80570d38 0f841afdffff     je      nt!FsRtlCurrentBatchOplock+0xf7 (80570a58)
   80570d3e e9cd440600       jmp    nt!FsRtlRegisterUncProvider+0x18b (805d5210)
   80570d43 90               nop
   80570d44 90               nop
   80570d45 90               nop
   80570d46 90               nop
   80570d47 90               nop
   nt!NtCreateFile:
   80570d48 8bff             mov     edi,edi
   ...

Well, just what we needed - five extra bytes in which to place a long jump (relative to the current selector). Those five bytes can get patched on-the-fly with the pointer to the real function. Because they're immediately before the MOV, they're in range for our short jump. The hotfix can then write a relative long jump into those NOP bytes that points to the replacement function.

This design is interestng for several reasons. First, Microsoft chose to use the "creative no-op", rather than two real NOP instructions, even though they then used five real NOPs for the long jump. The reason is easy - the MOV is called every time the function is called, whether it is hooked or not (or, more precisely, when it its not hooked). Because you could be talking about a measurable performance impact on high-traffic code paths, those two bytes should get executed as fast as possible, and with minimal side-effects. The mov is just what the doctor ordered - as a single instruction, it should execute faster than two consecutive NOP instructions.

Why didn't Microsoft just put the five NOPs in as the first bytes of the function (rather than using the short jump)? Again, the question is performance - they have optimized for the common case (the unhooked function), rather than the uncommon case. This costs two extra bytes per function statically, but they save you 2/3 of the instructions (assuming you reserved your five bytes with MOV / MOV / NOP).

One other question that came up was why Microsoft didn't just use Detours, or something much like it. Detours relies on disassembly to perform runtime hooks like these, and it winds up overwriting a number of bytes at the begining of the function to make this work. This is difficult, at best - the implemented solution is much more robust. The cost is an extra seven bytes per function, but it's worth it, IMO.

This is an interesting, and instructive, bit of low-level software engineering, where space, performance, and robustness must be balanced.