Virtualization is, by nature, extraordinarily complex, especially so on x86 hardware. Understanding the VirtualBox source code therefore requires, at least for some components, a great deal of understanding about the details of the x86 architecture as well as great knowledge about the implementations of the host and guest platforms involved.
There are several ways in which to approach how VirtualBox works. This document shall describe them in order of increasing complexity.
When you start the VirtualBox graphical user interface (GUI), at least one extra process gets started along the way -- the VirtualBox "service" process VBoxSVC.
Once you start a virtual machine (VM) from the GUI, you have two windows (the main window and the VM), but three processes running. Looking at your system from Task Manager (on Windows) or some system monitor (on Linux), you will see these:
(On Linux, there's another daemon process called VBoxXPCOMIPCD which is necessary for our XPCOM implementation to work. We will ignore this for now; see COM-XPCOM interoperability? for details.)
To the host operating system (OS), the VM that runs "inside" the second window looks like an ordinary program. VirtualBox is very well behaved in that respect: it pretty much takes over control over a large part of your computer, executing a complete OS with its own set of guest processes, drivers, and devices inside this VM process, but the host OS does not notice much of this. Whatever the VM does, it's just another process in your host OS.
We therefore have two sorts of encapsulation in place with the various VirtualBox files and processes:
This is why the service process (VBoxSVC) is needed: it keeps track of which VMs are running and what state they're in.
In fact, VirtualBox already comes with several frontends:
As said above, from the perspective of the host OS, a virtual machine is just another process. The host OS does not need much tweaking to support virtualization. Even though there is a ring-0 driver that must be loaded in the host OS for VirtualBox to work, this ring-0 driver does less than you may think. It is only needed for a few specific tasks, such as:
Most importantly, the host's ring-0 driver does not mess with your OS's scheduling or process management. The entire guest OS, including its own hundreds of processes, is only scheduled when the host OS gives the VM process a timeslice.
After a VM has been started, from your processor's point of view, your computer can be in one of several states (the following will require a good understanding of the x86 ring architecture):
Also, in the VirtualBox source code, you will find lots of references to "host context" or "guest context". Essentially, these mean:
With its latest processors, Intel has introduced hardware virtualization support, which they call "Vanderpool", "IVT", "VT-x", or "VMX" (for "virtual machine extensions"). As we started out rather early on this, we internally use the term "VMX". A thorough explanation of this architecture can be found on Intel's pages, but as a summary, with these extensions, a processor always operates in one of the following two modes:
One notable novelty is that all four privilege levels (rings) are supported in either mode, so guest software can theoretically run at any of them. VT-x then defines transitions from root to non-root mode (and vice versa) and calls these "VM entry" and "VM exit".
In non-root mode, the processor will automatically cause VM exits for certain privileged instructions and events. For some of these instructions, it is even configurable whether VM exits should occur.
Since, however, nearly all operating systems in use today only make use of ring-0 and ring-3, and since a lot of operations in non-root mode are very expensive, VirtualBox does not use VT-x exactly as intended by Intel. Instead, we make partial use of it -- only where it makes sense and where it helps us to improve performance. So, as said above, our hypervisor, on non-VT-x machines, lives in ring 0 of the guest context, below the guest ring-0 code that is actually run in ring 1. When VT-x is enabled, the hypervisor can safely live in ring 0 host context and gets activated automatically by use of the new VM exits.
We also have experimental support for AMD's equivalent to VT-x (called AMD-V or SVM). As you have read above, VT-x support is not of high practical importance and we have noticed that our implementation of AMD-V is currently even slower than VT-x. Over time we plan to improve it but it's not our top priority right now.
As described above, we normally try to execute all guest code natively and use the recompiler as a fallback only in very rare situations. For raw ring 3, the performance penalty caused by the recompiler is not a major problem as the number of faults is generally low (unless the guest allows port I/O from ring 3, something we cannot do as we don't want the guest to be able to access real ports).
However, as was also described previously, we manipulate the guest operating system to actually execute its ring-0 code in ring 1. This causes a lot of additional instruction faults, as ring 1 is not allowed to execute any privileged instructions (of which there are plenty in the guest's ring-0 code, of course). With each of these faults, our VMM must step in and emulate the code to achieve the desired behavior. While this normally works perfectly well, the resulting performance would be very poor since CPU faults tend to be very expensive and there will be thousands and thousands of them per second.
To make things worse, running ring-0 code in ring 1 causes some nasty occasional compatibility problems. Because of design flaws in the IA32/AMD64 architecture that were never addressed, some system instructions that should cause faults when called in ring 1 unfortunately do not. Instead, they just behave differently. It is therefore imperative that these instructions be found and replaced.
To address these two issues, we have come up with a set of unique techniques that we call "Patch Manager" (PATM) and "Code Scanning and Analysis Manager" (CSAM). Before executing ring 0 code, we scan it recursively to discover problematic instructions. We then perform in-situ patching, i.e. we replace the instruction with a jump to hypervisor memory where an integrated code generator has placed a more suitable implementation. In reality, this is a very complex task as there are lots of odd situations to be discovered and handled correctly. So, with its current complexity, one could argue that PATM is an advanced in-situ recompiler.
In addition, every time a fault occurs, we analyze the fault's cause to determine if it is possible to patch the offending code to prevent it from causing more expensive faults in the future. This turns out to work very well, and we can reduce the faults caused by our virtualization to a rate that performs much better than a typical recompiler, or even VT-x technology, for that matter.