快速求平方根的算法
Any 3D engine draws it’s power and speed from the mathematical models and implementations within, and trust John Carmack of ID software for using really good hacks. As it turns out, a very interesting hack is used in Quake III to calculate an inverse square root.
Preface
ID software has recently released the source code of Quake III engine with a GPL license. In this article we’ll see Carmack work his black magic to calculate the square root of a floating point number blazingly fast.
Carmack’s Unusual Inverse Square Root
A fast glance at the file game/code/q_math.c reveals many interesting performance hacks. The first thing that pops out is the use of the number 0×5f3759df in the function Q_rsqrt, which calculates the inverse square root of a float and why does this function actually work?
Observe the original function from q_math.c:
float Q_rsqrt( float number )
{
long i;
float x2, y;
const float threehalfs = 1.5F;
x2 = number * 0.5F;
y = number;
i = * ( long * ) &y; // evil floating point bit level hacking
i = 0x5f3759df - ( i >> 1 ); // what the fuck?
y = * ( float * ) &i;
y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration
// y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed
#ifndef Q3_VM
#ifdef __linux__
assert( !isnan(y) ); // bk010122 - FPE?
#endif
#endif
return y;
}
Not only does it work, on some CPU’s Carmack’s Q_rsqrt runs up to 4 times faster than (float)(1.0/sqrt(x), eventhough sqrt() is usually implemented using the FSQRT assembley instruction!
In another file, code/common/cm_trace.c , a neater implementation of the same hack can be found. This time it is used for calculating the square root of a float - sqrt(x). Note that the only real difference is in the return value - instead of returning y, return number*y as the square root:
/*
================
SquareRootFloat
================
*/
float SquareRootFloat(float number) {
long i;
float x, y;
const float f = 1.5F;
x = number * 0.5F;
y = number;
i = * ( long * ) &y;
i = 0x5f3759df - ( i >> 1 );
y = * ( float * ) &i;
y = y * ( f - ( x * y * y ) );
y = y * ( f - ( x * y * y ) );
return number * y;
}
Newton’s Approximation of Roots
The code above implements the well known Newton Approximation of roots [3]. As in most other iterative approximation calculations, The Newton approximation is supposed to be ran in iterations; each iteration enhances the accuracy until enough iterations have been made for reaching the desired accuracy.
The general idea behind Newton’s approximation is that whenever we have a guess y for the value of the square root of a number x, we can perform a simple manipulation to get a better guess (one closer to the actual square root) by averaging y with x/y. For example we can compute the square root of 2 as follows; Suppose the initial guess is 1:
2/1 = 2 ; (2 + 1) / 2 = 1.5 2/1.5 = 1.3333; ( 1.5 + 1.3333 ) / 2 = 1.4167 2/1.4167 = 1.4117; ( 1.4167 + 1.4117 ) / 2 = 1.4142 And so on...
As mentioned before, Newton’s approximation is a well known method of fast root calculations. However, Carmack picked an unusual first guess for the first iteration that enhances the approximation drastically and lowers the number of required iterations in Quake III’s calculations to only one iteration!
A Witchcraft Number
The really interesting aspect of this function is the magic constant 0×5f3759df, used to calculate the initial guess, in :
i = 0x5f3759df - ( i >> 1 );
Hence, dividing the input by 2 and subtracting it from the magic constant. This constant works almost perfectly - Only one Newton approximation iteration is required for a low relative error of 10^-3. . As the code illustrates in the commented out second iteration, this approximation is good enough for the Quake III engine.
It turns out that the magic constant 0×5f3759df is a real mystery. In the article “Fast Inverse Square Root” [2] , mathematician Chris Lomont of Purdue University researches the meaning of this magic constant. Using several elaborate techniques, Lomont tries to mathematically calculate this constant himself. The results are very surprising - Lomont’s well calculated mathematically optimal constant turns out to be slighltly different (0×5f37642f) , and in spite of being theoretically better, it yields worse results then the original constant used in the source code!! Indeed, John Carmack must have used genuine black magic to find this number.
Only by numerically searching for another better constant, Lomont found one, which is slightly better that the original. However, in practice the two constants are likely to yield similar results. Lomont proposes this function with the better constant:
float InvSqrt(float x)
{
float xhalf = 0.5f*x;
int i = *(int*)&x; // get bits for floating value
i = 0x5f375a86- (i>>1); // gives initial guess y0
x = *(float*)&i; // convert bits back to float
x = x*(1.5f-xhalf*x*x); // Newton step, repeating increases accuracy
return x;
}
References
- Quake III Source Code
- Fast Inverse Square Root , By Chris Lomont
- Physics Post - Newton’s Approximation of Roots
- General history of John Carmack’s ID software - “The Wizardry of Id” by David Kushner, editor of IEEE’s Spectrum Online magazine
For a better explanation about this, check out http://www.mceniry.net/papers/Fast%20Inverse%20Square%20Root.pdf
Comment by JDT — November 15, 2007 @ 11:40 pm
[…] Root Implementation In Quake III Carmack is a genius. A real one. (tags: programming games code) Last Modified : November 17th, 2007 Filed under : Del.icio.us Navigate : Previous post / Share: […]
Pingback by links for 2007-11-16 -- A Tempest of Thoughts — November 16, 2007 @ 11:30 pm
Could the “magic” number have been found also using an iterative, monte-carlo approach?
Comment by Dean — November 18, 2007 @ 5:53 pm
Nice article, Tomer! FYI, “(i>>1)” is mostly there to divide the floating-point *exponent* by 2, since sqrt(x) is just x^(1/2). We then take “-(i>>1)” to negate the exponent, to compute 1.0/sqrt(x), or x^(-1/2). Finally, the funky constant’s high bits correct the IEEE exponent bias (where to store 2^n, the exponent field actually holds n-127). Chris Lomont’s article gives the gory details on funky constant’s low bits, which improve the Newton iteration’s guess for the mantissa.
Comment by Dr_Lawlor — November 19, 2007 @ 3:36 am
Carmack is really smart but he didn’t come up with this..
Take a look at:
http://www.beyond3d.com/content/articles/8/
Comment by John S. — November 19, 2007 @ 6:17 am
This is incredibly clever, I was reading about this the other day, but Id never even heard of this magic number, I will need to look into this when I get the time!
Comment by peter — November 20, 2007 @ 4:06 pm
Actually, it was not John Carmack who wrote the code. Maybe he wrote the comments ;-) A nice article at http://www.beyond3d.com/content/articles/8/ reveals it’s author is probably Gary Tarolli of 3dfx fame.
Comment by Roman Kopac — November 20, 2007 @ 8:09 pm
[…] rynmonar wrote an interesting post today onHere’s a quick excerpt […]
Pingback by CodeMaestro » Magical Square Root Implementation In Quake III — November 20, 2007 @ 11:51 pm
This neat little function originated from Greg Walsh while at Ardent Computer, not from John Carmack.
Comment by archie4oz — November 21, 2007 @ 6:03 am
I would love to hear Mr Carmacks thoughts on this, how did he come by this, why does it work ?
How could a genetic algorithim improve this constant ?
Comment by mordain — November 21, 2007 @ 6:32 am
AFAIR this is not Carmack’s invention, but rather a guy from MIT or somewhere else. And Quake 3 is not the first Quake to have this code. So check the history, and (as Carmack did) you may find some interesting stuff.
Comment by jdz — November 21, 2007 @ 8:47 am
[…] [link][more] […]
Pingback by pepemosca » Blog Archive » Magical Square Root Implementation In Quake III — November 21, 2007 @ 9:08 am
And just to complete the References, here’s another excellent paper on the topic (probably more interesting than the one by Chris Lomont) http://www.mceniry.net/papers/Fast%20Inverse%20Square%20Root.pdf
Comment by Daniel Mueller — November 21, 2007 @ 10:01 am
The code is elegant but not written by Carmack.
Here is a short piece of the email
>Hi John,
>
>There’s a discussion on Beyond3D.com’s forums about who the author of
>the following is:
Not me, and I don’t think it is Michael. Terje Matheson perhaps?
John Carmack
And here you can find the rest of the search for the origin of this piece of code:
http://www.beyond3d.com/content/articles/8/
Comment by Mythor — November 21, 2007 @ 1:09 pm
links for 2007-11-22…
CodeMaestro » Magical Square Root Implementation In Quake III (tags: programming math) Mac Buyer’s Guide: Know When to Buy Your Mac, iPod or iPhone (tags: mac shopping)……
Trackback by Primordial Ooze — November 22, 2007 @ 2:26 am
Carmack didn’t invent this implementation. It’s an algorithm that has been around for about 20 years; Greg Walsh is suspected of inventing it. See the paper entitled, “Fast Inverse Square Root” by Chris Lomont for details. I think there’s a link to it on this blog; if not, you’ll find it on google for sure.
Comment by Greg Dolley — November 22, 2007 @ 4:37 am
[…] Article link […]
Pingback by Quest3D|Log » Blog Archive » Carmack’ magic constant — November 22, 2007 @ 10:09 am
Is anyone aware of a double precision implementation? Presumably the magic number at the heart of the speed-up would need to be different.
Comment by Graeme E. Smith — November 23, 2007 @ 10:26 am
[…] CodeMaestro: Magical Square Root Implementation in Quake III […]
Pingback by Cornell CS 322 - Intro to Scientific Computing » Blog Archive » Quake3’s fast invSqrt() — January 30, 2008 @ 4:01 am
What the ???
Why can’t someone ask Carmack how he conjured the magic number :)
And why isn’t Newton’s Approximation used in common mathematics?
Comment by ATOzTOA — February 5, 2008 @ 9:26 am
[…] After spending a lot of years writing C code it is sometimes hard to get some of those “tricks” out of your mind. The thing I always felt about C was it seemed to encourage you figure out or use cool tricks. Couple that with graphics coding you can get some strange stuff like the infamous 0×5f3759df q_sqrt function. Even though some of them are fairly worthless in modern computing, they are fun to look at and think about why they work! […]
Pingback by Those old C bit tricks | Oh Null! — March 5, 2008 @ 1:34 am
[…] For those who don’t know, Ken Silverman is the programmer that created the game-engine for Duke Nukem 3D. Silverman received lots of praise from John Carmack - which says a lot because Carmack is co-founder of ID Software and lead developer for games like Doom and Quake (he also wrote what is likely the fastest inverse-square-root function, but that’s a topic for another day). […]
Pingback by PNGout : Ramin Hossaini (blog) — April 10, 2008 @ 10:45 pm