Amdahl's Law
The 80/20 Rule
In the profiling example, we see that most of the CPU time is spent in a fraction of the total program. This is sometimes called the 80/20 rule—that is, 80% of the CPU time is spent in 20% of the program. These places where the computer spends most of its time are also called hot spots, inner loops, and kernels. This is the code you want to optimize.
Amdahl's Law
The fact that nearly all programs have hot spots is good news. This usually means that a little effort can have large payoffs. The bad news is that once you speed up the hot spots, you will face rapidly diminishing returns on your effort. Amdahl published a famous paper on the limits of parallel processing. He showed that if only part of a program can be sped up with parallel processing, then the sequential part of the program would determine and limit the possible speed up.
Suppose the program really spends 80% of its time in one spot, and suppose you can rewrite this spot to take a negligible amount of time. The program will now execute in 20% of its original time, meaning that it now runs 5 times as fast. So an infinite improvement in the hot spot only gives a 5-fold improvement overall. Getting additional speedup will then require many optimizations throughout the program, each improvement yielding only a small change in overall run time, and it is probable that these improvements won't be worth the effort.
In the profiling example, we see that most of the CPU time is spent in a fraction of the total program. This is sometimes called the 80/20 rule—that is, 80% of the CPU time is spent in 20% of the program. These places where the computer spends most of its time are also called hot spots, inner loops, and kernels. This is the code you want to optimize.
Amdahl's Law
The fact that nearly all programs have hot spots is good news. This usually means that a little effort can have large payoffs. The bad news is that once you speed up the hot spots, you will face rapidly diminishing returns on your effort. Amdahl published a famous paper on the limits of parallel processing. He showed that if only part of a program can be sped up with parallel processing, then the sequential part of the program would determine and limit the possible speed up.
Suppose the program really spends 80% of its time in one spot, and suppose you can rewrite this spot to take a negligible amount of time. The program will now execute in 20% of its original time, meaning that it now runs 5 times as fast. So an infinite improvement in the hot spot only gives a 5-fold improvement overall. Getting additional speedup will then require many optimizations throughout the program, each improvement yielding only a small change in overall run time, and it is probable that these improvements won't be worth the effort.
Knowing how to optimize is an important skill. Knowing when to stop optimizing is also important. You can use profiling results to predict the best possible improvement you can get from any particular optimization. For example, if your program spends 20% of its time in one loop, optimizing that loop can speed up the program at most by 1/(1–0.2) = 1.25—that is, by 25%.
Given:
- , the number of threads of execution,
- , the fraction of the algorithm which is strictly serial,
The time an algorithm takes to finish when being executed on thread(s) of execution corresponds to:
Therefore, the theoretical speedup that can be had by executing a given algorithm on a system capable of executing threads of execution is:
http://en.wikipedia.org/wiki/Amdahl's_law
http://baike.baidu.com/view/1349114.htm