CHAPTER 2: ALGORITHM ANALYSIS
第二章：算法分析
An algorithm is a clearly specified set of simple instructions to be followed to solve a problem. Once an algorithm is given for a problem and decided (somehow) to be correct, an important step is to determine how much in the way of resources, such as time or space, the algorithm will require. An algorithm that solves a problem but requires a year is hardly of any use. Likewise, an algorithm that requires a gigabyte of main memory is not (currently) useful.
一个算法就是一组明确指定的用来解决问题的简单指令。一旦给出一个关于某个问题的算法，并且确定它（在某种程度上）是正确的，一个重要的步骤是确定它要使用多少资源，例如这个算法需要的时间或空间。一个算法可以解决问题，但是需要花费一年的时间，那么这个算法几乎没用。同样的，如果一个算法需要1GB的内存，它（一般来说）也是没用的。
In this chapter, we shall discuss
· How to estimate the time required for a program.
· How to reduce the running time of a program from days or years to fractions of a second.
· The results of careless use of recursion.
· Very efficient algorithms to raise a number to a power and to compute the greatest common
divisor of two numbers.
在这一章我们将会讨论：
· 怎样估计一个程序需要的时间
· 怎样将一个程序的运行时间从几天或者几年减少到不到一秒
· 粗心地使用递归的后果
· 计算一个数的幂值以及计算两个数的最大公约数的非常有效的算法
2.1. Mathematical Background
2.1数学背景
The analysis required to estimate the resource use of an algorithm is generally a theoretical issue, and therefore a formal framework is required. We begin with some mathematical definitions.
这个要求估算一个算法资源使用情况的分析通常是一个理论上的问题，因此需要一个正式的框架。首先我们从一些数学定义开始。
Throughout the book we will use the following four definitions:
DEFINITION: T(n) = O(f(n)) if there are constants c and such thatwhen n .
need-to-insert-img
need-to-insert-img
need-to-insert-img
DEFINITION: T(n) = if there are constants c andsuch that T(n)cg(n) when n.
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
DEFINITION: T(n) =if and only if T(n) = O(h(n)) and T(n) =.
need-to-insert-img
need-to-insert-img
DEFINITION: T(n) = o(p(n)) if T(n) = O(p(n)) and T(n) .
need-to-insert-img
整本书我们都会用到下面四个定义：
定义：T(n) = O(f(n)) 表示存在两个正数常量c和n0，使得当n 时
need-to-insert-img
need-to-insert-img
定义：T(n) = ，表示存在两个正数常量c和n0，使得当n时，T(n)cg(n)
need-to-insert-img
need-to-insert-img
need-to-insert-img
定义：T(n) =，表示同时有T(n) = O(h(n)) 和T(n) =
need-to-insert-img
need-to-insert-img
定义：T(n) = o(p(n))，表示同时有T(n) = O(p(n)) 和T(n)
need-to-insert-img
The idea of these definitions is to establish a relative order among functions. Given two functions, there are usually points where one function is smaller than the other function, so it does not make sense to claim, for instance, f(n) < g(n). Thus, we compare their relative rates of growth. When we apply this to the analysis of algorithms, we shall see why this is the important measure.
这些定义的思想是建立一个函数之间的相对的规则。给定两个函数，总会有一个函数比另一个函数节约，所以它对声明没有任何意义，例如，f(n) < g(n)。因此，我们比较它们的相对增长速度。当我们把这个加到算法分析中的时候，我们就能知道为什么这是重要的方法。
Although 1,000n is larger than for small values of n, grows at a faster rate, and thus will eventually be the larger function. The turning point is n = 1,000 in this case. The first definition says that eventually there is some point past which is always at least as large as T(n), so that if constant factors are ignored, f(n) is at least as big as T(n). In our case, we have T(n) = 1,000n, f(n) = ,= 1,000, and c = 1. We could also use = 10 and c = 100. Thus, we can say that 1,000n = O() (order n-squared). This notation is known as Big-Oh notation. Frequently, instead of saying "order . . . ," one says "Big-Oh . . . ."
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
尽管对于一些小的n值来说，1000n比大，以一个更大的速率增长，因此最终将是更大的函数。在这个例子中拐点就是1000。第一个定义是说，最终总会有一个点，使得
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
T(n)，所以如果忽略常量，f(n) T(n)。我们举的例子中，我们令T(n) = 1,000n，f(n) = ,= 1,000, c = 1.并且已知= 10，c = 100。因此，我们就可以说1,000n = O() 。这个表示法就是著名的大写O表示法。经常性的，有人不说“order”，而是说“Big-Oh”。
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
If we use the traditional inequality operators to compare growth rates, then the first definition says that the growth rate of T(n) is less than or equal to that of f(n). The second definition, T(n) =(pronounced "omega"), says that the growth rate of T(n) is greater than or equal tothat of g(n). The third definition, T(n) =(pronounced "theta"), says that the growth rate of T(n) equals ( = ) the growth rate of h(n). The last definition, T(n) = o(p(n)) (pronounced "little-oh"), says that the growth rate of T(n) is less than (<)the growth rate of p(n). This is different from Big-Oh, because Big-Oh allows the possibility that the growth rates are the same.
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
如果我们使用传统不等式比较增长速度，那么第一个定义就是说T(n)的增长速率小于等于f(n)的增长速率。第二个定义，T(n) =，就是说T（n）的增长速率>=g(n)。第三个定义，T(n) =，就是说T(n)的增长速率等于h（n）。最后一个定义，T(n) = o(p(n)) 就是说T(n)小于p(n)的增长速率。这和大写O不同，因为大写O允许增长速率相同的可能性。
need-to-insert-img
need-to-insert-img
To prove that some function T(n) = O(f(n)), we usually do not apply these definitions formally but instead use a repertoire of known results. In general, this means that a proof (or determination that the assumption is incorrect) is a very simple calculation and should not involve calculus, except in extraordinary circumstances (not likely to occur in an algorithm analysis).
当我说T（n）= O(f(n)时，我们通常不会正式的应用这些定义而是使用一些已知结论的公式。一般而言，这就意味着一个证明（或者证明假设是错误的）是一个非常简单的计算并且不应该使用微积分，除非在特殊的情况下（不可能出现在算法分析中）。
When we say that T(n) = O(f(n)), we are guaranteeing that the function T(n) grows at a rate no faster than f(n); thus f(n) is an upper bound on T(n). Since this implies that , we say that T（n）is a lower bound on f(n).
need-to-insert-img
当我们说T(n) = O(f(n))时，我们确保函数T（n）的增长速度不会比f（n）快；因此f（n）是T（n）的上限。因为这个也可以推理出 ，我们说T（n）是f（n）的下界。
need-to-insert-img
As an example,grows faster than ，so we can say that or . f(n) =and grow at the same rate, so both f(n) = O(g(n)) and are true. When two functions grow at the same rate, then the decision whether or not to signify this with can depend on the particular context. Intuitively, if , then are all technically correct, but obviously the last option is the best answer. Writing says not only that , but also that the result is as good (tight) as possible.
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
举个例子，比增长的快，因此我们可以说或者。f(n) =和增长速率相同，所以f(n) = O(g(n)) 和都是正确的。当两个函数增长速率相同时，是否用表示取决于特殊的上下文环境。直观地，如果，那么在学术上来说都是正确的，但是很明显最后一个是最好的答案。表示不只是，而且结果是尽可能好的。
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
The important things to know are
need-to-insert-img
need-to-insert-img
要知道的重要的事情是：
need-to-insert-img
Figure 2.1 Typical growth rates
RULE 2:
If T(x) is a polynomial of degree n, then
need-to-insert-img
规则二：
如果T(x) 是一个n次多项式，那么
need-to-insert-img
RULE 3:
for any constant k. This tells us that logarithms grow very slowly.
need-to-insert-img
To see that rule 1(a) is correct, note that by definition there exist four constants such that
need-to-insert-img
need-to-insert-img
f(n) for n
need-to-insert-img
need-to-insert-img
and
need-to-insert-img
need-to-insert-img
need-to-insert-img
for n
need-to-insert-img
need-to-insert-img
. Let 。Then, for n
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
and
need-to-insert-img
need-to-insert-img
need-to-insert-img
，so that Let Then,
need-to-insert-img
need-to-insert-img
need-to-insert-img
need-to-insert-img
for c = and n
need-to-insert-img
need-to-insert-img
规则三：
对于任意常数k而言，(logn)^k=O(n)。这就告诉我们,对数运算增长速度很慢。
为了证明规则1(a)部分是正确的，根据定义假设四个常数C1，C2, n1, n2，因此，当n>=n1时T1(n)<=C1f(n),并且当n>=n2时，T2(n)<=C2g(n).令n0=max(n1,n2)。因此，对于n>=n0,T1(n)<=C1f(n),并且T2(n)<=C2g(n),因此T1(n)+T2(n)<=C1f(n)+C2f(n)。令C3=max(C1,C2),最终
T1(n)+T2(n)<=C3f(n)+C3g(n)
<=C3(f(n)+g(n))
<=2C3max(f(n),g(n))
<=Cmax(f(n),g(n))
对于C=2C3 并且n>=n0
Several points are in order. First, it is very bad style to include constants or low-order termsinside a Big-Oh. Do not say T(n) =or T(n) = OIn both cases, the correct form is T(n) =This means that in any analysis that will require a Big-Oh answer, all sorts of shortcuts are possible. Lower-order terms can generally be ignored, and constants can be thrown away. Considerably less precision is required in these cases.
need-to-insert-img
need-to-insert-img
need-to-insert-img
有一些注意点是有顺序的。首先,在大O规则中，包含常量或者低阶项是一个坏风格。请不要写成T(n)=O(2n^2)或T(n)=O(n^2+n)任何一种，正确的形式是T(n)=O(n^2)。这就意味着在任何答案中有大O的分析中,尽可能使用简写。低阶项一般被忽略，并且常量将会被忽略。在哪些情况中一般不要求高精度。
Secondly, we can always determine the relative growth rates of two functions f(n) and g(n) by computing f(n)/g(n), using rule if necessary.
need-to-insert-img
need-to-insert-img
*
第二，如果需要的话，可以通过洛必达法则，我们总是通过计算lim (x->∞)(f(n)/g(n))可以确定两个函数f(n)和g(n)的相对增长速率。
*L'Hôpital's rule states that if lim n ->∞f(n) = ∞and lim n->∞g(n) = ∞, then limn->∞f(n)/g(n) = limn->∞f'(n) / g'(n), where f'(n) and g'(n) are the derivatives of f(n) and g(n), respectively.
洛必达法则就是:如果n趋向于无穷的时候,f(n),g(n)都趋向于无穷,然后lim n->∞时f(n)/g(n)=lim n->∞f’(n)/g’(n)，前提是，f’(n)和g’(n)分别是f(n)和g(n)的导数。
The limit can have four possible values:
l The limit is 0: This means that f(n) = o(g(n)).
l The limit is c!= 0: This means that f(n) =θ(g(n)).
l The limit is : This means that g(n) = o(f(n)).
l The limit oscillates: There is no relation (this will not happen in our context).
这个极限有四种可能的值:
l 极限为零的情况:也就是说f(n)=o(g(n))。
l 极限为非零常数:也就是说f(n)=θ(g(n))。
l 极限是无穷时:也就是说g(n)=o(g(n))。
l 极限震荡间断：他们没关系(这个不会出现在本书中)
Using this method almost always amounts to overkill. Usually the relation between f(n) and g(n) can be derived by simple algebra. For instance, if f(n) = n log n and g(n) = n^1.5, then to decide which of f(n) and g(n) grows faster, one really needs to determine which of log n and n^0.5 grows faster. This is like determining which of log^2 n or n grows faster. This is a simple problem, because it is already known that n grows faster than any power of a log. Thus, g(n) grows faster than f(n).
使用这些方法几乎有些过分。通常f(n)和g(n)的关系都可以通过简单的代数推导出。比如，如果f(n)=nlogn并且g(n) = n^1.5,然后为了确定二者谁增长的更快，真正决定因素是logn和n^0.5谁增长得更快。这就像log^2n和n十岁增长得更快。这是一个简单的问题，因为很明显，n比任何logn的次方增长得都快，因此g(n)比f(n)增长的快。
One stylistic note: It is bad to say f(n) <=O(g(n)), because the inequality is implied by the definition. It is wrong to write f(n)>= O(g(n)), which does not make sense.
一个格式注意事项:f(n)<=O(g(n))是一个坏的习惯，因为这个不等式在定义中已经暗含了。写成f(n)>=O(g(n))也是错误的，因为他没有意义。
2.2. Model
2.2 模型
In order to analyze algorithms in a formal framework, we need a model of computation. Our model is basically a normal computer, in which instructions are executed sequentially. Our model has the standard repertoire of simple instructions, such as addition, multiplication, comparison, and assignment, but, unlike real computers, it takes exactly one time unit to do anything (simple). To be reasonable, we will assume that, like a modern computer, our model has fixed size (say 32-bit) integers and that there are no fancy operations, such as matrix inversion or sorting, that clearly cannot be done in one time unit. We also assume infinite memory.
为了分析在正式框架中的算法，我们需要一个计算模型。我们模型基于一个正常电脑,这个电脑可以按顺序执行指令。我们的模型有标准的简单指令集,比如+,*,-,/,但是,不想真正的电脑，它在一个确定的时间可以做任何简单的事情。为了证明它是合理的，我们假设想一个现代电脑，我们的模型有固定尺寸(一般是32个比特位)的整数并且没有复杂操作，比如矩阵求逆，或排序，这些操作很明显无法再一个单位时间内完成。我们也假设内存无穷大
This model clearly has some weaknesses. Obviously, in real life, not all operations take exactly the same time. In particular, in our model one disk read counts the same as an addition, even though the addition is typically several orders of magnitude faster. Also, by assuming infinite memory, we never worry about page faulting, which can be a real problem, especially for efficient algorithms. This can be a major problem in many applications.
这个模型很明显有缺点。显然，在实际生活中，并非所有的操作需要相同的时间。尤其是，在我们的模型中的磁盘读数据和做加法运算一样，尽管加法运算是典型的多好些个数量级。同时，通过假设的无穷大的内存，我们永远不用担心记录断层,记录单层其实是一个真实的问题，尤其是对于高效算法。这将是很多应用软件的主要问题。
2.3. What to Analyze
2.3.分析什么
The most important resource to analyze is generally the running time. Several factors affect the running time of a program. Some, such as the compiler and computer used, are obviously beyond the scope of any theoretical model, so, although they are important, we cannot deal with them here. The other main factors are the algorithm used and the input to the algorithm.
分析过程中最重要的资源一般是运行时间。有几个因素会影响一个程序运行时间。一些(比如使用的编译器和计算机)是明显的超出了任何理论模型的范围,因此,尽管他们是重要的，但是我们这里不处理它们。其他主要的因素是使用的算法和该算法的输入。
Typically, the size of the input is the main consideration. We define two functions, Tavg(n) and Tworst(n), as the average and worst-case running time, respectively, used by an algorithm on input of size n. Clearly, Tavg(n) <=Tworst(n). If there is more than one input, these functions may have more than one argument
典型的，输入的规模首先值得考虑。我们定义两个函数,Tavg(n)和Tworst(n)，作为运行时间的平均和最坏情况,分别运行该算法在输入规模为n的情况下。很明显，Tavg(n)<=Tworst(n)。如果有多输入的话，那么这两个函数可能有多个参数。
We remark that generally the quantity required is the worst-case time, unless otherwise specified. One reason for this is that it provides a bound for all input, including particularly bad input, that an average-case analysis does not provide. The other reason is that average-case bounds are usually much more difficult to compute. In some instances, the definition of "average" can affect the result. (For instance, what is average input for the following problem?)
我们说通常必须的数量就是最坏的情况的时间，除非另外有说明。另一原因是，他为所有的输入头提供了一个范围，包括特别坏的输入，不包括平均情况的分析。还有一个原因就是，平均情况的范围通常是更难计算得多。在一些例子中，平均的定义会影响结果。(例如，下面的问题平均输入是什么？)
As an example, in the next section, we shall consider the following problem:
我们将考虑接下来的例子:
MAXIMUM SUBSEQUENCE SUM PROBLEM:
Given (possibly negative) integers a1, a2, . . . , an, find the maximum value of . (For convenience, the maximum subsequence sum is 0 if all the integers are negative.)
最大子序列和的问题:
给顶一个(有可能是负数)int值的序列,a1，a2,.......an,找出的最大值.(方便起见,如果所有的int值都是负数，最大子序列和为0)
need-to-insert-img
Example: For input -2, 11, -4, 13, -5, -2, the answer is 20 (a2 through a4).
比如:对于输入-2,11，-4,13，-5.-2.这个答案是20(a2-a4)
This problem is interesting mainly because there are so many algorithms to solve it, and the performance of these algorithms varies drastically. We will discuss four algorithms to solve this problem. The running time on some computer (the exact computer is unimportant) for these algorithms is given in Figure 2.2.
这个问题总体而言是有趣,因为有很多算法可以解决它，并且这些算法的表现完全不同，我们将讨论四种解决这个问题的算法。这些算法在一些电脑(实际电脑不重要)上的运行时间在2.2图中已经给出。
There are several important things worth noting in this table. For a small amount of input, the algorithms all run in a blink of the eye, so if only a small amount of input is expected, it might be silly to expend a great deal of effort to design a clever algorithm. On the other hand, there is a large market these days for rewriting programs that were written five years ago based on a no-longer-valid assumption of small input size. These programs are now too slow, because they used poor algorithms. For large amounts of input, Algorithm 4 is clearly the best choice (although Algorithm 3 is still usable).
这有几个值得注意的事情这张表已经给出。对于一个小的输入规模而言，算法都可以眨眼间完成，因此如果预期的输入仅仅是一个小规模的话，花费很多的努力去设计一个精明的算法有点愚蠢。另一方面，重改五年之前写过的基于不能再运行小规模输入的假设很有市场。这些程序现在看起来运行起来太慢了，因为他们所使用的算法不好。对于大量输入而言，算法4明显是最好的选择(尽管算法3 也还能用)。
Second, the times given do not include the time required to read the input. For Algorithm 4, the time merely to read in the input from a disk is likely to be an order of magnitude larger than the time required to solve the problem. This is typical of many efficient algorithms. Reading the data is generally the bottleneck; once the data are read, the problem can be solved quickly. For inefficient algorithms this is not true, and significant computer resources must be used. Thus, it is important that, where possible, algorithms be efficient enough not to be the bottleneck of a problem.
第二，给出的时间并不包含所要求读取输入信息的过程。对于算法4而言，仅仅是从磁盘读取输入的时间就可能比解决这个问题的时间大一个数量级。这是很多高效算法的典型特征。数据读取速度通常是瓶颈；一旦数据读入了，问题可以很快被解决。对于低效算法而言这并不成立，并且大量的计算机资源将会被使用。因此,算法要尽可能的足够高效以至于不是问题的瓶颈很重要。
need-to-insert-img
Figure 2.2 Running time of several algorithms for maximum subsequence sum (in seconds)
图表2.2 最大子序列综合的几种算法运行(单位:秒)
Figure 2.3 shows the growth rates of the running times of the four algorithms. Even though this graph encompasses only values of n ranging from 10 to 100, the relative growth rates are still evident. Although the graph for Algorithm 3 seems linear, it is easy to verify that it is not, by using a straightedge (or piece of paper). Figure 2.4 shows the performance for larger values. It dramatically illustrates how useless inefficient algorithms are for even moderately large amounts of input.
图表2.3展示了四种算法运行时间的增长速度。尽管这个图表仅包含了n值从10到100区间的增长速度值，相对的增长速度依然很明显。尽管算法3的图表看起来像是线性的，但是通过咫尺(或一张纸)证明它非线性很容易。图表2.4展示了对于大输入它们的表现。它明显说明了废柴算法对于甚至适量大小的输入怎么个废柴法。
need-to-insert-img
图表2.3 ：(n/运行时间(毫秒))各种最大子序列和算法的图
need-to-insert-img
图表2.3 ：(n/运行时间(秒))各种最大子序列和算法的图
2.4. Running Time Calculations
2.4. 运行时间的计算
There are several ways to estimate the running time of a program. The previous table was obtained empirically. If two programs are expected to take similar times, probably the best way to decide which is faster is to code them both up and run them!
有很多种方式可以估算程序的运行时间。之前的表格是根据经验得出的。如果两种算法看起来花费的时间差不多,大概最好的方法确定哪个更快是把他们都编出来并且运行一下。
Generally, there are several algorithmic ideas, and we would like to eliminate the bad ones early, so an analysis is usually required. Furthermore, the ability to do an analysis usually provides insight into designing efficient algorithms. The analysis also generally pinpoints the bottlenecks, which are worth coding carefully.
一般而言，有几个算法思想，我们喜欢先排除低效的一个,因此通常需要分析一下。此外，进行一个分析通常需要设计有效算法的洞察力。分析通常也需要精确查找到瓶颈，瓶颈处需要我们精细地去编码。
To simplify the analysis, we will adopt the convention that there are no particular units of time. Thus, we throw away leading constants. We will also throw away low-order terms, so what we are essentially doing is computing a Big-Oh running time. Since Big-Oh is an upper bound, we must be careful to never underestimate the running time of the program. In effect, the answer provided is a guarantee that the program will terminate within a certain time period. The program may stop earlier than this, but never later.
为了简化分析，我们将采取无特殊时间单位的惯例。我们将忽略主要的常量。我们也会忽略低阶项，因此我们本质上所要做的就是计算大-O的运行时间。因为大-O是一个上界，我们必须小心绝不低估程序的运行时间。实际上，提供的答案是一种程序将在一定的时间范围内结束的保证。程序可能比这个时间更早结束，而不会超过这个。
2.4.1. A Simple Example
2.4.1. 一个简单的例子