普林斯顿算法课Part 1 Week 1 Analysis of Algorithms

这一课讲的是如何预测算法的性能及比较不同的算法。
普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第1张图片

1. Observations

例子:3-SUM
给定N个不同的integer,取三个相加之和为0的有多少种组合。

% more 8ints.txt
8
30 -40 -20 -10 40 0 10 5
% java ThreeSum 8ints.txt
4

存在如下几种组合:

30 -40 10
30 -20 -10
-40 40 0
-10 0 10

1.1 3-SUM: brute-force algorithm

public class ThreeSum
{
    public static int count(int[] a)
    {
        int N = a.length;
        int count = 0;
        for (int i = 0; i < N; i++)
            for (int j = i+1; j < N; j++)
                for (int k = j+1; k < N; k++)
                    if (a[i] + a[j] + a[k] == 0)
                        count++;
        return count;
    }

    public static void main(String[] args)
    {
        int[] a = In.readInts(args[0]);
        StdOut.println(count(a));
    }
}

1.2 度量运行时间

public static void main(String[] args)
{
    int[] a = In.readInts(args[0]);
    Stopwatch stopwatch = new Stopwatch();
    StdOut.println(ThreeSum.count(a));
    double time = stopwatch.elapsedTime();
}

1.3 经验分析:记录不同输入大小所耗时间

N time (seconds)
250 0.0
500 0.0
1,000 0.1
2,000 0.8
4,000 6.4
8,000 51.1
16,000 ?

运行时间与输入大小之间的关系
普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第2张图片

普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第3张图片
由此得到 T(N)=1.006×1010×N2.999 T ( N ) = 1.006 × 10 – 10 × N 2.999

1.4 Doubling hypothesis:快速估计指数b的方法

N time (seconds) ratio lg ratio
250 0.0
500 0.0 4.8 2.3
1,000 0.1 6.9 2.8
2,000 0.8 7.7 2.9
4,000 6.4 8.0 3.0
8,000 51.1 8.0 3.0

T(2N)T(N)=a(2N)baNb=2b T ( 2 N ) T ( N ) = a ( 2 N ) b a N b = 2 b

b=lg(T(2N)T(N)) b = l g ( T ( 2 N ) T ( N ) )

得到b之后可以代入 T(N)=aNb T ( N ) = a N b 求得a。
但注意这种方法无法用来估计存在对数关系的计算复杂度。

2. Mathematical models

总运行时间 = sum of cost × frequency for all operations.
・Need to analyze program to determine set of operations.
・Cost depends on machine, compiler.
・Frequency depends on algorithm, input data.

2.1 例子:1-Sum

How many instructions as a function of input size N ?

int count = 0;
for (int i = 0; i < N; i++)
    if (a[i] == 0)
        count++;
operation frequency
variable declaration 2
assignment statement 2
less than compare N + 1
equal to compare N
array access N
increment N to 2 N

2.2 例子:2-Sum

How many instructions as a function of input size N ?

int count = 0;
for (int i = 0; i < N; i++)
    for (int j = i+1; j < N; j++)
        if (a[i] + a[j] == 0)
            count++;
operation frequency
variable declaration 3
assignment statement 3
less than compare N+1+(N+N1+N2+...+1)=N+1+(N+1)N2=(N+1)(N+2)2 N + 1 + ( N + N − 1 + N − 2 + . . . + 1 ) = N + 1 + ( N + 1 ) ∗ N 2 = ( N + 1 ) ∗ ( N + 2 ) 2
equal to compare N1+N2+...+1=N(N1)2 N − 1 + N − 2 + . . . + 1 = N ∗ ( N − 1 ) 2
array access N1+N2+...+1=N(N1) N − 1 + N − 2 + . . . + 1 = N ∗ ( N − 1 )
increment N+N1+N2+...+1=N+(N+1)(N+2)2toN+N(N1) N + N − 1 + N − 2 + . . . + 1 = N + ( N + 1 ) ∗ ( N + 2 ) 2 t o N + N ∗ ( N − 1 )

然而上面这种计数每一个operation的方式非常麻烦,所以可以采用一些简化操作。

2.3 Simplification 1: cost model

Cost model. Use some basic operation as a proxy for running time
比如这里只看进行了多少次array access操作

2.4 Simplification 2: tilde notation

Estimate running time (or memory) as a function of input size N.
Ignore lower order terms.
- when N is large, terms are negligible
- when N is small, we don’t care
抹掉低阶项

operation frequency tilde notation
variable declaration N + 2 ~ N
assignment statement N + 2 ~ N
less than compare ½ (N + 1) (N + 2) ~ ½ N2
equal to compare ½ N (N − 1) ~ ½ N2
array access N (N − 1) ~ N2
increment ½ N (N − 1) to N (N − 1) ~ ½ N2 to ~ N2

2.5 3-Sum

int count = 0;
for (int i = 0; i < N; i++)
    for (int j = i+1; j < N; j++)
        for (int k = j+1; k < N; k++)
            if (a[i] + a[j] + a[k] == 0)
                count++;

3. Order-of-growth classifications

logN,N,NlogN,N2,N3,2N l o g N , N , N l o g N , N 2 , N 3 , 2 N

给定一个有序的数组,和一个key,在数组中找到这个key的index。

public static int binarySearch(int[] a, int key)
{
    int lo = 0, hi = a.length-1;
    while (lo <= hi)
    {
        int mid = lo + (hi - lo) / 2;
        if (key < a[mid]) hi = mid - 1;
        else if (key > a[mid]) lo = mid + 1;
        else return mid;
    }
    return -1;
}

Binary search uses at most 1+lgN 1 + l g N key compares to search in
a sorted array of size N.

3.2 An N2logN N 2 l o g N algorithm for 3-SUM

前面我们写了一个order of growth是 N3 N 3 的3-Sum算法,因为我们选择遍历N所有的3个的组合,并挨个判断是否和为0。在有了Binary Search后,一个将这个算法的order of growth降低到 N2logN N 2 l o g N 的方法是:
1. 首先将输入的数组进行排序,insertion sort的order of growth为 N2 N 2
2. 然后遍历数组两个的组合,即两层循环, N2 N 2 ,每一次使用binary search查找两个数字之和的负数, lgN l g N 的order of growth,因此共 N2lgN N 2 l g N

4. Theory of algorithms

普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第4张图片
Common mistake. Interpreting big-Oh as an approximate model

5. Memory

5.1 Basics

Bit. 0 or 1.
Byte. 8 bits.
Megabyte (MB). 1 million or 220 bytes.
Gigabyte (GB). 1 billion or 230 bytes.

常见数据类型的内存占用
普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第5张图片
Java Object的内存占用计算
Object overhead,每个primitive type占用的内存,Object内的array记得还要加上reference的占用,最后加起来的占用要进行padding变成8 bytes的倍数
普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第6张图片

普林斯顿算法课Part 1 Week 1 Analysis of Algorithms_第7张图片

你可能感兴趣的:(算法数据结构)