CS521 Advanced Algorithm Design 学习笔记（一）Course Intro and Hashing

Lecture 1 Course Intro and Hashing

Aim: how to design and analyze a variety of algorithms.

Grad algorithms is a lot about algorithms discovered since 1990. Gradual shift in the emphasis and goals of CS as it became a more mature field. It enjoys a broadened horizon and started looking at new problems, like big data, e-commerce and bioinformatics.

A lot more realistic things are required to concern:

1. Changing Graph: formulation is not easy
2. Changing Data Structures: data come from sources we do not control, e.g., noisy or inexact.
3. Changing notion of I/O: datas may from datastreams, online, social network graphs, etc. Hard to grasp the appropriate output.
4. Analysis: exact, work on all inputs-> approximation

Hashing

Preliminaries

存储一个巨大(e.g., $2^{32}$)集合U的子集$S$: $|S|=m$. 我们希望在$U$中支持对$S$的三种运算：插入，删除和询问，那么，哈希表正是我们需要的。

定义一个哈希函数$h:U\to [n]$，我们只需要开放n个地址来存储哈希值为$0\sim n-1$的数据，并且碰撞元素用链表连接。

两种假设：1. 输入随机 2. 输入给定，哈希函数随机

Hash函数

我们如何定义一个函数是随机的？

理想中，对给定的$x_1,...,x_m\in S$，任意的$a_1,...,a_m\in [n]$，随机函数$\mathcal{H}$应当满足：

• ${\mathrm{Pr}}_{h\in \mathcal{H}}\left[h\left(x_1\right)=a_1\right]=\frac{1}{n}$.
• ${\mathrm{Pr}}_{h\in \mathcal{H}}\left[h\left(x_1\right)=a_1\wedge h\left(x_2\right)=a_2\right]=\frac{1}{n^2}$. Pairwise independence.
• ${\mathrm{Pr}}_{h\in \mathcal{H}}\left[h\left(x_1\right)=a_1\wedge h\left(x_2\right)=a_2\wedge \cdots \wedge h\left(x_k\right)=a_k\right]=\frac{1}{n^k}$. $k$-wise independence.
• ${\mathrm{Pr}}_{h\in \mathcal{H}}\left[h\left(x_1\right)=a_1\wedge h\left(x_2\right)=a_2\wedge \cdots \wedge h\left(x_m\right)=a_m\right]=\frac{1}{n^m}$. Full independence (note that $|U|=m$ ). In this case we have $n^m$ possible $h$ (we store $h(x)$ for each $x\in U)$, so we need $m\log n$ bits to represent the each hash function. Since $m$ is usually very large, this is not practical.

For any $x$, let $L_x$ be the length of the linked list containing $x$, then $L_x$ is just the number of elements with the same hash value as $x$. Let random variable
$$I_y=\{\begin{array}{ll}1& \text{ if }h(y)=h(x)\\ 0& \text{ otherwise }\end{array}$$
So $L_x=1+{\sum }_{y\ne x}{I}_y$, and
$$E\left[L_x\right]=1+\sum _{y\ne x}E\left[I_y\right]=1+\frac{m-1}{n}$$
Note that we don’t need full independence to prove this property, and pairwise independence would actually suffice.

2-Universal Hash

定义函数族$\mathcal{H}$是2-universal的，如果对任意$x\ne y\in U$, $P{r}_{h\in H}[h(x)=h(y)]\le \frac{1}{n}$. （比2-independence要弱）

构造：选取一个质数$p\in [|U|,2|U|]$，令$f_{a,b}(x)=ax+b\mathrm{mod}p$（$a\ne 0$）即可

Since $[p]$ constitutes a finite field, we have that $a={\left(x_1-x_2\right)}^{-1}(s-t)$ and $b=s-a{x}_1$. Since we have $p(p-1)$ different hash functions in $\mathcal{H}$ in this case,
$${\mathrm{Pr}}_{h\in \mathcal{H}}\left[h\left(x_1\right)=s\wedge h\left(x_2\right)=t\right]=\frac{1}{p(p-1)}$$
定理： $\mathcal{H}=\left\{h_{a,b}:a,b\in [p]\wedge a\ne 0\right\}$ 是 2-universal的
证明：For any $x_1\ne x_2$,
$$\begin{array}{rl}& \mathrm{Pr}\left[h_{a,b}\left(x_1\right)=h_{a,b}\left(x_2\right)\right]\\ =& \sum _{s,t\in [p],s\ne t}{\delta }_{(s=t\mathrm{mod}n)}\mathrm{Pr}\left[f_{a,b}\left(x_1\right)=s\wedge f_{a,b}\left(x_2\right)=t\right]\\ =& \frac{1}{p(p-1)}\sum _{s,t\in [p],s\ne t}{\delta }_{(s=t\mathrm{mod}n)}\\ \le & \frac{1}{p(p-1)}\frac{p(p-1)}{n}\\ =& \frac{1}{n}\end{array}$$
注意到，如果选取$n$充分大，使得$n\ge m^2$，那么hash表中的碰撞数量期望为：

$E[{\sum }_{x_1\ne x_2}h(x_1)=h(x_2)]\le \left(\genfrac{}{}{0ex}{}{m}{2}\right)\frac{1}{n}\le \frac{1}{2}$.

不过，我们还有更简单的办法：双人博弈哈希表。假设第i个位置有$s_i$个碰撞，那么我们构造一个大小为$s_i^2$的哈希表，即可容纳所有碰撞。

注意到，$E({\sum }_i{s}_i^2)=E({\sum }_i{s}_i(s_i-1))+E({\sum }_i{s}_i)=\frac{m(m-1)}{n}+m\le 2m$.

负载均衡

在负载均衡问题中，我们可以想象我们尝试把球放到桶里。如果我们有n个球和n个桶，并且随机放，那么第i个桶有k个球的概率$\le \left(\genfrac{}{}{0ex}{}{n}{k}\right)\frac{1}{n^k}$.（把其他桶看成一个整体，用几何分布）$\le \frac{1}{k!}$.

根据斯特林公式，选取$k=O(\frac{\log n}{\log \log n})$，我们有$\frac{1}{k!}\le \frac{1}{n^2}$. 因此，存在一个桶有k个球的概率$\le \frac{1}{n}$.

因此，以$\le 1-\frac{1}{n}$的概率，我们的最大负载不超过$O(\frac{\log n}{\log \log n})$.

改进：当球到来时，随机选两个桶，并且把球放在有更少的球的桶里，那么最大负载以高概率不超过$O(\log \log n)$，这是一个巨大的改进！