Proof of Cover's Function Counting Theorem
Question Description:
Suppose we have points in . We have partitions of these points into two classes.
The question is: how many these partitions yield linearly separable classes, i.e. where two classes can be perfectly separated by -dimensional hyperplane?
We assume all the points are in general position, which means that any subset of or fewer points is linearly independent（线性无关）.
Proof:
Denote the number of linearly separable partition by . We will find the expression for by induction. Image first having points and then adding one more point. Now, considering the linearly separable partitions of previous points, there are two possibilities:
Case 1: there is a separating hyperplane for the previous points passing through the new point, in which case each such linearly separable partition of the previous points gives rise to two distinct linearly separable partitions as the hyperplane can be shifted infinitesimally to place the new point in either class.
Case 2: there is no separating hyperplane for the previous points passing through the new point, in which case each such linearly separable partition gives rise to only one linearly separable partition.
The number of linearly separable partition in Case 1 is precisely , because restricting the separating hyperplane to pass through a fixed point is the same as eliminating one degree of freedom and thus projecting the points to a -dimensional space. This can be understood if the new point is on the axis, for example - then the hyperplane has axes left to work with. If the point is not on the axis, then rotate the axes of space around to get the point on the x axis, and this of course has no effect on the geometry of the problem.
The recursive relation:
, where is the number of separable hyperplanes in Case 2, and is the number of separable hyperplanes in Case 1.
Iterating the recursion once, we have
Continue to iterate the recursion (twice)
After iterations, we have
where
So, finally we have
where