Applying Eigenvalues to the Fibonacci Problem

http://scottsievert.github.io/blog/2015/01/31/the-mysterious-eigenvalue/

The Fibonacci problem is a well known mathematical problem that models population growth and was conceived in the 1200s. Leonardo of Pisa aka Fibonacci decided to use a recursive equation:  xn=xn−1+xn−2  with the seed values  x0=0  and  x1=1 . Implementing this recursive function is straightforward:1

1 2 3 4

def fib(n): if n==0: return 0 if n==1: return 1 else: return fib(n-1) + fib(n-2)

Since the Fibonacci sequence was conceived to model population growth, it would seem that there should be a simple equation that grows almost exponentially. Plus, this recursive calling is expensive both in time and memory.2. The cost of this function doesn’t seem worthwhile. To see the surprising formula that we end up with, we need to define our Fibonacci problem in a matrix language.3

[xnxn−1]=xn=A⋅xn−1=[1110]⋅[xn−1xn−2]

Calling each of those matrices and vectors variables and recognizing the fact that  xn−1  follows the same formula as  xn  allows us to write

xn=A⋅xn−1=A⋅A⋯A⋅x0=An⋅x0

where we have used  An  to mean  n  matrix multiplications. The corresponding implementation looks something like this:

1 2 3 4 5

def fib(n): A = np.asmatrix('1 1; 1 0') x_0 = np.asmatrix('1; 0') x_n = np.linalg.matrix_power(A, n).dot(x_0) return x_n[1]

While this isn’t recursive, there’s still an  n−1  unnecessary matrix multiplications. These are expensive time-wise and it seems like there should be a simple formula involving  n . As populations grow exponentially, we would expect this formula to involve scalars raised to the  n th power. A simple equation like this could be implemented many times faster than the recursive implementation!

The trick to do this rests on the mysterious and intimidating eigenvalues and eigenvectors. These are just a nice way to view the same data but they have a lot of mystery behind them. Most simply, for a matrix  A  they obey the equation

A⋅x=λ⋅x

for different eigenvalues  λ  and eigenvectors  x . Through the way matrix multiplication is defined, we can represent all of these cases. This rests on the fact that the left multiplied diagonal matrix  Λ just scales each  xi  by  λi . The column-wise definition of matrix multiplication makes it clear that this is represents every case where the equation above occurs.

A⋅[x1x2]=[x1x2]⋅[λ100λ2]

Or compacting the vectors  xi  into a matrix called  X  and the diagonal matrix of  λi ’s into  Λ , we find that

A⋅X=X⋅Λ

Because the Fibonacci eigenvector matrix is invertible,4

A=X⋅Λ⋅X−1

And then because a matrix and it’s inverse cancel

An=X⋅Λ⋅X−1⋅…⋅X⋅Λ⋅X−1=X⋅Λn⋅X−1

Λn  is a simple computation because  Λ  is a diagonal matrix: every element is just raised to the  n th power. That means the expensive matrix multiplication only happens twice now. This is a powerful speed boost and we can calculate the result by substituting for  An

xn=X⋅Λn⋅X−1⋅x0

For this Fibonacci matrix, we find that  Λ=diag(1+5√2,1−5√2)=diag(λ1,λ2) . We could define our Fibonacci function to carry out this matrix multiplication, but these matrices are simple:  Λ  is diagonal and  x0=[1;0] . So, carrying out this fairly simple computation gives

xn=15√(λn1−λn2)≈15√⋅1.618034n

We would not expect this equation to give an integer. It involves the power of two irrational numbers, a division by another irrational number and even the golden ratio phi  ϕ≈1.618 ! However, it gives exactly the Fibonacci numbers – you can check yourself!

This means we can define our function rather simply:

1 2 3 4 5 6 7

def fib(n): lambda1 = (1 + sqrt(5))/2 lambda2 = (1 - sqrt(5))/2 return (lambda1**n - lambda2**n) / sqrt(5) def fib_approx(n) # for practical range, percent error < 10^-6 return 1.618034**n / sqrt(5)

As one would expect, this implementation is fast. We see speedups of roughly  1000  for  n=25 , milliseconds vs microseconds. This is almost typical when mathematics are applied to a seemingly straightforward problem. There are often large benefits by making the implementation slightly more cryptic!

I’ve found that mathematics5 becomes fascinating, especially in higher level college courses, and can often yield surprising results. I mean, look at this blog post. We went from a expensive recursive equation to a simple and fast equation that only involves scalars. This derivation is one I enjoy and I especially enjoy the simplicity of the final result. This is part of the reason why I’m going to grad school for highly mathematical signal processing. Real world benefits  +  neat theory  =  <3.

1. The complete implementation can be found on Github.↩

2. Yes, in some languages some compilers are smart enough to get rid of recursion for some functions.↩

3. I’m assuming you have taken a course that deals with matrices.↩

4. This happens when a matrix is diagonalizable.↩

5. Not math. Courses beyond calculus deserve a different name.↩

Posted by Scott Sievert Jan 31st, 2015  math