An extension for AI HL, Hodder section 3D (Eigenvalues and eigenvectors). Ten simple examples of eigenvectors at work.
1 · What does an eigenvector actually do?
Take \(A=\begin{pmatrix}2&1\\1&2\end{pmatrix}\). Two directions are special:
\[A\begin{pmatrix}1\\1\end{pmatrix}=3\begin{pmatrix}1\\1\end{pmatrix},\qquad A\begin{pmatrix}1\\-1\end{pmatrix}=1\begin{pmatrix}1\\-1\end{pmatrix}.\]So the eigenvalues are \(\lambda=3\) and \(\lambda=1\). What happens to \(\lambda^n\) as \(n\) grows tells you what happens in real life:
| \(\lambda\) | \(\lambda^n\) as \(n\) grows | In real life |
|---|---|---|
| \(\lambda>1\) | grows without limit | population boom, epidemic |
| \(\lambda=1\) | stays the same | steady state, long-run share |
| \(0<\lambda<1\) | shrinks to \(0\) | dies out, settles down |
| \(\lambda<0\) | flips sign each step | see-saw, oscillation |
Inside AI models: training passes numbers through many layers, a bit like multiplying by matrices again and again. If the eigenvalues are too big the numbers explode; too small and they vanish.
2 · Why does the biggest eigenvalue take over?
Suppose \(A\) has eigenvectors \(\mathbf{v}_1,\mathbf{v}_2\) with \(|\lambda_1|>|\lambda_2|\). Write any starting vector as a mix of them:
\[\mathbf{x}_0=c_1\mathbf{v}_1+c_2\mathbf{v}_2.\]Each time we apply \(A\), each part is just scaled by its own \(\lambda\):
\[A^n\mathbf{x}_0=c_1\lambda_1^{\,n}\mathbf{v}_1+c_2\lambda_2^{\,n}\mathbf{v}_2=\lambda_1^{\,n}\left(c_1\mathbf{v}_1+c_2\left(\tfrac{\lambda_2}{\lambda_1}\right)^{n}\mathbf{v}_2\right).\]Because \(\left|\tfrac{\lambda_2}{\lambda_1}\right|<1\), that last term shrinks to zero. So in the long run \(A^n\mathbf{x}_0\approx c_1\lambda_1^{\,n}\mathbf{v}_1\): the direction settles on \(\mathbf{v}_1\) and the growth factor settles on \(\lambda_1\). This is the same thing as diagonalisation, \(A^n=PD^nP^{-1}\).
Simple case. With \(A=\begin{pmatrix}2&1\\1&2\end{pmatrix}\) and \(\begin{pmatrix}1\\0\end{pmatrix}=\tfrac12\begin{pmatrix}1\\1\end{pmatrix}+\tfrac12\begin{pmatrix}1\\-1\end{pmatrix}\):
\[A^n\begin{pmatrix}1\\0\end{pmatrix}=\tfrac12\,3^n\begin{pmatrix}1\\1\end{pmatrix}+\tfrac12\,1^n\begin{pmatrix}1\\-1\end{pmatrix}.\]3 · Which arrows survive a shear?
A shear slants a picture. Almost every arrow turns; the ones that don't are the eigenvectors.
\[S=\begin{pmatrix}1&1\\0&1\end{pmatrix},\qquad \det(S-\lambda I)=(1-\lambda)^2=0\;\Rightarrow\;\lambda=1.\] \[(S-I)\mathbf{v}=\begin{pmatrix}0&1\\0&0\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix}=\mathbf{0}\;\Rightarrow\;y=0,\qquad \mathbf{v}=\begin{pmatrix}1\\0\end{pmatrix}.\]A shear has only one eigenvector direction. Italic fonts and the "slant" tool in photo apps are shears.
4 · What stays still when the Earth spins?
Every point moves except the ones on the axis, so the axis is an eigenvector with \(\lambda=1\). A quarter turn about the \(z\)-axis:
\[R=\begin{pmatrix}0&-1&0\\1&0&0\\0&0&1\end{pmatrix},\qquad R\begin{pmatrix}0\\0\\1\end{pmatrix}=\begin{pmatrix}0\\0\\1\end{pmatrix}.\]Euler showed in 1775 that every rotation in 3D has an axis like this. In 2D, though, a quarter turn has no real eigenvector at all:
\[Q=\begin{pmatrix}0&-1\\1&0\end{pmatrix},\qquad \det(Q-\lambda I)=\lambda^2+1=0\;\Rightarrow\;\lambda=\pm i.\]Every arrow in the plane turns. Complex eigenvalues mean rotation, and in our last chapter they are why some solutions spiral.
5 · How did Google use an eigenvector to rank web pages?
Google's first ranking (PageRank, 1998): a page is important if important pages link to it. Imagine someone clicking links at random forever; a page's rank is the share of time they spend there.
Column = from, row = to:
\[T=\begin{pmatrix}0&0&\tfrac12\\ \tfrac12&0&\tfrac12\\ \tfrac12&1&0\end{pmatrix}.\]The long-run share \(\mathbf{s}\) solves \(T\mathbf{s}=\mathbf{s}\), i.e. \((T-I)\mathbf{s}=\mathbf{0}\) with \(a+b+c=1\):
\[\mathbf{s}=\begin{pmatrix}2/9\\3/9\\4/9\end{pmatrix}\approx\begin{pmatrix}0.22\\0.33\\0.44\end{pmatrix}.\]Ranking: C, then B, then A. \(\mathbf{s}\) is the eigenvector of \(T\) for \(\lambda=1\). With billions of pages nobody solves the equations: Google multiplied a starting vector by \(T\) again and again, and the \(\lambda=1\) eigenvector took over.
6 · How do eigenvectors predict how a population grows?
Track juveniles \(J\) and adults \(A\). Each year every adult has 3 young, half the juveniles grow up, and half the adults survive:
\[\begin{pmatrix}J_{n+1}\\A_{n+1}\end{pmatrix}=\underbrace{\begin{pmatrix}0&3\\ \tfrac12&\tfrac12\end{pmatrix}}_{L}\begin{pmatrix}J_n\\A_n\end{pmatrix},\qquad \det(L-\lambda I)=\tfrac12(2\lambda-3)(\lambda+1)=0.\]So \(\lambda=\tfrac32\) or \(\lambda=-1\). For \(\lambda=\tfrac32\) the eigenvector is \(\begin{pmatrix}2\\1\end{pmatrix}\). Starting with 10 adults:
| Year | \(J\) | \(A\) | \(J/A\) | Growth |
|---|---|---|---|---|
| 1 | 30 | 5 | 6.00 | ×3.50 |
| 2 | 15 | 17.5 | 0.86 | ×0.93 |
| 3 | 52.5 | 16.3 | 3.23 | ×2.12 |
| 10 | 680 | 350 | 1.94 | ×1.47 |
| 20 | 39 891 | 19 956 | 2.00 | ×1.50 |
It wobbles, then settles on 2 juveniles per adult (the eigenvector) and 50% growth a year (\(\lambda=\tfrac32\)). A 1987 study of loggerhead sea turtles used this method and found that protecting older juveniles at sea mattered more than protecting eggs on beaches.
7 · Why do Fibonacci numbers approach the golden ratio?
Each term is the sum of the two before: \(1,1,2,3,5,8,13,\dots\) As a matrix:
\[\begin{pmatrix}F_{n+1}\\F_n\end{pmatrix}=\underbrace{\begin{pmatrix}1&1\\1&0\end{pmatrix}}_{M}\begin{pmatrix}F_n\\F_{n-1}\end{pmatrix},\qquad \det(M-\lambda I)=\lambda^2-\lambda-1=0.\] \[\lambda=\frac{1+\sqrt5}{2}\approx1.618\qquad\text{or}\qquad\lambda=\frac{1-\sqrt5}{2}\approx-0.618.\]The big one is the golden ratio \(\varphi\). The other has \(|\lambda|<1\), so its part dies away and \(F_n/F_{n-1}\to\varphi\). The ratios go \(2\), \(1.5\), \(1.667\), \(1.6\), …, \(55/34=1.6176\), \(144/89=1.6180\).
8 · How fast does a disease spread?
Epidemiologists' "\(R\) number" is the largest eigenvalue of a matrix. Column = who is ill, row = who they infect on average:
\[K=\begin{pmatrix}1.5&0.5\\1&1\end{pmatrix}\ \begin{matrix}\leftarrow\text{children}\\ \leftarrow\text{adults}\end{matrix},\qquad \det(K-\lambda I)=(\lambda-2)(\lambda-0.5)=0.\]So \(R=2\) with eigenvector \(\begin{pmatrix}1\\1\end{pmatrix}\). Starting with one ill child, the totals go \(2.5,\ 5.25,\ 10.6,\ \dots,\ 85.3\) by generation 6: doubling, with children and adults settling to \(1:1\).
Vaccinate half of everyone and every entry of \(K\) halves, so \(\lambda\) halves to \(1\): the outbreak stops growing. During COVID-19, modellers estimated \(R\) this way from contact surveys split by age group.
9 · How do computers find patterns in data?
For standardised maths and physics test scores, the covariance matrix might be
\[C=\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix},\qquad \det(C-\lambda I)=(1-\lambda)^2-0.64=0\;\Rightarrow\;\lambda=1.8\text{ or }0.2.\]\(\lambda=1.8\) has eigenvector \((1,1)\), "good at both"; \(\lambda=0.2\) has \((1,-1)\), "better at one than the other". Since \(\tfrac{1.8}{1.8+0.2}=90\%\) of the spread lies along \((1,1)\), one number per student keeps most of the information.
The same idea gives face recognition ("eigenfaces"), image compression, and the 2D maps researchers draw of how an AI language model groups words: thousands of numbers per word, squashed onto the top two eigenvectors.
10 · Coming in our last chapter
In coupled differential equations, \(\dfrac{d\mathbf{x}}{dt}=A\mathbf{x}\), and the eigenvalues of \(A\) decide the whole shape of the motion:
- Both negative: everything settles to rest, like a swing slowing down.
- Opposite signs: a "saddle": in along one eigenvector, out along the other.
- Complex: spirals, like the quarter turn with \(\lambda=\pm i\); predators and prey.
Engineers use the same idea: a bridge's natural wobble frequencies come from eigenvalues. London's Millennium Bridge swayed on its opening day in 2000 because crowds' footsteps matched one of them, and it closed for nearly two years to be fixed.
Check your understanding
1. Show that \(\begin{pmatrix}1\\1\end{pmatrix}\) is an eigenvector of \(\begin{pmatrix}4&1\\2&3\end{pmatrix}\) and state its eigenvalue.
\(\begin{pmatrix}4&1\\2&3\end{pmatrix}\begin{pmatrix}1\\1\end{pmatrix}=\begin{pmatrix}5\\5\end{pmatrix}=5\begin{pmatrix}1\\1\end{pmatrix}\), so \(\lambda=5\).
2. A population matrix has eigenvalues \(0.9\) and \(-0.4\). What happens to the population in the long run?
The largest \(|\lambda|\) is \(0.9<1\), so the population shrinks by about 10% a year and dies out.
3. If the links in the PageRank example change, which eigenvalue do we still look for, and why?
\(\lambda=1\), whatever the links. Each column of a transition matrix adds to 1, so \(\lambda=1\) is always an eigenvalue and its eigenvector is the long-run share.
See also: What does it mean for a matrix to be singular? (AHL 5.4). Eigenvalues only exist because of singular matrices.
