Zipf's Law
Zipf's law describes a striking pattern in the frequency of words in a language: the most common word occurs about twice as often as the second most common, three times as often as the third, and so on.
Where this idea comes from
Start here — these are the sources that inspired this exploration.
Introduction
Count how often each word appears in a long text and rank them from most to least frequent. Zipf's law states that frequency is roughly proportional to 1 divided by rank, a pattern that shows up not just in language but in city populations and website traffic. This exploration tests the law against real text, then checks whether the same pattern, and the same exponent, holds elsewhere. Fitting and interpreting the exponent formally uses AI HL techniques (linearizing with logarithms and non-linear regression); AI SL students can still test the 1/rank proportionality using a log-log plot without the formal regression machinery.
Guiding Questions
- Count word frequencies in a long text. What pattern links a word's rank to its frequency?
- Test Zipf's law: is frequency roughly proportional to 1/rank? Use a log-log plot.
- Does the law hold in a different language, or in song lyrics versus news articles?
- Where else does the same pattern appear — city sizes, website visits? Test one and compare exponents.