Shannon and the birth of information

Context

The most important master's thesis ever written. Claude Shannon arrived at MIT in 1936 to tend Vannevar Bush's differential analyzer, a room-sized analog computer. His 1937 thesis noticed something nobody had: the relay circuits switching the machine obeyed Boole's 1854 algebra of logic — hardware could calculate propositions. Every digital circuit since is designed on that observation. It was, people later said, the most important master's thesis of the century — and it was the lesser of his two ideas.

1948. At Bell Labs, in a paper colleagues didn't know he'd been writing, Shannon founded information theory whole: "A Mathematical Theory of Communication" defined information as reduction of uncertainty, gave it a unit (the bit — a name suggested by John Tukey), proved every channel has a capacity, and showed error-free communication is possible up to that capacity through coding. Before the paper, "information" was a vague word; after it, a measurable quantity with theorems. When Shannon needed a name for his uncertainty measure, von Neumann (so the story goes — probably embellished, possibly true) advised "entropy": partly because the formula matched Boltzmann's, partly because "nobody knows what entropy really is, so in a debate you will always have the advantage." The connection turned out to be no joke — that is the next unit. Shannon himself spent the later years juggling, riding a unicycle through Bell Labs' halls, and building a maze-solving mechanical mouse named Theseus — one of the first machine-learning devices — while pointedly refusing to extend his theory to "meaning."

The warning that outranks any book here. The word information now does four different jobs, and most confusion in this whole curriculum comes from letting them blur: Shannon information (probability, bits — rock-solid), algorithmic information (Kolmogorov — precise but uncomputable), physical information (Landauer, Bekenstein — bits with thermodynamic cost), and semantic information (meaning — no settled formal definition, and the place where most "the universe is information" claims secretly live). Keep the four distinct and most popular confusion dissolves on contact. See Act IV of the timeline.

How to read it

Anchor. Colah, Visual Information Theory (~1 hour). The friendliest introduction that's still mathematically honest.

Companion (read second). James Gleick, The Information: A History, a Theory, a Flood. The best popular-history writer on this material, structured as narrative. ~10 pleasurable hours.

Companion (optional). Shannon's original 1948 paper, A Mathematical Theory of Communication. Shorter than you'd expect and readable — worth 2–3 hours after Gleick, for the experience of the original.

Also be able to say precisely what Shannon did not address: meaning.