Information as a Building Block · information·1

Shannon and the birth of information

est 25h · logged 0h · started — · touched —

The mathematical foundation. Shannon's 1948 paper turned information from a vague word into a measurable quantity with units and theorems; this unit gets you fluent in that language.

next action

done when you can

Define Shannon entropy in one sentence and explain why it's measured in bits.Explain mutual information and why it matters.State the noisy-channel coding theorem in plain language.Say precisely what Shannon did not address (meaning; semantics).

resources

●Visual Information Theory— Christopher Olah (Colah)anchor · blog post
○The Information: A History, a Theory, a Flood— James Gleickcompanion · book
○A Mathematical Theory of Communication— Claude Shannonoptional · paper

sessions

unlocks Everything. Shannon is the prerequisite for every other unit.

open in notion ↗