DS-GA 3001 (MATH-GA 2830): Information Theory for Statistics and Learning

Most materials will be taken from the book “Information Theory: from Coding to Learning” by Y. Polyanskiy and Y. Wu.

Lecture notes will be provided as the course progresses.

• Lecture 1: Entropy and mutual information

• Lecture 2: KL divergence

• Lecture 3: f-divergence