EEE 551-Information Theory

Textbook: T. Cover and J. Thomas, Elements of Information Theory

Outline

I. Entropy and Mutual Information

A. Definitions and properties of entropy
B. Definitions and properties of mutual information
C. Relationships between entropy and mutual information
D. Jensen's and related inequalities

II. Asymptotic Equipartition Property

III. Lossless Source Coding

A. Kraft inequality
B. Optimal codes and bounds on average codeword length
C. Huffman coding
D. Arithmetic coding
E. Lempel-Ziv coding

IV. Channel Capacity

A. Definition and computation of information capacity
B. Channel coding theorem

V. Rate Distortion Theory

A. Definition and computation of the rate-distortion function
B. Rate distortion theorem
C. Vector quantization

Return to Communications Courses