Information Theory

COURSE: Information Theory

Code: ФЕИТ10030

ECTS points: 6 ЕCTS

Number of classes per week: 3+0+0+3

Lecturer: prof. d-r Venceslav Kafedziski

Subject of the course content: Entropy, relative entropy, and mutual information. Asymptotic equipartition property. Entropy rate of random processes. Lossless data compression. Huffman coding. Shannon-Fano-Elias coding. Universal coding of information sources. Arithmetic coding. Lempel-Ziv coding. Capacity of discrete memoryless channels. Capacity of Gaussian memoryless channels. Capacity of vector Gaussian channels. Lossy data compression. Rate distortion theory. Quantization. Rate distortion of Gaussian information sources. Network information theory. Fundamental problems of network information theory and approaches to their solution. Capacity of the multiple access channel and capacity of the broadcast channel. Capacity of the relay channel. Distributed coding of multiple information sources. Information theory and statistics. 

Literature:

T. M. Cover, J. A. Thomas, “Elements of Information Theory”, Wiley, 2006.