Information Theory

Course: Information Theory

Code: 3ФЕИТ10035

ECTS points: 6 ЕКТС

Number of classes per week: 3+0+0+3

Lecturer: Prof. Dr. Venceslav Kafedjiski

Course Goals (acquired competencies): Upon completing the course, it is expected that the student will know the notions, concepts and techniques of information theory, to know how to apply those techniques  to solve different problems of information theory, to know how to determine limits of compression and information transmission rate,  to know the fundamentals of the network information theory and network coding, and to be capable of researching in the area of information theory.

Course Syllabus: Entropy, relative entropy, and mutual information. The asymptotic equipartition property. Entropy rate of a random process. Lossless compression. Universal source coding. Capacity of a discrete memoryless channel. Capacity of a Gaussian memoryless channel. Capacity of a vector Gaussian channel. Lossy compression. Rate distortion theory. Rate distortion of a Gaussian source. Rate distortion of a vector Gaussian source. Network information theory. Fundamental problems of network information theory and methods of their solving. Capacity of multiple access channel and capacity of broadcast channel. Network coding: fundamentals and applications in wireless communications, streaming and multimedia, content distribution networks, data storage systems. Distributed data storage.

Literature: