coursemapLoading course
School of Engineering · ANU College of Engineering Computing & Cybernetics
This course will introduce students to the main concepts of information theory. Information theory has many applications in communications theory and practice, computer science, and statistics. For example, it quantifies fundamental limits of reliable communications in the presence of noise without restricting the treatment to a particular method of transmission or reception. Specific topics include: - Entropy, Relative Entropy, Mutual Information. - Entropy Rates of a Stochastic Process. - Data Compression and Source Coding. - Channel Capacity and Channel Coding Theorem. - Differential Entropy. - Capacity of Gaussian Channels. - Network Information Theory (such as Multiple-access Channel, Broadcast Channel, Relay Channel). - Introduction to Network Coding (if time allows).
Assignments:
Learning outcomes 1, 2, 3, 4, 5, 6, 7, 8
Active participation:
Learning outcomes 1, 2, 3, 4, 5, 7, 8
Group research project:
Learning outcomes 5, 6, 8, 9
Mid-term exam:
Learning outcomes 1, 2, 3, 4, 5, 7, 8
Final exam:
Learning outcomes 1, 2, 3, 4, 5, 6, 7, 8
2 x 2-hour lectures per week (4 hours total), 1 x 1-hour tutorial (or self-taught tutorial) per week, 5 hours independent/or group-based study per week. Other Information Professional Skills Mapping: Mapping of Learning Outcomes to Assessment and Professional Competencies
Textbook: 'Elements of Information Theory', Thomas Cover and Joy Thomas, Second Edition, 2006. Optional Classic Textbook Suggestion: 'Information Theory and Reliable Communication', Robert G. Gallager, New York, Wiley, 1968