Modern Channel Coding, Inference and Learning, 6 credits (TSKS12)

Main field of study

Electrical Engineering

Course level

Second cycle

Advancement level

A1X

Course offered for

• Computer Science and Engineering, M Sc in Engineering
• Industrial Engineering and Management - International, M Sc in Engineering
• Industrial Engineering and Management, M Sc in Engineering
• Applied Physics and Electrical Engineering, M Sc in Engineering
• Communication Systems, Master's programme
• Computer Science, Master's programme
• Mathematics, Master's programme
• Information Technology, M Sc in Engineering
• Applied Physics and Electrical Engineering - International, M Sc in Engineering
• Computer Science and Software Engineering, M Sc in Engineering

Entry requirements

Note: Admission requirements for non-programme students usually also include admission requirements for the programme and threshold requirements for progression within the programme, or corresponding.

Prerequisites

Linear algebra, Probability, Statistics and basic programming skills. Knowledge in algorithms, data structures and communication systems is desirable but not a requirement.

Intended learning outcomes

After completed course the student should be able to:

• define correctly and explain about the following notions: Hamming distance, linear error-correcting code, LDPC code, “Turbo” code, optimal decoding, iterative decoding, decoding region, channel capacity, density evolution, Monte Carlo simulations, marginalization, neural network;
• passably implement decoding algorithms for modern channel codes as well as plot and analyze performance of those;
• fairly well handle necessary mathematical tools: random variables variables, Bayesian inference, Monte Carlo methods, neural networks;
• independently use advanced channel coding techniques in practical applications;
• implement K-means clustering algorithms for sets of data points;

Course content

• Introduction to information theory and fundamental limits for communication over noisy channels;
• Modern error-correcting codes: LDPC codes and "Turbo" codes;
• Optimal decoding: ML- och MAP- decoding;
• Iterative decoding algorithms and analysis av their performance;
• Bayesian inference and examples of its applications;
• K-means clustering algorithms;
• Exact marginalization;
• Monte Carlo methods for simulation of physical systems;
• Introduction to neural networks: single neurons and examples;
• Capacity of a single neuron;

Teaching and working methods

Teaching is organized in lectures, exercises and laboratory work. The laboratory work consists of programming tasks connected to the theory presented during the lectures. The programming can be carried out in R, C++, Python, Matlab or similar programming language.

Examination

LAB1	Laboratory work	U, G	2 credits
TEN1	Written examination	U, 3, 4, 5	4 credits

Grades

Four-grade scale, LiU, U, 3, 4, 5

Department

Institutionen för systemteknik

Director of Studies or equivalent

Klas Nordberg

Examiner

Danyo Danev

Course website and other links

Education components

Preliminary scheduled hours: 48 h
Recommended self-study hours: 112 h

Course literature

Additional literature

Books

David J.C. MacKay, (2003) Information Theory, Inference and Learning Algorithms
ISBN: 0521642981

Cambridge University Press