Applied Information Theory

1.

Subject title

Applied Information Theory

Применета теорија на информации

2.

Code

m23_w_018

3.

Study program

IT management, Bioinformatics, Cloud Computing, Еducation with ICT, Eco-informatics, Inteligent Systems, Internet Technologies and cyber security, Computer Science, Statistics and Data Analytics, Software for embedded systems, Software Engineering, Cloud Computing, IT management, Bioinformatics, Security, Cryptography and Coding, Statistics and Data Analytics, Software Engineering, Data science in computer science and engineering, Security, Cryptography and Coding,

4.

Organizer of the study program (unit, institute, department, division)

Faculty of Information Sciences and Computer Engineering

5.

Study cycle (first, second, third)

Втор циклус

6.

Academic year / semester

5 / Зимски

7. Number of ECTS credits

6.0

8.

Instructor

проф. д-р Александра Поповска Митровиќ проф. д-р Наташа Илиевска проф. д-р Верица Бакева

9.

Prerequisites for enrollment

10.

Subject goals and competencies:

Studying the advanced aspects of a mathematical model of a communication system.

11.

Subject content:

Communication system. Entropy. Information. Data compression: loss coding. Asymptotic Equipartition Property (AEP) for independent random variables. Shannon`s theorem for source signal coding. Loss -free coding. Symbolic codes. Problem of only decoding. Instant codes. Kraftovo inequality. Theorem of silent coding. Construction of optimal codes. Communication through a noise channel (communication channel. Communication channel models. Discrete channel without memory. Discrete channel capacity without memory). Sources of information: Markov`s chains. Source of information. Regular Markov source. Entropy of source. Source order. Approximation of a general source of information with a final order source. Earnest source. Shannon theorem - McMillan (Asymptotic Equipartition Property (AEP)). Discrete channel with memory: Model models with memory. Channel with a finally set of states. Capacity of general discreet channel. The coding theorem for a regular channel with a finally set of conditions. Continuous channels: entropy of continuous random variables. Entropy of Gaussian random variable. Types of non -jet channels. Gaussian channel (time discreet). AEP for continuous random variables. Coding theorem for Gaussian Channel.

12.

Learning methods:

Предавања, проекти, дискусии, работилници

13.

Total available time fund

6.0 ECTS x 30 hours = 180 hours

14.

Time distribution

60 + + 45 + 45 + 30 = 180 hours

15.

Forms of teaching activities

15.1.

Lectures - theoretical teaching

60 hours

15.2.

Exercises (laboratory, classroom), seminars, team work

hours

16.

Other forms of activities

16.1.

Project tasks

45 hours

16.2.

Independent tasks

45 hours

16.3.

Homework

30 hours

17.

Grading method

17.1.

Tests

0 points

17.2.

Seminar work / project (presentation: written and oral)

45 points

17.3.

Activities and learning

0 points

17.4.

Final exam

50 points

18.

Grading criteria (points / grade)

up to 50 points

5 (five) (F)

from 51 to 60 points

6 (six) (E)

from 61 to 70 points

7 (seven) (D)

from 71 to 80 points

8 (eight) (C)

from 81 to 90 points

9 (nine) (B)

from 91 to 100 points

10 (ten) (A)

19.

Condition for signature and taking final exam

Реализирани активности 15, 16

20.

Language of instruction

Македонски и англиски

21.

Quality assurance method

Механизам на интерна евалуација и анкети

22.

Literature

22.1.

Mandatory literature

22.2.

Additional literature