• Non ci sono risultati.

Introduction to Information Theory

N/A
N/A
Info
Scarica
Protected

Academic year: 2022

Condividi "Introduction to Information Theory"

Copied!
1
0
0

Caricamento.... (Visualizza il testo completo ora)

Scarica adesso ( 1 pagine )

Testo completo

(1)

Introduction to Information Theory

Deniz Gunduz

1

1Imperial College, London, UK Email: d.gunduz@imperial.ac.uk

Timetable: 16 hrs. Class meets on Monday 11:00-13:00 and Tuesday 14:00-16:00 for four weeks starting October 08, 2018, with the exception of the week of Oct. 15 in which the classes will be on Thursday 11:00-13:00 and Friday 14:00-16:00. Department of Information Engineering, Via Gradenigo 6B, Room: Sala Riunioni DEI/D.

Course requirements:

Examination and grading: Grades will be based on a final exam SSD: Information Engineering

Aim: The aim of this course is to introduce basic information theoretic concepts to students. We will start by introducing entropy, divergence, and, mutual information, and their mathematical properties. The rest of the course will be dedicated to illustrating engineering applications of these seemingly abstract quantities. We will see that entropy corresponds to the ultimate limit in data compression, divergence provides the best error exponent in hypothesis testing (i.e., binary classification), and mutual information sets the limit of how much data one can transmit reliably over a noisy communication channel.

Syllabus:

1. Week 1: Information measures

• Day 1: Entropy, divergence, mutual information

• Day 2: Properties of information measures (chain rule, data processing inequality, convexity)

2. Week 2: Lossless data compression

• Day 1: Asymptotic equipartition property (AEP)

• Day 2: Kraft inequality, Huffman coding and its optimality 3. Week 3: Information theory and learning

• Day 1: Method of types, universal source coding, large deviations: Sanovs theorem

• Day 2: Hypothesis testing, Steins lemma, Chernoff exponent 4. Week 4: Channel coding

• Day 1: Channel capacity theorem, achievability, joint AEP

• Day 2: Converse to channel coding theorem, feedback capacity, Joint source-channel coding

References:

1. R. B. Ash, Information Theory, Dover, 1990.

2. T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, 1991.

3. R. G. Gallager, Information Theory and Reliable Communication, Wiley, 1968.

DEI-5

Riferimenti

Scarica adesso ( PDF - 1 pagine - 56.69 KB )

Documenti correlati

At the proof stage, we will require you to read, sign, and return the

Additionally, the dipole moment of trans-propargylamine was determined by Stark effect mea- surements (the values µ a = 0.586 D and µ b = 0.445 D being derived), but the very

Introduction to the course System and Control Theory. Prof. Roberto Zanasi

Examples of modeling, simulation and control of physical systems using the Matlab/Simulink environment.. Zanasi Roberto -

Introduction to the course

Atzeni, Ceri, Fraternali, Paraboschi, Torlone, Basi di dati – Architetture e linee di evoluzione , McGraw Hill, 2007 Atzeni, Ceri, Paraboschi, Torlone, Database systems ,

Introduction to Game Theory

Likewise, if your accomplice confesses while you remain silent, he will go free (freedom) while you do the time (4 years in jail). • If you both confess I get two convictions, but

Introduction to the R Language Data Types and Basic Operations

Each element of the list can be thought of as a column and the length of each element of the list is the number of rows Unlike matrices, data frames can store different classes

The number corresponds to the database number of the bibentry

Government Printing Office , Social Security Administration (US)..

Introduction to Financial Econometrics Chapter 5 Introduction to Portfolio Theory

For portfolios with expected returns above the T-bill rate, efficient portfolios can also be characterized as those portfolios that have minimum risk (as measured by portfolio

We want to show that:

Cartesian reference system belonging to the triangle