Capacity of an AWGN channel

Now we study the capacity of the analog AWGN channel, assuming that we do not use any digital modulator. Then the situation is the following (see figure 1.4):

Fig.1.4: Block diagram for the AWGN channel

 An analog symbol ξ with a given probability density function 𝑓 _ξ(𝑥) is transmitted over the channel; it is assumed that the variation of ξ is equal to 𝜎_𝜉² and the mean 𝜇_ξ is zero, but there are no further restriction on 𝑓 _ξ(𝑥).

 The AWGN channel adds ν, a Gaussian random variable with variance 𝜎_𝜈² and mean value zero (the probability density function of 𝜈 is denoted as 𝑓 _𝜈(𝑥)).

 The receiver gets η = ξ + ν, a random variable with probability density function 𝑓_η (𝑥) = 𝑓 _ξ(𝑥) ∗ 𝑓 _𝜈(𝑥) (where ∗ stands for convolution).

For the case of an analog AWGN channel, the capacity is obtained by maximizing just ℎ(𝜂). But we know from section 1.1.1.2 that the maximum entropy of an analog source 𝑋 is obtained when the analog source has a Gaussian probability density function. In particular we showed that, for a Gaussian source 𝑥

ℎ(𝑥) =1

2log₂(2𝜋𝑒𝜎_𝑥²)

where 𝜎_𝑥² is the variance of 𝑥. In this case, if the source ξ is Gaussian with zero mean, then also 𝜂 = 𝜉 + 𝜈 is Gaussian, being the sum of two statistically independent Gaussian random variable, and 𝜂 has mean equal to the sum of the means and variance equal to the sum of variance of 𝜉 and 𝜈. So if 𝜉 has variance

ℎ(𝜂) =1

2log₂[2𝜋𝑒(𝜎_𝜉²+ 𝜎_𝜈²)]

and this is the maximum value of ℎ(𝜂) (for the given and fixed 𝜎_𝜉²).

Let us then complete the evaluation of the conditional entropy ℎ(𝜂|𝜉).

𝑓_η|ξ(𝑦|𝑥) = 1

√2𝜋𝜎_𝜈²𝑒𝑥𝑝 {−(𝑦 − 𝑥)² 2𝜎_𝜈² } ℎ(𝜂|𝜉) = − ∫ 𝑓^∞ _η|ξ(𝑤|𝑢)

−∞

log₂𝑓_η|ξ(𝑤|𝑢)𝑑𝑤 = (1

2log₂[2𝜋𝑒𝜎_𝜈²])

In the overall, when the input is Gaussian mutual information is 𝐼(ξ; η) =1

2log₂[2𝜋𝑒(𝜎_𝜉²+ 𝜎_𝜈²)] −1

2log₂[2𝜋𝑒𝜎_𝜈²] =1

2log₂𝜎_𝜉²+ 𝜎_𝜈² 𝜎_𝜈² But this is also the capacity of the AWGN channel:

𝐶 =1

2log₂𝜎_𝜉²+ 𝜎_𝜈² 𝜎_𝜈²

So, each time the AWGN channel is used, it carries at most 𝐶 information bits, and 𝐶 depends on the signal to noise ratio ^𝜎_𝜎^𝜉2

𝜈2: if the noise variance reduces or the source increases, then the capacity increases.

Let us consider now not just the transmission of one analog symbol 𝜉, but a sequence of symbols, and let us limit the problem to the case of a bandlimited channel, in particular a low-pass channel with bandwidth B. Then, only a process 𝜉(𝑡) with bandwidth at most equal to B can pass through the channel without being distorted, and we can represent the information content of 𝜉(𝑡)using just its samples, taken at sampling frequency 2B⁶. Then the entropy of the Gaussian source is

ℎ(𝜉(𝑡)) = 2𝐵ℎ(𝜉) = 𝐵 log₂(2𝜋𝑒𝜎_𝜉²)

where 𝜎_𝜉² is the variance of the process; remember that, if the process is statistically and ergodic, which we will assume then 𝜎_𝜉² does not change with time and is equal to mean power 𝑃_𝜉 of the process.

The channel output process 𝜂(𝑡) is the sum of 𝜉(𝑡) and the white Gaussian noise 𝜈(𝑡) having power spectral density 𝑁₀/2. The receiver has an initial low pass filter followed by a sampler at frequency 2B, so that we can write that the input of the detector is a sequence of samples, generated as rate 2B samples per seconds, which are the sum of the samples of 𝜉(𝑡) and noise random variables with variance

𝜎_𝜈² = 𝑁₀

2 2𝐵 = 𝑁₀𝐵 The entropy of 𝜂(𝑡) = 𝜉(𝑡) + 𝜈(𝑡), sampled at rate 2B, is

ℎ(𝜂(𝑡)) = 2𝐵ℎ(𝜂) = 𝐵 log₂[2𝜋𝑒(𝜎_𝜉²+ 𝜎_𝜈²)]

The conditional entropy is ℎ(𝜂(𝑡)|𝜉(𝑡)) = 𝐵 log₂[2𝜋𝑒𝜎_𝜈²] as before.

The AWGN channel capacity is then

𝐶^′= ℎ(𝜂(𝑡)) − ℎ(𝜂(𝑡)|𝜉(𝑡)) = 𝐵 log₂𝜎_𝜉²+ 𝜎_𝜈² 𝜎_𝜈² We can substitute the values of the variances and obtain

6 According to the sampling theorem that states that if a signal 𝑥(𝑡) has bandwidth B, it is possible to exactly evaluate 𝑥(𝑡) from its samples, provided that the sampling frequency is larger than 2𝐵𝑥.

𝐶^′= 𝐵 log₂(1 + 𝑃_𝜉 𝑁₀𝐵)

where now the unit of measure of 𝐶^′is bits of information per second (not just bit of information).

In brief, compare to 𝐶 and 𝐶^′:

 𝐶 is capacity per channel use 𝐶 =¹₂log₂(1 +_𝑁^𝑃𝜉

0𝐵) [information bit per channel use]

 𝐶^′ is capacity measured, which use the channel 2B times per second. If we do not use the low pass filter, the capacity is zero for sure.

𝐶^′ = 𝐵 log₂(1 +_𝑁^𝑃𝜉

0𝐵) [information bit per second]

Let us see if we can relate the discrete channel capacities with the AWGN channel capacity. We can imagine that process 𝜉(𝑡) is the output of a digital modulator that generates bits (real bits “1” or “0”) at rate 𝑅_𝑏 bits/s, so that the power 𝑃_𝜉 can be expressed as 𝑃_𝜉 = ^𝐸_𝑇^𝑏

𝑏= 𝐸_𝑏𝑅_𝑏, where 𝐸_𝑏 is the energy per bit. So we have, for the AWGN channel, 𝐶^′ = 𝐵 log₂(^𝐸_𝑁^𝑏𝑅𝑏

0𝐵 + 1) or ^𝐶_𝐵^′ = log₂(^𝐸_𝑁^𝑏𝑅𝑏

0𝐵 + 1).

It is not possible to get an error probability equal to zero if the input entropy is larger than the channel capacity. At most one bit transmitted by the digital modulator carries one information bit, so that we can say that the source entropy is 𝐻(𝑋) = 𝑅_𝑏 information bits per second, and, if we assume that we are working at the limit, i.e.

the best case, with 𝐻(𝑋) = 𝐶^′, we have 𝐶^′

𝐵 = 𝑅_𝑏 𝐵 which leads to

𝑅_𝑏

𝐵 = log₂(𝐸_𝑏𝑅_𝑏 𝑁₀𝐵 + 1)

which provides a relationship between the signal to noise ration 𝐸_𝑏/𝑁₀ and the modulation efficiency 𝑅_𝑏/𝐵 (measured in bits/second per hertz). In particular, we can write

𝐸_𝑏

𝑁₀ = 2^{𝑅𝑏/𝐵}− 1 𝑅_𝑏/𝐵

 If ^𝑅_𝐵^𝑏= 1, then ^𝐸_𝑁^𝑏

0 = 1 (0 𝑑𝐵);

 if ^𝑅𝑏

𝐵 → 0, ^𝐸_𝑁^𝑏

0→ ∞;

 if ^𝑅𝑏

𝐵 → ∞, then ^𝐸_𝑁^𝑏

0 → ln2 (−1.6 𝑑𝐵):

𝑅_𝑏lim/𝐵→∞

2^{𝑅𝑏/𝐵}− 1

𝑅_𝑏/𝐵 = lim

𝑅_𝑏/𝐵→∞

𝑒^{𝑅𝑏/𝐵 log𝑒2}− 1

𝑅_𝑏/𝐵 = lim

𝑅_𝑏/𝐵→∞

1 − 𝑅_𝑏/𝐵 log_𝑒2 − 1

𝑅_𝑏/𝐵 = ln 2

= 0.693

The last limit is quite interesting: it starts that it is possible to transmit with error probability equal to zero if the signal to noise ratio is 𝐸_𝑏/𝑁₀ > −1.6 𝑑𝐵, provided that the bandwidth B is infinite. Note that 𝐸_𝑏/𝑁₀ = 1.6 𝑑𝐵 in the case in which the noise variance 𝑁₀/2 is equal to 0.72𝐸_𝑏, really very high. Another interesting consideration is that we can trade energy with bandwidth: if we increase the bandwidth, we can reduce 𝐸_𝑏/𝑁₀ and vice-versa.

Note that it is not possible to get error probability equal to zero if, having fixed 𝐸_𝑏/𝑁₀, the spectral efficiency is higher than the value shown in the curve of Fig. 1.5;

similarly it is not possible to get error probability equal to zero, if, having fixed the spectral efficiency 𝑅_𝑏/𝐵, the signal to noise ratio is lower than the value shown in the curve of Fig.1.5. In principle, any transmission system specified by a couple of values (𝐸_𝑏/𝑁₀), (𝑅_𝑏/𝐵) below the curve in Fig.1.5 can work with error probability equal to zero.

Fig.1.5: Plot Shannon channel capacity curve of spectral efficiency 𝑅_𝑏/𝐵 versus 𝐸_𝑏/𝑁₀ for the AWGN channel (channel capacity limit)

In summary, the Shannon channel capacity curve, meaning the theoretical tightest upper bound on the information rate of data that can be communicated at an arbitrarily low error rate using an average received signal power through an analog communication channel subject to additive white Gaussian noise of power:

𝐶^′= 𝐵 log₂(1 + 𝑃_𝜉 𝑁₀𝐵) where

 𝐶^′ is the channel capacity in information bits per second, a theoretical upper bound on the net bit rate (information rate) excluding error-correction codes;

 𝐵 is the bandwidth of the channel in hertz (passband bandwidth in case of a bandpass signal);

 𝑃_𝜉 is the average received signal power over the bandwidth (in case of a carrier-modulated passband transmission), measured in watts (or volts squared);

 𝑁₀ is the average power of the noise and interference over the bandwidth, measured in watts (or volts squared);

 𝑃_𝜉/(𝑁₀𝐵) is the signal-to-noise ratio (SNR) of the communication signal to the noise and interference at the receiver (expressed as a linear power ratio, not as logarithmic decibels).