Channel Capacity

Channel Capacity in Information Theory

Channel capacity represents the maximum amount of information that can be reliably transmitted over a communication channel. It is determined by the maximum mutual information between the input and output, optimized over all possible input probability distributions.

---

A. Channel Capacity per Symbol

For a discrete memoryless channel (DMC), the capacity per symbol is:

\[ C_s = \max_{P(x_i)} I(X; Y) \quad \text{(bits per symbol)} \]

• The maximization is over all input probability distributions \(P(x_i)\)
• This depends solely on the channel’s transition probabilities

---

B. Channel Capacity per Second

If \(r\) symbols are transmitted per second, the capacity becomes:

\[ C = r \cdot C_s \quad \text{(bits per second)} \]

This represents the highest achievable data rate for reliable communication over the channel.

---

Capacities of Special Channels

1. Lossless Channel

• All source information reaches the output intact:
  \[ H(X|Y) = 0, \quad I(X; Y) = H(X) \]
• Maximum entropy occurs when input is uniformly distributed: \[ C_s = \max_{P(x_i)} H(X) = \log_2 m \] where \(m\) is the number of input symbols

Example:
If \(m = 4\), then
\[ C_s = \log_2 4 = 2 \text{ bits/symbol} \]

---

2. Deterministic Channel

• The output is completely determined by the input:
  \[ H(Y|X) = 0, \quad I(X; Y) = H(Y) \]
• Capacity per symbol: \[ C_s = \max_{P(x_i)} H(Y) = \log_2 n \] where \(n\) is the number of output symbols

---

3. Noiseless Channel

• Both lossless and deterministic: \[ I(X; Y) = H(X) = H(Y) \]
• Capacity per symbol: \[ C_s = \log_2 m = \log_2 n \]

---

4. Binary Symmetric Channel (BSC)

For a binary channel with crossover probability \(p\), the capacity is:

\[ C_s = 1 + p \log_2 p + (1 - p) \log_2 (1 - p) \]

This formula accounts for the probability of errors introduced by the channel.

---