A source emits letters A {$a_1$, $a_2$, $a_3$, $a_4$} with probabilities p ($a_1$) = 0.5, p ($a_2$) 0. I5, p ($a_3$) = 0.25, p ($a_4$) = 0.10 calculate the entropy of the source. Also find the huffman code with minimum variance.

p ($a_1$) = 0.5

p ($a_2$) = 0. I5

p ($a_3$) = 0.25

p ($a_4$) = 0.10

Huffman code using minimum variance

Average length:

L = ∑ P(i) x No. of bits

= 0.5 x 2 + 0.15 x 2 + 0.25 x 2 + 0.10 x 2

= 1 + 0.3 + 0.5 + 0.2

L = 2 bits/symbol

Entropy:

H = ${-∑_{i=1}^4}$ $P_i$ $log_2$ ($P_i$)

= - 1/log⁡2 [0.5 log (0.5) + 0.15 log (0.15) + 0.25 log (0.25) + 0.10 log (0.10)]

= -3.322 (-0.151 – 0.124 – 0.151 – 0.1)

H = 1.747 bits/symbol