An Introduction to Information Theory

Written for an engineering audience, this book has a threefold purpose: (1) to present elements of modern probability theory - discrete, continuous, and stochastic; (2) to present elements of information theory with emphasis on its basic roots in probability theory; and (3) to present elements of coding theory.The emphasis throughout the book is on such basic concepts as sets, the probability measure associated with sets, sample space, random variables, information measure, and capacity. Thes…