Which are good books for applications of Shannon Information Theory?

I am a math student, and I'm doing my final graduation project on the Shannon's Information Theory for Continuous Gaussian Channels (Differential Entropy, Time Discrete and Time Continuous Gaussian Channels, and the famous Shannon-Hartley Theorem for Capacity). I would like to discuss some applications of this theory, however, I only find so-advanced-papers, which I really don't understand. Does someone have an accessible book for an Info Theory Applications beginner?

Thanks!

Solution 1:

If you want to start on the subject, a light book with tons of problems is "Elements of Information Theory" by Thomas Cover.

Elements of Information Theory

I recommend this book, it's for elementary level and begging to the subject and I started with it myself, such an intuitive and beautifully written text, totally recommend it. After this if you want to delve deeper into theory, you can read

"Information Theory and Reliable communication" by Robert G. Gallager.

Information Theory and Reliable communication

The second one is an old book but delves more into the mathematics of subject (I studied this one partially and it's much harder to understand). After those you can go even further by studying Network Information Theory, which is information theory applied to scenarios involving multiple receivers or transmitters and there is still many open problems there (which is left unsolved due to difficulty on finding a coding scheme that characterizes the whole capacity region). For that I recommend "Network Information Theory" by Abbas El Gamal

Network Information Theory

If you want to understand the basic communication theory needed, you can study the "Wireless Communications" by Andrea Goldsmith

Wireless Communications

This one gives a good basic understanding of the problem setup you want to study from an engineering point of view (which is more calculating and less abstract math but it is a nice introduction).

Of course I assumed you are already familiar with Random Processes and Random Variables which is necessary to study any of the above books (not in the sense of measure theory and analysis, just knowing the concepts is enough). Furthermore, understanding linear time invariant systems (LTI) and linear time varying systems (LTV) and general signal processing is a huge help. There is also the Holy Bible of digital communication, "Digital Communications" by John G Proakis and Masoud Salehi

Digital Communications

but to be honest I don't think you need to study this one, Goldsmith's book is enough. There is also book on channel coding, source coding and of course, quantum information theory if you want to read more application of the subject and generalization to quantum bits.

You can find downloadable links of the books by searching google, the authors have put a draft version online for download (except "Digital Communications").