With all the meetings, conferences, papers, deadlines, and professional commitments these days, it is difficult to find time to think. However, certain tasks and commitments invite reflection. Being President of the Information Theory Society is one such position. Among other opportunities, reflection is invited when one is preparing the agenda for the three Board of Governors' meetings each year and in selecting appointees for the several positions of responsibility within the Society. This year another opportunity for reflection presented itself -- collecting the materials and preparing for the five-year Society review with the IEEE Technical Activities Board. To begin, I reread Tony Ephremides' September, 1987 President's Message where he took time "to ponder" and "to contemplate" the identity of the Information Theory Group (then). Further, my own research and the fact that I am teaching Information Theory this semester led me to think more deeply about my association with the Information Theory Society and my attraction to the field itself. In the process, I discovered a very personal view of Information Theory, both the Society and the field.
Like most other members of the Information Theory Society, I am a member of, and active in, other IEEE societies, and I enjoy those meetings, associations, and perspectives. In recent years, however, I have spent most of the time that I have allocated to professional activities involved with the Information Theory Society. What is this attraction? Like Tony, I think that it is a shared viewpoint of our field, but the definition of this viewpoint is probably a little different for everyone, so this is my definition. Since my research has encompassed detection and estimation theory, signal processing, communications, source coding, and rate distortion theory, it is clear that I am attracted by more than Shannon Theory in the narrowest sense. However, because of the Society's origins, Shannon Theory, as it applies to communications problems, is a good place to start.
At the beginning of Shannon's seminal paper [1], he makes it clear that he is concerned with practical communications problems, but that mathematical idealizations of physical entities will be necessary to develop the general theory of communications that he is seeking. Further, he relies on intuition in the problem formulation. Even here the key components of this viewpoint are beginning to reveal themselves. What makes this approach attractive, however, are the relevance and simplicity of Shannon's results. I could cite volumes here, and it is difficult not to, but let me note just three results: (1) His typical sequence proof of the Noiseless Source Coding Theorem [1], (2) the capacity expression for the bandlimited additive white Gaussian noise channel [1], and (3) the rate distortion function for a Gaussian source subject to the squared error distortion measure [2]. I note the first result as much for the insight and intuition available from the typical sequence approach as for the final answer. The last two results I cite because of the elegance and simplicity of the answers themselves and for their fundamental practical significance. To borrow words from Cover and Thomas [3], for these three results, investigating the answer illuminates the problem.
The essential ingredients of this viewpoint are available in other researcher's contributions as well, and many come to mind. Rather than list a few of these and exclude countless others, I suggest contemplating the list of Shannon Award Winners in the September 1996 Newsletter and the accompanying photograph. As one reads each name, what paper, book, or presentation comes to mind? Present in these, I would offer, are an elegance, originality, depth, and clarity of exposition worth seeking [4].
This viewpoint is certainly not limited to Shannon Theory. Interestingly, however, to develop this point, I refer to the panel discussion "Shannon Theory: Present and Future" held at the 1994 IEEE Workshop on Information Theory in Moscow, as transcribed in the December 1994 Newsletter [5]. The participants in the panel discussion were Dick Blahut, Imre Csisz\'ar, David Forney, Prakash Narayan, Mark Pinsker, and Sergio Verd\'u. In this stimulating discussion, the panelists repeatedly emphasized a host of research areas that fall within our broad areas of emphasis in the Society. To further delineate the viewpoint that I feel is present in the Society, I excerpt three passages. During the discussion, Sergio Verd\'u stated, "I like to work on models that are practically relevant, but yet simple enough to admit solutions from which useful lessons can be learned." Practical problems and simple solutions that illuminate. Later, David Forney says, "I think that within the engineering community, information theory has traditionally and seems still to attract people with a taste for analytical approaches to hard but relevant questions." People attracted to analytical solutions of relevant problems. And Sergio again, "..., Information Theory can be characterized by a way of thinking that combines intellectual rigor and engineering insights." Rigorous analyses paired with engineering intuition. Clearly, these characteristics can be sought irrespective of one's research area.
In light of this discussion, Tony, in his 1987 Message, captured much of the spirit of what attracts me to the Information Theory Society and to the field of Information Theory. Relevant, practical, elegant, rigorous, simple, illuminating, intuitive--These terms define this spirit, this viewpoint. Maybe the word elegance catches it all--"refined gracefulness," or perhaps grace--"proportion of line or expression," these are clearly present in many of the results that attract me to the field. But I think I would go further. Thomas Moore writes, "We know intuitively that soul has to do with genuineness and depth,...," [6]. There is soul in Information Theory and there is soul in a certain set of results that naturally appeals to the reader when these results are first understood. Present in this set of results is a quality that one feels is profound -- "de profundis" -- from the depths, which produces a reverberation with one's own Soul, creating a basic, natural attraction.
Throughout the discussion, I have emphasized practical relevance as a motivator and as an important component of the problem, so I reiterate --This is my personal view, not my interpretation of others interests. There are many researchers stimulated by the mathematics of Information Theory and the elegance and structure of the results. We need them, their insights, and enthusiasm. Their contributions are significant and their results are a part of the field available for physical interpretation and application.
It is a great pleasure for me to be affiliated with the IEEE Information Theory Society and to work in the field of Information Theory. Only a relative few are sufficiently talented to make the contributions that define our field, and I certainly would not claim to be one of those. However, to be able to observe and to understand these results and to appreciate their elegance, their grace, is in itself a satisfying accomplishment.
"Tradition teaches that soul lies midway between understanding and unconsciousness, and that its instrument is neither the mind nor the body, but imagination."
Thomas Moore, Care of the Soul
References:
[1] C. E. Shannon, "A Mathematical Theory of Communication," Bell System Tech. J., Vol. 27 (July and Oct. 1948), pp. 379-423 and 623-656.
[2] C. E. Shannon, "Coding Theorems for a Discrete Source with a Fidelity Criterion," IRE Int. Convention Record, Vol. 7 (Pt. 4, 1959), pp. 142-163.
[3] T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley & Sons, Inc., 1991.
[4] A. Ephremides, President's Message, IEEE IT Society Newsletter, Sept. 1987.
[5] S. Verd'u, Moderator, "Shannon Theory: Present and Future," IEEE IT Society Newsletter, Dec. 1994.
[6] T. Moore, Care of the Soul, HarperCollins Publishers, 1992.