Information Theoretic Methods in Probability and Statistics

I. Csiszár, Budapest

Abstract:

Ideas of information theory have found fruitful applications not only in various fields of science and engineering but also within mathematics, both pure and applied. This is illustrated by several typical applications of information theory specifically in probability and statistics.

• Introduction
  • Preliminaries on I-divergence
• Applications of IT in probability
  • Gaussian measures dichotomy theorem
  • Large deviations: Sanov's theorem, Gibbs' conditioning principle
  • Measure concentration
  • Other topics
• Information theoretic methods in statistics
  • Early results
  • Hypothesis testing: exponential rate optimal tests
  • Iterative scaling, EM algorithm
  • Minimum description length (MDL)
  • Mutual information in statistics
  • Other topics
• About this document ...