Find in Library
Search millions of books, articles, and more
Indexed Open Access Databases
Information-Theoretic Analysis of Memoryless Deterministic Systems
oleh: Bernhard C. Geiger, Gernot Kubin
Format: | Article |
---|---|
Diterbitkan: | MDPI AG 2016-11-01 |
Deskripsi
The information loss in deterministic, memoryless systems is investigated by evaluating the conditional entropy of the input random variable given the output random variable. It is shown that for a large class of systems the information loss is finite, even if the input has a continuous distribution. For systems with infinite information loss, a relative measure is defined and shown to be related to Rényi information dimension. As deterministic signal processing can only destroy information, it is important to know how this information loss affects the solution of inverse problems. Hence, we connect the probability of perfectly reconstructing the input to the information lost in the system via Fano-type bounds. The theoretical results are illustrated by example systems commonly used in discrete-time, nonlinear signal processing and communications.