### Abstract

The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.

Original language | English |
---|---|

Pages (from-to) | 185-222 |

Number of pages | 38 |

Journal | Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete |

Volume | 12 |

Issue number | 3 |

DOIs | |

Publication status | Published - Sep 1969 |

### Fingerprint

### ASJC Scopus subject areas

- Statistics and Probability
- Analysis
- Mathematics(all)

### Cite this

*Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete*,

*12*(3), 185-222. https://doi.org/10.1007/BF00534840

**Information sources with different cost scales and the principle of conservation of entropy.** / Csiszár, I.; Katona, G.; Tusnády, G.

Research output: Contribution to journal › Article

*Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete*, vol. 12, no. 3, pp. 185-222. https://doi.org/10.1007/BF00534840

}

TY - JOUR

T1 - Information sources with different cost scales and the principle of conservation of entropy

AU - Csiszár, I.

AU - Katona, G.

AU - Tusnády, G.

PY - 1969/9

Y1 - 1969/9

N2 - The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.

AB - The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.

UR - http://www.scopus.com/inward/record.url?scp=4243884957&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=4243884957&partnerID=8YFLogxK

U2 - 10.1007/BF00534840

DO - 10.1007/BF00534840

M3 - Article

AN - SCOPUS:4243884957

VL - 12

SP - 185

EP - 222

JO - Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete

JF - Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete

SN - 0178-8051

IS - 3

ER -