Information sources with different cost scales and the principle of conservation of entropy

I. Csiszár, G. Katona, G. Tusnády

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.

Original languageEnglish
Pages (from-to)185-222
Number of pages38
JournalZeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete
Volume12
Issue number3
DOIs
Publication statusPublished - Sep 1969

Fingerprint

Conservation
Entropy
Costs
Encoding
Coding
Theorem
Information sources
Unit
Concepts

ASJC Scopus subject areas

  • Statistics and Probability
  • Analysis
  • Mathematics(all)

Cite this

Information sources with different cost scales and the principle of conservation of entropy. / Csiszár, I.; Katona, G.; Tusnády, G.

In: Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, Vol. 12, No. 3, 09.1969, p. 185-222.

Research output: Contribution to journalArticle

@article{a59848db21b1400fabe5233f2209a13d,
title = "Information sources with different cost scales and the principle of conservation of entropy",
abstract = "The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the {"}principle of conservation of entropy{"} and the {"}noiseless coding theorem{"} are proved under very general conditions.",
author = "I. Csisz{\'a}r and G. Katona and G. Tusn{\'a}dy",
year = "1969",
month = "9",
doi = "10.1007/BF00534840",
language = "English",
volume = "12",
pages = "185--222",
journal = "Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete",
issn = "0178-8051",
publisher = "Springer New York",
number = "3",

}

TY - JOUR

T1 - Information sources with different cost scales and the principle of conservation of entropy

AU - Csiszár, I.

AU - Katona, G.

AU - Tusnády, G.

PY - 1969/9

Y1 - 1969/9

N2 - The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.

AB - The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.

UR - http://www.scopus.com/inward/record.url?scp=4243884957&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=4243884957&partnerID=8YFLogxK

U2 - 10.1007/BF00534840

DO - 10.1007/BF00534840

M3 - Article

AN - SCOPUS:4243884957

VL - 12

SP - 185

EP - 222

JO - Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete

JF - Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete

SN - 0178-8051

IS - 3

ER -