The aim of this paper is to provide a mathematically rigorous and sufficiently general treatment of the basic information-theoretic problems concerning sources with symbols of different costs and noiseless coding in a general sense. The main new concepts defined in this paper are the entropy rate (entropy per unit cost) of a source with respect to a stochastic cost scale and the encoding (in particular decodable encoding) of a source in a general sense. On the basis of these concepts, we prove some general theorems on the relation of entropy rates with respect to different cost scales and on the effect of encoding to the entropy rate. In particular, the "principle of conservation of entropy" and the "noiseless coding theorem" are proved under very general conditions.
|Number of pages||38|
|Journal||Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete|
|Publication status||Published - Sep 1 1969|
ASJC Scopus subject areas
- Statistics and Probability