Why least squares and maximum entropy? An axiomatic approach to inverse problems

Research output: Paper

305 Citations (Scopus)

Abstract

Summary form only given, as follows. For the set S of all real-valued or all probability mass functions with a given finite domain, all conceivable rules for selecting an element of a feasible subset of S, determined by linear constraints, are considered, and those satisfying certain natural postulates are characterized. Two basic postulates imply that the selection should minimize some function defined on S which, if a prior guess is available, is a measure of distance from the latter. It is shown how invariance properties and a transitivity postulate restrict the class of permissible distances, leading to characterizations of some well-known families of distances and also some new ones. As corollaries, unique characterizations of the methods of least squares and minimum discrimination information are arrived at. The latter are also uniquely characterized by a postulate of composition consistence. As a special case, a unique characterization of the method of maximum entropy from a small set of natural axioms is obtained.

Original languageEnglish
Number of pages1
Publication statusPublished - dec. 1 1990
Event1990 IEEE International Symposium on Information Theory - San Diego, CA, USA
Duration: jan. 14 1990jan. 19 1990

Other

Other1990 IEEE International Symposium on Information Theory
CitySan Diego, CA, USA
Period1/14/901/19/90

ASJC Scopus subject areas

  • Engineering(all)

Fingerprint Dive into the research topics of 'Why least squares and maximum entropy? An axiomatic approach to inverse problems'. Together they form a unique fingerprint.

  • Cite this

    Csiszar, I. (1990). Why least squares and maximum entropy? An axiomatic approach to inverse problems. Paper presented at 1990 IEEE International Symposium on Information Theory, San Diego, CA, USA, .