Why least squares and maximum entropy? An axiomatic approach to inverse problems

Research output: Contribution to conferencePaper

305 Citations (Scopus)

Abstract

Summary form only given, as follows. For the set S of all real-valued or all probability mass functions with a given finite domain, all conceivable rules for selecting an element of a feasible subset of S, determined by linear constraints, are considered, and those satisfying certain natural postulates are characterized. Two basic postulates imply that the selection should minimize some function defined on S which, if a prior guess is available, is a measure of distance from the latter. It is shown how invariance properties and a transitivity postulate restrict the class of permissible distances, leading to characterizations of some well-known families of distances and also some new ones. As corollaries, unique characterizations of the methods of least squares and minimum discrimination information are arrived at. The latter are also uniquely characterized by a postulate of composition consistence. As a special case, a unique characterization of the method of maximum entropy from a small set of natural axioms is obtained.

Original languageEnglish
Number of pages1
Publication statusPublished - Dec 1 1990
Event1990 IEEE International Symposium on Information Theory - San Diego, CA, USA
Duration: Jan 14 1990Jan 19 1990

Other

Other1990 IEEE International Symposium on Information Theory
CitySan Diego, CA, USA
Period1/14/901/19/90

    Fingerprint

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Csiszar, I. (1990). Why least squares and maximum entropy? An axiomatic approach to inverse problems. Paper presented at 1990 IEEE International Symposium on Information Theory, San Diego, CA, USA, .