Improved determination of the best fitting sine wave in ADC testing

István Kollár, Jerome J. Blair

Research output: Contribution to journalConference article

6 Citations (Scopus)


The sine wave test of an ADC means to excite the ADC with a pure sine wave, look for the sine wave which best fits the output in least squares sense, and analyze the difference. This is described in the IEEE standards 1241-2000 and 1057-1994. Least squares is the 'best' fitting method most of us can imagine, and it yields very good results indeed. Its known properties are achieved when the error (the deviation of the samples from the true sine wave) is random, white (the error samples are all independent), with zero mean Gaussian distribution. Then the LS fit coincides with the maximum likelihood estimate of the parameters. However, in sine wave testing of ADC's these assumptions are far from being true. The quantization error is partly deterministic, and the sample values are strongly interdependent. This makes the sine fit worse than expected, and since small changes in the sine wave affect the residuals significantly, especially close to the peaks, ADC error analysis may become misleading. Processing of the residuals (e.g. the calculation of the effective number of bits, ENOB) can exhibit serious errors. This paper describes this phenomenon, analyses its consequences, and suggests modified processing of samples and residuals to reduce the errors to negligible level.

Original languageEnglish
Pages (from-to)829-834
Number of pages6
JournalConference Record - IEEE Instrumentation and Measurement Technology Conference
Publication statusPublished - Oct 8 2004
EventProceedings of the 21st IEEE Instrumentation and Measurement Technology Conference, IMTC/04 - Como, Italy
Duration: May 18 2004May 20 2004


  • ADC test
  • Analog-to-digtial converter
  • Effective number of bits
  • ENOB
  • IEEE Standard 1057-1994
  • IEEE Standard 1241-2000
  • Least squares
  • Sine wave fitting

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Improved determination of the best fitting sine wave in ADC testing'. Together they form a unique fingerprint.

  • Cite this