Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment

Domonkos Varga, Dietmar Saupe, T. Szirányi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

This paper presents a blind image quality assessment (BIQA) method based on deep learning with convolutional neural networks (CNN). Our method is trained on full and arbitrarily sized images rather than small image patches or resized input images as usually done in CNNs for image classification and quality assessment. The resolution independence is achieved by pyramid pooling. This work is the first that applies a fine-tuned residual deep learning network (ResNet-101) to BIQA. The training is carried out on a new and very large, labeled dataset of 10, 073 images (KonIQ-10k) that contains quality rating histograms besides the mean opinion scores (MOS). In contrast to previous methods we do not train to approximate the MOS directly, but rather use the distributions of scores. Experiments were carried out on three benchmark image quality databases. The results showed clear improvements of the accuracy of the estimated MOS values, compared to current state-of-the-art algorithms. We also report on the quality of the estimation of the score distributions.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Multimedia and Expo, ICME 2018
PublisherIEEE Computer Society
ISBN (Electronic)9781538617373
DOIs
Publication statusPublished - Oct 8 2018
Event2018 IEEE International Conference on Multimedia and Expo, ICME 2018 - San Diego, United States
Duration: Jul 23 2018Jul 27 2018

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2018-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2018 IEEE International Conference on Multimedia and Expo, ICME 2018
CountryUnited States
CitySan Diego
Period7/23/187/27/18

Fingerprint

Image quality
Image classification
Neural networks
Experiments
Deep learning

Keywords

  • Blind image quality assessment
  • CNN
  • deep learning
  • spatial pyramid pooling

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Varga, D., Saupe, D., & Szirányi, T. (2018). Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment. In 2018 IEEE International Conference on Multimedia and Expo, ICME 2018 [8486528] (Proceedings - IEEE International Conference on Multimedia and Expo; Vol. 2018-July). IEEE Computer Society. https://doi.org/10.1109/ICME.2018.8486528

Deeprn : A Content Preserving Deep Architecture for Blind Image Quality Assessment. / Varga, Domonkos; Saupe, Dietmar; Szirányi, T.

2018 IEEE International Conference on Multimedia and Expo, ICME 2018. IEEE Computer Society, 2018. 8486528 (Proceedings - IEEE International Conference on Multimedia and Expo; Vol. 2018-July).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Varga, D, Saupe, D & Szirányi, T 2018, Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment. in 2018 IEEE International Conference on Multimedia and Expo, ICME 2018., 8486528, Proceedings - IEEE International Conference on Multimedia and Expo, vol. 2018-July, IEEE Computer Society, 2018 IEEE International Conference on Multimedia and Expo, ICME 2018, San Diego, United States, 7/23/18. https://doi.org/10.1109/ICME.2018.8486528
Varga D, Saupe D, Szirányi T. Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment. In 2018 IEEE International Conference on Multimedia and Expo, ICME 2018. IEEE Computer Society. 2018. 8486528. (Proceedings - IEEE International Conference on Multimedia and Expo). https://doi.org/10.1109/ICME.2018.8486528
Varga, Domonkos ; Saupe, Dietmar ; Szirányi, T. / Deeprn : A Content Preserving Deep Architecture for Blind Image Quality Assessment. 2018 IEEE International Conference on Multimedia and Expo, ICME 2018. IEEE Computer Society, 2018. (Proceedings - IEEE International Conference on Multimedia and Expo).
@inproceedings{1635bb5ba7964967852c0ad46f52bf6f,
title = "Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment",
abstract = "This paper presents a blind image quality assessment (BIQA) method based on deep learning with convolutional neural networks (CNN). Our method is trained on full and arbitrarily sized images rather than small image patches or resized input images as usually done in CNNs for image classification and quality assessment. The resolution independence is achieved by pyramid pooling. This work is the first that applies a fine-tuned residual deep learning network (ResNet-101) to BIQA. The training is carried out on a new and very large, labeled dataset of 10, 073 images (KonIQ-10k) that contains quality rating histograms besides the mean opinion scores (MOS). In contrast to previous methods we do not train to approximate the MOS directly, but rather use the distributions of scores. Experiments were carried out on three benchmark image quality databases. The results showed clear improvements of the accuracy of the estimated MOS values, compared to current state-of-the-art algorithms. We also report on the quality of the estimation of the score distributions.",
keywords = "Blind image quality assessment, CNN, deep learning, spatial pyramid pooling",
author = "Domonkos Varga and Dietmar Saupe and T. Szir{\'a}nyi",
year = "2018",
month = "10",
day = "8",
doi = "10.1109/ICME.2018.8486528",
language = "English",
series = "Proceedings - IEEE International Conference on Multimedia and Expo",
publisher = "IEEE Computer Society",
booktitle = "2018 IEEE International Conference on Multimedia and Expo, ICME 2018",

}

TY - GEN

T1 - Deeprn

T2 - A Content Preserving Deep Architecture for Blind Image Quality Assessment

AU - Varga, Domonkos

AU - Saupe, Dietmar

AU - Szirányi, T.

PY - 2018/10/8

Y1 - 2018/10/8

N2 - This paper presents a blind image quality assessment (BIQA) method based on deep learning with convolutional neural networks (CNN). Our method is trained on full and arbitrarily sized images rather than small image patches or resized input images as usually done in CNNs for image classification and quality assessment. The resolution independence is achieved by pyramid pooling. This work is the first that applies a fine-tuned residual deep learning network (ResNet-101) to BIQA. The training is carried out on a new and very large, labeled dataset of 10, 073 images (KonIQ-10k) that contains quality rating histograms besides the mean opinion scores (MOS). In contrast to previous methods we do not train to approximate the MOS directly, but rather use the distributions of scores. Experiments were carried out on three benchmark image quality databases. The results showed clear improvements of the accuracy of the estimated MOS values, compared to current state-of-the-art algorithms. We also report on the quality of the estimation of the score distributions.

AB - This paper presents a blind image quality assessment (BIQA) method based on deep learning with convolutional neural networks (CNN). Our method is trained on full and arbitrarily sized images rather than small image patches or resized input images as usually done in CNNs for image classification and quality assessment. The resolution independence is achieved by pyramid pooling. This work is the first that applies a fine-tuned residual deep learning network (ResNet-101) to BIQA. The training is carried out on a new and very large, labeled dataset of 10, 073 images (KonIQ-10k) that contains quality rating histograms besides the mean opinion scores (MOS). In contrast to previous methods we do not train to approximate the MOS directly, but rather use the distributions of scores. Experiments were carried out on three benchmark image quality databases. The results showed clear improvements of the accuracy of the estimated MOS values, compared to current state-of-the-art algorithms. We also report on the quality of the estimation of the score distributions.

KW - Blind image quality assessment

KW - CNN

KW - deep learning

KW - spatial pyramid pooling

UR - http://www.scopus.com/inward/record.url?scp=85054363836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85054363836&partnerID=8YFLogxK

U2 - 10.1109/ICME.2018.8486528

DO - 10.1109/ICME.2018.8486528

M3 - Conference contribution

AN - SCOPUS:85054363836

T3 - Proceedings - IEEE International Conference on Multimedia and Expo

BT - 2018 IEEE International Conference on Multimedia and Expo, ICME 2018

PB - IEEE Computer Society

ER -