No-reference video quality assessment via pretrained CNN and LSTM networks

Domonkos Varga, T. Szirányi

Research output: Contribution to journalArticle

Abstract

A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.

Original languageEnglish
JournalSignal, Image and Video Processing
DOIs
Publication statusPublished - Jan 1 2019

Fingerprint

Neural networks
Time series
Long short-term memory
Experiments

Keywords

  • Convolutional neural network
  • Long short-term memory
  • No-reference video quality assessment

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

No-reference video quality assessment via pretrained CNN and LSTM networks. / Varga, Domonkos; Szirányi, T.

In: Signal, Image and Video Processing, 01.01.2019.

Research output: Contribution to journalArticle

@article{1d0c817bd4a94c239243e1bf0d1b866e,
title = "No-reference video quality assessment via pretrained CNN and LSTM networks",
abstract = "A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.",
keywords = "Convolutional neural network, Long short-term memory, No-reference video quality assessment",
author = "Domonkos Varga and T. Szir{\'a}nyi",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/s11760-019-01510-8",
language = "English",
journal = "Signal, Image and Video Processing",
issn = "1863-1703",
publisher = "Springer London",

}

TY - JOUR

T1 - No-reference video quality assessment via pretrained CNN and LSTM networks

AU - Varga, Domonkos

AU - Szirányi, T.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.

AB - A general-purpose no-reference video quality assessment algorithm based on a long short-term memory (LSTM) network and a pretrained convolutional neural network (CNN) is introduced. Considering video sequences as a time series of deep features extracted with the help of a CNN, an LSTM network is trained to predict subjective quality scores. In contrast to previous methods, the resulting algorithm was trained on the recently published Konstanz Natural Video Quality Database (KoNViD-1k), which is the only publicly available database that contains sequences with authentic distortions. The results of experiments on KoNViD-1k demonstrate that the proposed method outperforms other state-of-the-art algorithms. Furthermore, these results are also confirmed using tests on the LIVE Video Quality Assessment Database, which consists of artificially distorted videos.

KW - Convolutional neural network

KW - Long short-term memory

KW - No-reference video quality assessment

UR - http://www.scopus.com/inward/record.url?scp=85067180146&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067180146&partnerID=8YFLogxK

U2 - 10.1007/s11760-019-01510-8

DO - 10.1007/s11760-019-01510-8

M3 - Article

AN - SCOPUS:85067180146

JO - Signal, Image and Video Processing

JF - Signal, Image and Video Processing

SN - 1863-1703

ER -