Reducing human efforts in video segmentation annotation with reinforcement learning

Viktor Varga, András Lőrincz

Research output: Contribution to journalArticle


Manual annotation of video segmentation datasets requires an immense amount of human effort, thus, reduction of human annotation costs is an active topic of research. While many papers deal with the propagation of masks through frames of a video, only a few results attempt to optimize annotation task selection. In this paper we present a deep learning based solution to the latter problem and train it using Reinforcement Learning. Our approach utilizes a modified version of the Dueling Deep Q-Network sharing weight parameters across the temporal axis of the video. This technique enables the trained agent to select annotation tasks from the whole video. We evaluate our annotation task selection method by means of a hierarchical supervoxel segmentation based mask propagation algorithm.

Original languageEnglish
Pages (from-to)247-258
Number of pages12
Publication statusPublished - Sep 10 2020


  • Interactive annotation
  • Reinforcement learning
  • Video segmentation

ASJC Scopus subject areas

  • Computer Science Applications
  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Reducing human efforts in video segmentation annotation with reinforcement learning'. Together they form a unique fingerprint.

  • Cite this