Technical report on visual quality assessment for frame interpolation

Hui Men; Hanhe Lin; Vlad Hosu; Daniel Maurer; Andrés Bruhn; Dietmar Saupe

doi:10.48550/arXiv.1901.05362

Technical report on visual quality assessment for frame interpolation

Hui Men
, Hanhe Lin
, Vlad Hosu
, Daniel Maurer
, Andrés Bruhn
, Dietmar Saupe

Computing

Research output: Working paper/Preprint › Preprint

35 Downloads (Pure)

Abstract

Current benchmarks for optical flow algorithms evaluate the estimation quality by comparing their predicted flow field with the ground truth, and additionally may compare interpolated frames, based on these predictions, with the correct frames from the actual image sequences. For the latter comparisons, objective measures such as mean square errors are applied. However, for applications like image interpolation, the expected user's quality of experience cannot be fully deduced from such simple quality measures. Therefore, we conducted a subjective quality assessment study by crowdsourcing for the interpolated images provided in one of the optical flow benchmarks, the Middlebury benchmark. We used paired comparisons with forced choice and reconstructed absolute quality scale values according to Thurstone's model using the classical least squares method. The results give rise to a re-ranking of 141 participating algorithms w.r.t. visual quality of interpolated frames mostly based on optical flow estimation. Our re-ranking result shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks.

Original language	English
Place of Publication	Cornell University
Publisher	arXiv
Number of pages	11
DOIs	https://doi.org/10.48550/arXiv.1901.05362
Publication status	Published - 26 Mar 2019

Keywords

visual quality assessment
optical flow
frame interpolation

Access to Document

10.48550/arXiv.1901.05362

Preprint
The URI http://arxiv.org/licenses/nonexclusive-distrib/1.0/ is used to record the fact that the submitter granted the following license to arXiv.org on submission of an article: I grant arXiv.org a perpetual, non-exclusive license to distribute this article. I certify that I have the right to grant this license. I understand that submissions cannot be completely removed once accepted. I understand that arXiv.org reserves the right to reclassify or reject any submission.
Submitted manuscript, 7.14 MBLicence: Other

Cite this

@techreport{740631ef950942b1985eabc849c2eb90,

title = "Technical report on visual quality assessment for frame interpolation",

abstract = "Current benchmarks for optical flow algorithms evaluate the estimation quality by comparing their predicted flow field with the ground truth, and additionally may compare interpolated frames, based on these predictions, with the correct frames from the actual image sequences. For the latter comparisons, objective measures such as mean square errors are applied. However, for applications like image interpolation, the expected user's quality of experience cannot be fully deduced from such simple quality measures. Therefore, we conducted a subjective quality assessment study by crowdsourcing for the interpolated images provided in one of the optical flow benchmarks, the Middlebury benchmark. We used paired comparisons with forced choice and reconstructed absolute quality scale values according to Thurstone's model using the classical least squares method. The results give rise to a re-ranking of 141 participating algorithms w.r.t. visual quality of interpolated frames mostly based on optical flow estimation. Our re-ranking result shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks. ",

keywords = "visual quality assessment, optical flow, frame interpolation",

author = "Hui Men and Hanhe Lin and Vlad Hosu and Daniel Maurer and Andr{\'e}s Bruhn and Dietmar Saupe",

note = "Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Projektnummer 251654672 – TRR 161 (Project A05 and B04).",

year = "2019",

month = mar,

day = "26",

doi = "10.48550/arXiv.1901.05362",

language = "English",

publisher = "arXiv",

type = "WorkingPaper",

institution = "arXiv",

}

TY - UNPB

T1 - Technical report on visual quality assessment for frame interpolation

AU - Men, Hui

AU - Lin, Hanhe

AU - Hosu, Vlad

AU - Maurer, Daniel

AU - Bruhn, Andrés

AU - Saupe, Dietmar

N1 - Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Projektnummer 251654672 – TRR 161 (Project A05 and B04).

PY - 2019/3/26

Y1 - 2019/3/26

N2 - Current benchmarks for optical flow algorithms evaluate the estimation quality by comparing their predicted flow field with the ground truth, and additionally may compare interpolated frames, based on these predictions, with the correct frames from the actual image sequences. For the latter comparisons, objective measures such as mean square errors are applied. However, for applications like image interpolation, the expected user's quality of experience cannot be fully deduced from such simple quality measures. Therefore, we conducted a subjective quality assessment study by crowdsourcing for the interpolated images provided in one of the optical flow benchmarks, the Middlebury benchmark. We used paired comparisons with forced choice and reconstructed absolute quality scale values according to Thurstone's model using the classical least squares method. The results give rise to a re-ranking of 141 participating algorithms w.r.t. visual quality of interpolated frames mostly based on optical flow estimation. Our re-ranking result shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks.

AB - Current benchmarks for optical flow algorithms evaluate the estimation quality by comparing their predicted flow field with the ground truth, and additionally may compare interpolated frames, based on these predictions, with the correct frames from the actual image sequences. For the latter comparisons, objective measures such as mean square errors are applied. However, for applications like image interpolation, the expected user's quality of experience cannot be fully deduced from such simple quality measures. Therefore, we conducted a subjective quality assessment study by crowdsourcing for the interpolated images provided in one of the optical flow benchmarks, the Middlebury benchmark. We used paired comparisons with forced choice and reconstructed absolute quality scale values according to Thurstone's model using the classical least squares method. The results give rise to a re-ranking of 141 participating algorithms w.r.t. visual quality of interpolated frames mostly based on optical flow estimation. Our re-ranking result shows the necessity of visual quality assessment as another evaluation metric for optical flow and frame interpolation benchmarks.

KW - visual quality assessment

KW - optical flow

KW - frame interpolation

U2 - 10.48550/arXiv.1901.05362

DO - 10.48550/arXiv.1901.05362

M3 - Preprint

BT - Technical report on visual quality assessment for frame interpolation

PB - arXiv

CY - Cornell University

ER -

Technical report on visual quality assessment for frame interpolation

Abstract

Keywords

Access to Document

Fingerprint

Cite this