Source-conditioned visual quality assessment

CodecArena: Codec Quality Assessment via Visual Reinforcement Learning

Jiaye Fu^1,2,* Weiqi Li^1,3,* Qiankun Gao¹ Yanchen Zhao² Xiandong Meng³ Jian Zhang¹ Siwei Ma² Jiaqi Zhang^2,†

¹ School of Electronic and Computer Engineering, Peking University ² School of Computer Science, Peking University ³ Pengcheng Laboratory

^* Equal contribution ^† Corresponding author

Paper Code Hugging Face

Facet-GRPO framework with elastic temporal queries, facet verdicts, rewards, and policy optimization — **CodecArena with Facet-GRPO.** An elastic temporal context query presents the reference and two reconstructions from a single still image to a multi-frame clip. The policy emits a structured facet verdict, and preference, facet-anchored, and calibration rewards drive the Facet-GRPO update.

Abstract

Video coding is advancing into the low and ultra-low bitrate regime, driven by end-to-end codecs that replace the hand-crafted pipeline with jointly optimized neural networks and generative codecs that exploit the priors of video generation models. Yet the dominant metrics, LPIPS and DISTS, measure feature and texture similarity rather than content fidelity: a reconstruction that hallucinates a wrong face or blurs text into convincing strokes can still score well, even when a human rejects it instantly. To address this, we propose CodecArena, the first vision-language framework for video coding quality assessment, casting codec evaluation as source-conditioned comparative reasoning between a reference and its reconstructions. We optimize CodecArena with Facet-GRPO, a visual reinforcement learning scheme that aligns pairwise codec preferences while grounding the verdict in five fidelity facets: identity, objects, text, texture, and temporal consistency. Its facet-anchored reward uses automatically derived facet directions as weak anchors, rather than human per-facet labels, to prevent any single sub-score from dominating the holistic preference and to yield interpretable fine-grained quality judgments. To support training and evaluation in this underexplored regime, we construct two complementary resources: CodecArena-1K, a fully automatic preference dataset of 1,500 comparison groups built from traditional, neural, and generative codec reconstructions with fused vision-language and objective supervision; and CodecArena-Bench, a human-ranked benchmark with source-disjoint videos for fair out-of-domain evaluation. Extensive experiments demonstrate that CodecArena achieves state-of-the-art agreement with human judgments on source-disjoint content across diverse codecs and bitrates, surpassing perceptual metrics and prior vision-language evaluators.

Part 1 · The problem

Sharp is not the same as faithful.

LPIPS and DISTS average feature distances over the whole frame. At low and ultra-low bitrates, that can reward globally realistic detail while missing the identity shift, unreadable text, fabricated texture, or temporal drift that makes a human reject the reconstruction.

The bias is self-reinforcing because many generative codecs are trained with LPIPS-style perceptual losses. Evaluating them with the same insensitive signal can reward a reconstruction for exploiting the metric rather than preserving the source. CodecArena instead compares the source with its reconstructions and asks what information remains faithful across five complementary facets.

Matched-bitrate comparison where GLC-Video receives better LPIPS and DISTS scores but corrupts face identity and on-screen text, while DCVC-RT preserves both and is preferred by CodecArena — LPIPS and DISTS rank GLC-Video above DCVC-RT, yet GLC-Video corrupts the face identity (top) and on-screen text (bottom) that DCVC-RT preserves. CodecArena instead prefers DCVC-RT, matching human judgment. ↓ / ↑ indicates whether lower or higher is better.

Part 2 · Our method

Facet-GRPO: grounded preference learning

Human rankings provide reliable holistic preferences, but they do not determine how a model should score identity, objects, text, texture, and temporal consistency separately. Facet-GRPO closes this gap with automatically derived facet directions, producing fine-grained judgments without human per-facet labels.

1 Elastic query 2 Facet verdict 3 Structured rewards 4 GRPO update

The failure mode

A correct winner can hide the wrong reason.

A holistic preference reward only constrains which side wins. Because the final quality is averaged over facets, the policy can inflate one salient score—often synthesized texture—while a face or caption is corrupted. The ranking may be correct even though the explanation and sub-scores are not.

Why group-relative optimization

The training objective matches the comparison task.

GRPO learns from the relative quality of several sampled verdicts for the same query. It removes the explicit critic used by PPO, reducing compute overhead and avoiding instability from a learned value function, while a frozen reference model and KL term keep policy updates controlled.

One reward is not enough: each term closes a different loophole

Preference

Bradley–Terry latent qualities reward the correct A/B choice in proportion to preference confidence.

Constrains the ordering

Calibration

Side-level scores are matched to fused objective quality, so confidence has a meaningful magnitude.

Constrains the scale

Facet anchoring

Every clearly separated facet must follow its automatically derived direction; no one sub-score can carry the comparison.

Constrains the decomposition

Reasoning + format

Explicit visual reasoning precedes a parseable verdict, preventing collapse into unsupported score-only outputs.

Constrains the evidence

Elastic temporal context

One judge, from a still image to a full clip.

Training samples K ∈ {1, 3, 5, 7}. At K=1, the temporal facet is marked not scorable and removed from the anchor set; at K>1, all five facets apply. The same policy therefore handles image and video assessment and is evaluated at K=14 to test temporal extrapolation beyond the training context.

Coding-oriented supervision

1,500CodecArena-1K groups

1,050 / 450training / validation groups

80source-disjoint benchmark videos

10 + 5experts + non-domain viewers

Fully automatic training labels

Kimi-K2.6 and Opus-4.7 pairwise judgments are fused with seven objective signals and a within-codec rate-monotonicity constraint. Human rankings are reserved for CodecArena-Bench evaluation.

Multi-codec groups

Re-encode natural clips with HEVC, VVC, DCVC-DC, DCVC-RT, and GLC-Video across medium-low to ultra-low bitrates.

Automatic annotation

Fuse pairwise VLM judgments with face identity, perceptual, text, no-reference, and temporal quality signals.

Human-ranked benchmark

Aggregate rankings from coding experts and non-domain viewers on source-disjoint content for out-of-domain evaluation.

Part 3 · Key findings

Closer agreement with human judgment

Both CodecArena variants occupy the top SRCC positions at every temporal granularity and rank first and second in pairwise accuracy. The Qwen3-VL-8B judge remains strong at K=14 despite training only with K≤7.

0.895SRCC · Qwen3-VL-8B · K=7

0.923PLCC · Qwen3-VL-8B · K=7

94.9%Pairwise accuracy · K=7

Pairwise examples

More Results

Each Test-2 example shows a reference and a randomly ordered A/B pair. The tables report every score used for the comparison and the reconstruction selected by each evaluator; ↑ / ↓ indicates whether higher or lower is better.

Image · K=1

Identity fidelity

Human preference · B

Reference image of a man standing outdoors — Reference

Reconstruction A with altered facial identity — A · human rank 6.6

Preferred reconstruction B preserving the subject identity more faithfully — B · human rank 3.0 **Preferred**

Identity fidelity scores for reconstructions A and B
Evaluator	A	B	Choice
Human mean rank ↓	6.6	3.0	B · aligned
CodecArena ↑	0.20	0.53	B · aligned
LPIPS ↓	0.2264	0.3595	A · not aligned
DISTS ↓	0.1508	0.2421	A · not aligned
Q-Insight ↑	3.37	2.80	A · not aligned

A appears locally detailed but changes the subject's face. LPIPS, DISTS, and Q-Insight favor A; human ranking and CodecArena choose the more identity-faithful B.

Video · K=7

Text fidelity

Human preference · A

Reference video frame containing a poem in white text — Reference

Preferred reconstruction A retaining readable poem text — A · human rank 4.0 **Preferred**

Reconstruction B corrupting the poem letterforms — B · human rank 7.1

Text fidelity scores for reconstructions A and B
Evaluator	A	B	Choice
Human mean rank ↓	4.0	7.1	A · aligned
CodecArena ↑	0.62	0.50	A · aligned
LPIPS ↓	0.2867	0.1318	B · not aligned
DISTS ↓	0.1778	0.0991	B · not aligned
VQ-Insight ↑	66	68	B · not aligned

B preserves a plausible scene appearance but corrupts the poem into malformed characters. LPIPS, DISTS, and VQ-Insight favor B; human ranking and CodecArena choose A's readable text.

Video · K=7

Identity and temporal state

Human preference · B

Reference video frame of two children in costume — Reference

Reconstruction A with a sharper but altered facial state — A · human rank 6.8

Preferred reconstruction B closer to the reference facial state — B · human rank 3.1 **Preferred**

Identity and temporal state scores for reconstructions A and B
Evaluator	A	B	Choice
Human mean rank ↓	6.8	3.1	B · aligned
CodecArena ↑	0.34	0.64	B · aligned
LPIPS ↓	0.1234	0.1392	A · not aligned
DISTS ↓	0.1212	0.1443	A · not aligned
VQ-Insight ↑	62	56	A · not aligned

A looks sharper in isolation but changes the child's facial state. LPIPS, DISTS, and VQ-Insight favor A; human ranking and CodecArena prefer B's closer source fidelity.

Resources

Use CodecArena

Paper, implementation, and model/data resources are collected below. The model/data link currently returns to the project page and can be replaced when its final destination is ready.

PaperMethod, experiments, and analysisOpen ↗ CodeTraining and evaluation toolkitOpen ↗ Hugging FaceModels and data collectionsOpen ↗

BibTeX

Neutral project citation placeholder

@article{fu2026codecarena,
  title={CodecArena: Codec Quality Assessment via Visual Reinforcement Learning},
  author={Fu, Jiaye and Li, Weiqi and Gao, Qiankun and Zhao, Yanchen and Meng, Xiandong and Zhang, Jian and Ma, Siwei and Zhang, Jiaqi},
  year={2026}
}