aom: rtc: Condition QP adjustment on rc->q_1/2_frame > 0

From 18925daf07b4b7781f8cef73424d1bed734d8ec1 Mon Sep 17 00:00:00 2001
From: Marco Paniconi <[EMAIL REDACTED]>
Date: Thu, 30 May 2024 20:32:48 -0700
Subject: [PATCH] rtc: Condition QP adjustment on rc->q_1/2_frame > 0

This fix only affects the initial QPs after very first
frame for TL enhancement frames in CBR mode.
Move the check that rc->q_1/2_frame is set to the top
of the adjustment logic, otherwise if there are dropped
frames after the very first frame, the QP can end up
with oscillation on temporal enhancement frames.

Because of some further logic/adjustments this
behavior would likely only happen when the TL enhancement
frame_bandwidth is not smaller then TL0 frame_bandwidth.

Bug: b/343429192
Change-Id: I121934c2c7e7522e4e40762191ed3005437841c2
---
 av1/encoder/ratectrl.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/av1/encoder/ratectrl.c b/av1/encoder/ratectrl.c
index af707be5b..86dbdcaf7 100644
--- a/av1/encoder/ratectrl.c
+++ b/av1/encoder/ratectrl.c
@@ -588,18 +588,19 @@ static int adjust_q_cbr(const AV1_COMP *cpi, int q, int active_worst_quality,
   // Apply some control/clamp to QP under certain conditions.
   // Delay the use of the clamping for svc until after num_temporal_layers,
   // to make they have been set for each temporal layer.
+  // Check for rc->q_1/2_frame > 0 in case they have not been set due to
+  // dropped frames.
   if (!frame_is_intra_only(cm) && rc->frames_since_key > 1 &&
+      rc->q_1_frame > 0 && rc->q_2_frame > 0 &&
       (!cpi->ppi->use_svc ||
        svc->current_superframe > (unsigned int)svc->number_temporal_layers) &&
       !change_target_bits_mb && !cpi->rc.rtc_external_ratectrl &&
       (!cpi->oxcf.rc_cfg.gf_cbr_boost_pct ||
        !(refresh_frame->alt_ref_frame || refresh_frame->golden_frame))) {
     // If in the previous two frames we have seen both overshoot and undershoot
-    // clamp Q between the two. Check for rc->q_1/2_frame > 0 in case they have
-    // not been set due to dropped frames.
+    // clamp Q between the two.
     if (rc->rc_1_frame * rc->rc_2_frame == -1 &&
-        rc->q_1_frame != rc->q_2_frame && rc->q_1_frame > 0 &&
-        rc->q_2_frame > 0 && !overshoot_buffer_low) {
+        rc->q_1_frame != rc->q_2_frame && !overshoot_buffer_low) {
       int qclamp = clamp(q, AOMMIN(rc->q_1_frame, rc->q_2_frame),
                          AOMMAX(rc->q_1_frame, rc->q_2_frame));
       // If the previous frame had overshoot and the current q needs to