8-cut/docs/plans/2026-04-14-keyframe-crop-modes-implementation.md

Keyframe Crop Modes Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Extend crop keyframes to snapshot ratio and random crop flags, so different sub-clips in a batch inherit different crop settings based on timeline position.

Architecture: Widen the keyframe tuple from (time, center) to (time, center, ratio, rand_portrait, rand_square). All existing keyframe code paths (set, delete, clear, render, apply-at-export, preview-on-scrub) are updated to carry and use the new fields. Diamond rendering on the timeline uses color to indicate which random flags are set.

Tech Stack: Python, PyQt6 (QPainter for diamonds)

---

Task 1: Add a helper to resolve the effective keyframe at a given time

The keyframe lookup pattern (iterate sorted list, take latest where kt <= t + 0.05) is repeated 3 times in main.py. Extract it as a pure function so we can test it and reuse it cleanly with the new wider tuple.

Files:

• Modify: main.py:48-53 (module-level functions area)
• Test: tests/test_utils.py

Step 1: Write the failing tests

Add to tests/test_utils.py:

from main import resolve_keyframe

def test_resolve_keyframe_empty():
    assert resolve_keyframe([], 5.0) is None

def test_resolve_keyframe_before_first():
    kfs = [(3.0, 0.5, None, False, False)]
    assert resolve_keyframe(kfs, 1.0) is None

def test_resolve_keyframe_exact():
    kfs = [(2.0, 0.3, "9:16", True, False)]
    assert resolve_keyframe(kfs, 2.0) == (2.0, 0.3, "9:16", True, False)

def test_resolve_keyframe_between():
    kfs = [
        (1.0, 0.2, None, False, False),
        (5.0, 0.8, "1:1", False, True),
    ]
    assert resolve_keyframe(kfs, 3.0) == (1.0, 0.2, None, False, False)

def test_resolve_keyframe_after_last():
    kfs = [
        (1.0, 0.2, None, False, False),
        (5.0, 0.8, "1:1", False, True),
    ]
    assert resolve_keyframe(kfs, 10.0) == (5.0, 0.8, "1:1", False, True)

def test_resolve_keyframe_tolerance():
    kfs = [(4.0, 0.5, None, True, True)]
    # 4.0 <= 3.96 + 0.05 = 4.01, so it should match
    assert resolve_keyframe(kfs, 3.96) == (4.0, 0.5, None, True, True)

Step 2: Run tests to verify they fail

Run: pytest tests/test_utils.py -k resolve_keyframe -v Expected: FAIL (ImportError — function does not exist yet)

Step 3: Write the implementation

Add to main.py after the format_time function (around line 53):

def resolve_keyframe(
    keyframes: list[tuple[float, float, str | None, bool, bool]],
    t: float,
    tolerance: float = 0.05,
) -> tuple[float, float, str | None, bool, bool] | None:
    """Return the latest keyframe at or before *t*, or None."""
    result = None
    for kf in keyframes:
        if kf[0] <= t + tolerance:
            result = kf
        else:
            break
    return result

Step 4: Run tests to verify they pass

Run: pytest tests/test_utils.py -k resolve_keyframe -v Expected: 6 PASS

Step 5: Commit

git add main.py tests/test_utils.py
git commit -m "feat: add resolve_keyframe helper for widened keyframe tuples"

---

Task 2: Widen keyframe tuple and update storage

Change _crop_keyframes from list[tuple[float, float]] to list[tuple[float, float, str | None, bool, bool]] in both TimelineWidget and MainWindow. Update set_crop_keyframes signature.

Files:

• Modify: main.py:735 (TimelineWidget._crop_keyframes)
• Modify: main.py:786-787 (TimelineWidget.set_crop_keyframes)
• Modify: main.py:1755 (MainWindow._crop_keyframes)

Step 1: Update TimelineWidget

At line 735, change:

self._crop_keyframes: list[tuple[float, float]] = []  # [(time, center)]

to:

self._crop_keyframes: list[tuple[float, float, str | None, bool, bool]] = []

At lines 786-787, change:

def set_crop_keyframes(self, kfs: list[tuple[float, float]]) -> None:
    self._crop_keyframes = kfs

to:

def set_crop_keyframes(self, kfs: list[tuple[float, float, str | None, bool, bool]]) -> None:
    self._crop_keyframes = kfs

Step 2: Update MainWindow

At line 1755, change:

self._crop_keyframes: list[tuple[float, float]] = []  # [(time, center), ...] sorted

to:

self._crop_keyframes: list[tuple[float, float, str | None, bool, bool]] = []  # sorted by time

Step 3: Run existing tests

Run: pytest tests/ -v Expected: All 46 + 6 new = 52 PASS (no existing tests touch keyframes directly)

Step 4: Commit

git add main.py
git commit -m "refactor: widen keyframe tuple to carry ratio and random flags"

---

Task 3: Update keyframe creation to snapshot crop state

When the user clicks the crop bar in lock mode, snapshot the current ratio, rand_portrait, and rand_square into the keyframe.

Files:

• Modify: main.py:2519-2538 (_on_crop_click lock-mode branch)

Step 1: Update the keyframe creation code

At lines 2525-2532, change:

            self._crop_keyframes = [
                (t, c) for t, c in self._crop_keyframes
                if abs(t - play_t) > 0.05
            ]
            self._crop_keyframes.append((play_t, frac))
            self._crop_keyframes.sort()
            self._timeline.set_crop_keyframes(self._crop_keyframes)
            _log(f"Crop keyframe: t={play_t:.2f}s center={frac:.3f} ({len(self._crop_keyframes)} total)")

to:

            ratio_text = self._cmb_portrait.currentText()
            kf_ratio = None if ratio_text == "Off" else ratio_text
            kf_rand_p = self._chk_rand_portrait.isChecked()
            kf_rand_s = self._chk_rand_square.isChecked()
            self._crop_keyframes = [
                kf for kf in self._crop_keyframes
                if abs(kf[0] - play_t) > 0.05
            ]
            self._crop_keyframes.append((play_t, frac, kf_ratio, kf_rand_p, kf_rand_s))
            self._crop_keyframes.sort()
            self._timeline.set_crop_keyframes(self._crop_keyframes)
            _log(f"Crop keyframe: t={play_t:.2f}s center={frac:.3f} ratio={kf_ratio} rp={kf_rand_p} rs={kf_rand_s} ({len(self._crop_keyframes)} total)")

Step 2: Update keyframe deletion filter

At lines 2356-2357, change:

        self._crop_keyframes = [
            (t, c) for t, c in self._crop_keyframes
            if abs(t - time) > 0.05
        ]

to:

        self._crop_keyframes = [
            kf for kf in self._crop_keyframes
            if abs(kf[0] - time) > 0.05
        ]

Step 3: Run existing tests

Run: pytest tests/ -v Expected: All 52 PASS

Step 4: Commit

git add main.py
git commit -m "feat: snapshot ratio and random flags into crop keyframes"

---

Task 4: Update export to apply full keyframe state

Replace the keyframe application loop and random crop logic in _on_export to use the new fields.

Files:

• Modify: main.py:2754-2782 (keyframe application + random crop logic in _on_export)
• Test: tests/test_utils.py

Step 1: Write a test for the export keyframe resolution logic

Add to tests/test_utils.py:

from main import apply_keyframes_to_jobs

def test_apply_keyframes_no_keyframes():
    jobs = [(0.0, "/out/a", None, 0.5), (3.0, "/out/b", None, 0.5)]
    result = apply_keyframes_to_jobs(jobs, [], base_center=0.5, base_ratio=None,
                                     base_rand_p=True, base_rand_s=False)
    # No keyframes: jobs get base values; rand flags come from base
    assert result == [
        (0.0, "/out/a", None, 0.5, True, False),
        (3.0, "/out/b", None, 0.5, True, False),
    ]

def test_apply_keyframes_with_keyframes():
    kfs = [
        (0.0, 0.3, "9:16", True, False),
        (4.0, 0.7, None, False, True),
    ]
    jobs = [
        (0.0, "/out/a", None, 0.5),
        (3.0, "/out/b", None, 0.5),
        (6.0, "/out/c", None, 0.5),
    ]
    result = apply_keyframes_to_jobs(jobs, kfs, base_center=0.5, base_ratio=None,
                                     base_rand_p=False, base_rand_s=False)
    assert result == [
        (0.0, "/out/a", "9:16", 0.3, True, False),
        (3.0, "/out/b", "9:16", 0.3, True, False),
        (6.0, "/out/c", None, 0.7, False, True),
    ]

def test_apply_keyframes_before_first_uses_base():
    kfs = [(5.0, 0.8, "1:1", False, True)]
    jobs = [(1.0, "/out/a", None, 0.5)]
    result = apply_keyframes_to_jobs(jobs, kfs, base_center=0.5, base_ratio="4:5",
                                     base_rand_p=True, base_rand_s=False)
    assert result == [(1.0, "/out/a", "4:5", 0.5, True, False)]

Step 2: Run tests to verify they fail

Run: pytest tests/test_utils.py -k apply_keyframes -v Expected: FAIL (ImportError)

Step 3: Write the apply_keyframes_to_jobs function

Add to main.py after resolve_keyframe:

def apply_keyframes_to_jobs(
    jobs: list[tuple[float, str, str | None, float]],
    keyframes: list[tuple[float, float, str | None, bool, bool]],
    base_center: float,
    base_ratio: str | None,
    base_rand_p: bool,
    base_rand_s: bool,
) -> list[tuple[float, str, str | None, float, bool, bool]]:
    """Resolve each job's crop state from keyframes, returning widened tuples.

    Returns list of (start, path, ratio, center, rand_portrait, rand_square).
    """
    result = []
    for s, o, _r, _c in jobs:
        kf = resolve_keyframe(keyframes, s)
        if kf is not None:
            _, center, ratio, rp, rs = kf
        else:
            center, ratio, rp, rs = base_center, base_ratio, base_rand_p, base_rand_s
        result.append((s, o, ratio, center, rp, rs))
    return result

Step 4: Run tests to verify they pass

Run: pytest tests/test_utils.py -k apply_keyframes -v Expected: 3 PASS

Step 5: Update _on_export to use the new functions

Replace lines 2754-2782 (the keyframe application block + random crop block) with:

            # Apply crop keyframes (or fall back to base state).
            rand_portrait = self._chk_rand_portrait.isChecked()
            rand_square = self._chk_rand_square.isChecked()
            widened = apply_keyframes_to_jobs(
                jobs, self._crop_keyframes,
                base_center=base_center, base_ratio=base_ratio,
                base_rand_p=rand_portrait, base_rand_s=rand_square,
            )

            # Random crop: for each clip whose effective flags are set,
            # ~1 in 3 gets a random ratio applied.
            final_jobs = []
            # Collect indices eligible for random crop, grouped by flag combo.
            portrait_eligible = [i for i, w in enumerate(widened) if w[4]]
            square_eligible = [i for i, w in enumerate(widened) if w[5]]
            rand_indices: set[int] = set()
            if portrait_eligible and n_clips > 1:
                n = max(1, len(portrait_eligible) // 3)
                rand_indices.update(random.sample(portrait_eligible, min(n, len(portrait_eligible))))
            if square_eligible and n_clips > 1:
                n = max(1, len(square_eligible) // 3)
                rand_indices.update(random.sample(square_eligible, min(n, len(square_eligible))))

            for i, (s, o, ratio, center, rp, rs) in enumerate(widened):
                if i in rand_indices:
                    pool = []
                    if rp:
                        pool.append("9:16")
                    if rs:
                        pool.append("1:1")
                    if pool:
                        ratio = random.choice(pool)
                jobs.append((s, o, ratio, center))

            # Replace jobs with the resolved list.
            jobs = jobs[n_clips:]  # drop the original entries, keep the new ones

Note: jobs was built with n_clips entries in the loop above. We append resolved entries and then slice off the originals.

Actually, a cleaner rewrite of the tail — replace the entire block from the keyframe comment through the random crop block with:

            # Apply crop keyframes (or fall back to base state).
            rand_portrait = self._chk_rand_portrait.isChecked()
            rand_square = self._chk_rand_square.isChecked()
            widened = apply_keyframes_to_jobs(
                jobs, self._crop_keyframes,
                base_center=base_center, base_ratio=base_ratio,
                base_rand_p=rand_portrait, base_rand_s=rand_square,
            )

            # Random crop: eligible clips (per their keyframe flags) have
            # ~1 in 3 chance of getting a random ratio applied.
            portrait_eligible = [i for i, w in enumerate(widened) if w[4]]
            square_eligible = [i for i, w in enumerate(widened) if w[5]]
            rand_indices: dict[int, list[str]] = {}
            if portrait_eligible and n_clips > 1:
                n = max(1, len(portrait_eligible) // 3)
                for i in random.sample(portrait_eligible, min(n, len(portrait_eligible))):
                    rand_indices.setdefault(i, []).append("9:16")
            if square_eligible and n_clips > 1:
                n = max(1, len(square_eligible) // 3)
                for i in random.sample(square_eligible, min(n, len(square_eligible))):
                    rand_indices.setdefault(i, []).append("1:1")

            jobs = []
            for i, (s, o, ratio, center, _rp, _rs) in enumerate(widened):
                if i in rand_indices:
                    ratio = random.choice(rand_indices[i])
                jobs.append((s, o, ratio, center))

Step 6: Run all tests

Run: pytest tests/ -v Expected: All PASS

Step 7: Commit

git add main.py tests/test_utils.py
git commit -m "feat: apply keyframe crop modes during export"

---

Task 5: Update diamond rendering with color coding

Color-code timeline keyframe diamonds based on their random flags.

Files:

• Modify: main.py:898-910 (TimelineWidget.paintEvent keyframe diamond section)

Step 1: Replace the diamond rendering block

Replace lines 898-910:

            # ── crop keyframe diamonds ────────────────────────────────────
            if self._crop_keyframes and self._duration > 0:
                for (kt, _kc) in self._crop_keyframes:
                    kx = int(kt / self._duration * w)
                    d = 4  # half-size of diamond
                    ky = h - d - 2  # near bottom of track
                    diamond = QPolygon([
                        QPoint(kx, ky - d), QPoint(kx + d, ky),
                        QPoint(kx, ky + d), QPoint(kx - d, ky),
                    ])
                    p.setBrush(QColor(255, 180, 0))
                    p.setPen(Qt.PenStyle.NoPen)
                    p.drawPolygon(diamond)

with:

            # ── crop keyframe diamonds ────────────────────────────────────
            if self._crop_keyframes and self._duration > 0:
                _KF_GOLD = QColor(255, 180, 0)
                _KF_RED = QColor(220, 60, 60)
                _KF_BLUE = QColor(60, 180, 220)
                for kf in self._crop_keyframes:
                    kt = kf[0]
                    rp = kf[3] if len(kf) > 3 else False
                    rs = kf[4] if len(kf) > 4 else False
                    kx = int(kt / self._duration * w)
                    d = 4  # half-size of diamond
                    ky = h - d - 2  # near bottom of track
                    if rp and rs:
                        # Split diamond: left half red, right half blue
                        left = QPolygon([
                            QPoint(kx, ky - d), QPoint(kx, ky + d),
                            QPoint(kx - d, ky),
                        ])
                        right = QPolygon([
                            QPoint(kx, ky - d), QPoint(kx + d, ky),
                            QPoint(kx, ky + d),
                        ])
                        p.setPen(Qt.PenStyle.NoPen)
                        p.setBrush(_KF_RED)
                        p.drawPolygon(left)
                        p.setBrush(_KF_BLUE)
                        p.drawPolygon(right)
                    else:
                        diamond = QPolygon([
                            QPoint(kx, ky - d), QPoint(kx + d, ky),
                            QPoint(kx, ky + d), QPoint(kx - d, ky),
                        ])
                        if rp:
                            color = _KF_RED
                        elif rs:
                            color = _KF_BLUE
                        else:
                            color = _KF_GOLD
                        p.setPen(Qt.PenStyle.NoPen)
                        p.setBrush(color)
                        p.drawPolygon(diamond)

Step 2: Update the context menu keyframe hit detection

At line 980, change:

        for (kt, _kc) in self._crop_keyframes:

to:

        for kf in self._crop_keyframes:
            kt = kf[0]

And remove the _kc reference — use kf[0] for kt only. The rest of the hit-detection logic stays the same.

Step 3: Run all tests

Run: pytest tests/ -v Expected: All PASS

Step 4: Manual test

Launch the app, load a video, enable lock mode, set keyframes with different combinations of random portrait/square. Verify:

• Gold diamond when no random flags set
• Red diamond when only portrait
• Blue diamond when only square
• Split red/blue when both

Step 5: Commit

git add main.py
git commit -m "feat: color-code keyframe diamonds by crop mode"

---

Task 6: Update lock-mode scrub preview

When scrubbing in lock mode, update the crop bar, overlay, and (visually) the random checkboxes to reflect the effective keyframe state at the playback position.

Files:

• Modify: main.py:2605-2621 (_on_seek_changed)

Step 1: Replace the keyframe preview block

Replace lines 2610-2621:

        if self._crop_keyframes:
            center = self._crop_center
            for kt, kc in self._crop_keyframes:
                if kt <= t + 0.05:
                    center = kc
                else:
                    break
            self._crop_bar.set_crop_center(center)
            ratio = self._cmb_portrait.currentText()
            if ratio != "Off":
                self._mpv.set_crop_overlay(_RATIOS[ratio], center)

with:

        if self._crop_keyframes:
            kf = resolve_keyframe(self._crop_keyframes, t)
            if kf is not None:
                _, center, ratio, rp, rs = kf
                self._crop_bar.set_crop_center(center)
                if ratio is not None:
                    self._mpv.set_crop_overlay(_RATIOS[ratio], center)
                else:
                    self._update_rand_overlays()

Step 2: Run all tests

Run: pytest tests/ -v Expected: All PASS

Step 3: Commit

git add main.py
git commit -m "feat: preview effective keyframe crop state during lock-mode scrub"

---

Task 7: Update overwrite-mode keyframe application

The overwrite path (lines 2727-2738) also builds jobs. It doesn't currently apply keyframes, but should for consistency.

Files:

• Modify: main.py:2727-2738 (overwrite branch in _on_export)

Step 1: Check and update

After the overwrite jobs are built, apply the same apply_keyframes_to_jobs logic if keyframes exist. The overwrite branch builds jobs as (start, path, base_ratio, base_center) — same shape as the normal path.

Add after line 2738 (self._overwrite_group = []):

            rand_portrait = self._chk_rand_portrait.isChecked()
            rand_square = self._chk_rand_square.isChecked()
            if self._crop_keyframes:
                widened = apply_keyframes_to_jobs(
                    jobs, self._crop_keyframes,
                    base_center=base_center, base_ratio=base_ratio,
                    base_rand_p=rand_portrait, base_rand_s=rand_square,
                )
                jobs = [(s, o, r, c) for s, o, r, c, _rp, _rs in widened]

Step 2: Run all tests

Run: pytest tests/ -v Expected: All PASS

Step 3: Commit

git add main.py
git commit -m "feat: apply keyframe crop modes in overwrite exports too"

---

Task 8: Update import in test file and final validation

Files:

• Modify: tests/test_utils.py:2 (import line)

Step 1: Update imports

At line 2, add the new functions to the import:

from main import build_export_path, format_time, build_ffmpeg_command, build_sequence_dir, build_audio_extract_command, build_annotation_json_path, upsert_clip_annotation, resolve_keyframe, apply_keyframes_to_jobs

(This should already be done incrementally in Tasks 1 and 4, but verify it's correct.)

Step 2: Run full test suite

Run: pytest tests/ -v Expected: All 55 tests PASS (46 original + 6 resolve_keyframe + 3 apply_keyframes)

Step 3: Manual integration test

1. Launch python main.py, load a video
2. Enable lock mode (G or click lock button)
3. Scrub to a position, enable "1 random portrait", click crop bar → red diamond appears
4. Scrub forward, disable portrait, enable "1 random square", click crop bar → blue diamond appears
5. Scrub forward, enable both, click crop bar → split red/blue diamond
6. Set clip count to 6+, spread to 2s, export
7. Verify that sub-clips falling in each keyframe region get the correct random crop behavior
8. Right-click a diamond to delete it — verify it disappears

Step 4: Commit

git add tests/test_utils.py
git commit -m "test: verify imports for keyframe crop mode helpers"