diff --git a/docs/superpowers/plans/2026-07-04-audio-timeline-picker-heal-cut-implementation.md b/docs/superpowers/plans/2026-07-04-audio-timeline-picker-heal-cut-implementation.md new file mode 100644 index 0000000..729ad70 --- /dev/null +++ b/docs/superpowers/plans/2026-07-04-audio-timeline-picker-heal-cut-implementation.md @@ -0,0 +1,677 @@ +# Audio Timeline Picker And Heal Cut Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the existing audio timeline band practical for picking regions and add a one-click Heal Cut editor workflow for removing short artifacts and rejoining the audio smoothly. + +**Architecture:** Keep the current single-lane `TimelineWidget` as the source-region picker and keep `MainWindow` as the synchronization owner between timeline, cursor, length spinbox, and waveform. Add a pure `core.ffmpeg.build_audio_heal_delete_command()` command builder, then wire it into `AudioEditorDialog` as the primary destructive operation using the existing rendered-version undo/redo stack. + +**Tech Stack:** Python 3, PyQt6, ffmpeg filtergraphs, pytest. Run `tests/test_ui_structure.py` and `tests/test_utils.py` in separate processes with `LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen`. + +--- + +## Baseline + +Run before implementation: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -q +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -q +``` + +Expected baseline: + +- `tests/test_ui_structure.py`: 70 passed. +- `tests/test_utils.py`: 90 passed, 3 known pre-existing failures: + - `test_ffmpeg_command_no_resize` + - `test_db_get_markers_returns_sorted` + - `test_audio_extract_timing` + +## File Structure + +- `core/ffmpeg.py`: add pure Heal Cut command builder and small helper for automatic fade duration. +- `tests/test_utils.py`: add pure tests for Heal Cut command generation. +- `main.py`: import builder, improve `TimelineWidget` audio interactions, add editor Heal Cut UI, join preview range, loop playback, and Save to Library. +- `tests/test_ui_structure.py`: add focused Qt structure/behavior tests for timeline picking and editor integration. + +## Task 1: Pure Heal Cut ffmpeg Builder + +**Files:** +- Modify: `core/ffmpeg.py` +- Modify: `tests/test_utils.py` + +- [ ] **Step 1: Write failing tests** + +Append these tests after `test_audio_delete_empty_head` in `tests/test_utils.py`: + +```python +def test_audio_heal_delete_command_crossfades_join(): + from core.ffmpeg import build_audio_heal_delete_command + cmd = build_audio_heal_delete_command("/in.wav", 2.0, 4.0, "/o/o.wav", crossfade=0.1) + assert cmd[0] == "ffmpeg" + assert cmd.count("-i") == 1 + fc = cmd[cmd.index("-filter_complex") + 1] + assert "atrim=end=2.0" in fc + assert "atrim=start=4.0" in fc + assert "acrossfade=d=0.1:c1=qsin:c2=qsin[out]" in fc + assert cmd[cmd.index("-map") + 1] == "[out]" + assert "pcm_s16le" in cmd + assert cmd[-1] == "/o/o.wav" + + +def test_audio_heal_delete_command_auto_crossfade_clamped(): + from core.ffmpeg import build_audio_heal_delete_command + cmd = build_audio_heal_delete_command("/in.wav", 10.0, 12.0, "/o/o.mp3") + fc = cmd[cmd.index("-filter_complex") + 1] + assert "acrossfade=d=0.25:c1=qsin:c2=qsin[out]" in fc + assert "libmp3lame" in cmd + + +def test_audio_heal_delete_command_near_start_shortens_crossfade(): + from core.ffmpeg import build_audio_heal_delete_command + cmd = build_audio_heal_delete_command("/in.wav", 0.03, 1.0, "/o/o.wav") + fc = cmd[cmd.index("-filter_complex") + 1] + assert "acrossfade=d=0.03:c1=qsin:c2=qsin[out]" in fc + + +def test_audio_heal_delete_command_rejects_invalid_region(): + import pytest + from core.ffmpeg import build_audio_heal_delete_command + with pytest.raises(ValueError): + build_audio_heal_delete_command("/in.wav", 3.0, 3.0, "/o/o.wav") +``` + +- [ ] **Step 2: Verify RED** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -k "audio_heal_delete" -q +``` + +Expected: collection/import failure or test failure because `build_audio_heal_delete_command` does not exist. + +- [ ] **Step 3: Implement minimal builder** + +In `core/ffmpeg.py`, add this helper and command builder after `build_audio_delete_command`: + +```python +def _auto_heal_crossfade(start: float, end: float, + requested: float | None = None) -> float: + if end <= start: + raise ValueError("heal delete end must be greater than start") + if requested is not None: + fade = max(0.0, float(requested)) + else: + fade = min(0.25, max(0.04, (end - start) * 0.25)) + # Without knowing total duration, clamp only to available pre-roll. + fade = min(fade, max(0.0, float(start))) + return round(fade, 3) + + +def build_audio_heal_delete_command(input_path: str, start: float, end: float, + out_path: str, + crossfade: float | None = None) -> list[str]: + """Remove [start, end] and heal the join with a short equal-power crossfade.""" + if end <= start: + raise ValueError("heal delete end must be greater than start") + s, e = round(start, 3), round(end, 3) + xf = _auto_heal_crossfade(s, e, crossfade) + ext = os.path.splitext(out_path)[1].lower() + codec = _AUDIO_CODEC_BY_EXT.get(ext, []) + if xf <= 0: + fc = (f"[0]atrim=end={s},asetpts=PTS-STARTPTS[a];" + f"[0]atrim=start={e},asetpts=PTS-STARTPTS[b];" + f"[a][b]concat=n=2:v=0:a=1[out]") + else: + fc = (f"[0]atrim=end={s},asetpts=PTS-STARTPTS[a];" + f"[0]atrim=start={e},asetpts=PTS-STARTPTS[b];" + f"[a][b]acrossfade=d={xf}:c1=qsin:c2=qsin[out]") + return [_bin("ffmpeg"), "-y", "-i", input_path, + "-filter_complex", fc, "-map", "[out]", *codec, out_path] +``` + +- [ ] **Step 4: Verify GREEN** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -k "audio_heal_delete or audio_delete or audio_silence or audio_reverse" -q +``` + +Expected: all selected tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add core/ffmpeg.py tests/test_utils.py +git commit -m "feat: add heal cut ffmpeg command" +``` + +## Task 2: Timeline Audio Band Interaction Upgrade + +**Files:** +- Modify: `main.py` +- Modify: `tests/test_ui_structure.py` + +- [ ] **Step 1: Write failing tests** + +Add these tests after `test_timeline_audio_band_resize_right` in `tests/test_ui_structure.py`: + +```python +def test_timeline_audio_click_outside_moves_region_start(win): + tl = win._timeline + tl._duration = 20.0 + tl._view_start = 0.0 + tl._view_span = 20.0 + tl.resize(400, tl.height() or 80) + tl.set_audio_mode(True) + tl.set_audio_region(4.0, 8.0) + got = [] + tl.audio_region_changed.connect(lambda s, e: got.append((s, e))) + tl._audio_begin_drag_at_x(tl._time_to_x(12.0)) + tl._audio_end_drag() + assert got[-1] == tl._audio_region + s, e = tl._audio_region + assert abs(s - 12.0) < 0.1 + assert abs((e - s) - 4.0) < 0.1 + + +def test_timeline_audio_drag_empty_creates_region(win): + tl = win._timeline + tl._duration = 20.0 + tl._view_start = 0.0 + tl._view_span = 20.0 + tl.resize(400, tl.height() or 80) + tl.set_audio_mode(True) + tl.set_audio_region(4.0, 8.0) + tl._audio_begin_drag_at_x(tl._time_to_x(12.0)) + tl._audio_drag_to_x(tl._time_to_x(15.0)) + tl._audio_end_drag() + s, e = tl._audio_region + assert abs(s - 12.0) < 0.1 + assert abs(e - 15.0) < 0.1 + + +def test_timeline_audio_hover_cursor_state(win): + tl = win._timeline + tl._duration = 20.0 + tl._view_start = 0.0 + tl._view_span = 20.0 + tl.resize(400, tl.height() or 80) + tl.set_audio_mode(True) + tl.set_audio_region(4.0, 8.0) + assert tl._audio_hit_at_x(tl._time_to_x(4.0)) == "left" + assert tl._audio_hit_at_x(tl._time_to_x(8.0)) == "right" + assert tl._audio_hit_at_x(tl._time_to_x(6.0)) == "move" + assert tl._audio_hit_at_x(tl._time_to_x(12.0)) == "create" +``` + +- [ ] **Step 2: Verify RED** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "timeline_audio" -q +``` + +Expected: failures because `_audio_hit_at_x` does not exist and empty-space click/drag does not create/move the region. + +- [ ] **Step 3: Implement minimal timeline changes** + +In `TimelineWidget.__init__`, change `_AUDIO_EDGE_PX = 6` to `_AUDIO_EDGE_PX = 10`. + +Add a helper near `_audio_begin_drag_at_x`: + +```python + def _audio_hit_at_x(self, x: float) -> str | None: + if not self._audio_mode or self._audio_region is None: + return None + a0, a1 = self._audio_region + ax0 = self._time_to_x(a0) + ax1 = self._time_to_x(a1) + if abs(x - ax0) <= self._AUDIO_EDGE_PX: + return "left" + if abs(x - ax1) <= self._AUDIO_EDGE_PX: + return "right" + if ax0 < x < ax1: + return "move" + return "create" +``` + +Update `_audio_begin_drag_at_x` so `"create"` preserves current width until the pointer moves: + +```python + def _audio_begin_drag_at_x(self, x: float) -> None: + if not self._audio_mode or self._audio_region is None: + return + hit = self._audio_hit_at_x(x) + a0, a1 = self._audio_region + t = max(0.0, min(self._pos_to_time(int(x)), self._duration)) + self._audio_drag = hit + if hit == "move": + self._audio_drag_anchor = t - a0 + elif hit == "create": + width = max(self._AUDIO_MIN_W, a1 - a0) + a0 = max(0.0, min(t, max(0.0, self._duration - width))) + self._audio_region = (a0, a0 + width) + self.update() + if self._audio_drag is not None: + self._audio_drag_orig = self._audio_region +``` + +Update `_audio_drag_to_x` with a create branch: + +```python + elif self._audio_drag == "create": + anchor = self._audio_drag_orig[0] if self._audio_drag_orig else a0 + a0 = min(anchor, t) + a1 = max(anchor + self._AUDIO_MIN_W, max(anchor, t)) +``` + +Update `mouseMoveEvent` hover cursor logic before marker hover: + +```python + if self._audio_mode and self._audio_region is not None: + hit = self._audio_hit_at_x(x) + if hit in ("left", "right"): + self.setCursor(Qt.CursorShape.SizeHorCursor) + elif hit == "move": + self.setCursor(Qt.CursorShape.OpenHandCursor) + elif hit == "create": + self.setCursor(Qt.CursorShape.CrossCursor) + return +``` + +In `paintEvent`, draw handle rectangles in Audio mode after the audio band edge lines: + +```python + p.fillRect(ax1 - 4, rh, 8, th, QColor(0, 220, 190, 120)) + p.fillRect(ax2 - 4, rh, 8, th, QColor(0, 220, 190, 120)) +``` + +- [ ] **Step 4: Verify GREEN** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "timeline_audio" -q +``` + +Expected: all selected timeline audio tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add main.py tests/test_ui_structure.py +git commit -m "feat: improve audio region timeline picking" +``` + +## Task 3: Editor Heal Cut UI And Undo Integration + +**Files:** +- Modify: `main.py` +- Modify: `tests/test_ui_structure.py` + +- [ ] **Step 1: Write failing tests** + +Update `test_audio_editor_dialog_scaffold` so the button list includes `_btn_heal_cut`: + +```python + for name in ("_btn_heal_cut", "_btn_delete", "_btn_silence", "_btn_reverse", + "_btn_trim", "_btn_undo", "_btn_redo", "_btn_save_as"): + assert isinstance(getattr(dlg, name), QPushButton) +``` + +Add this test after `test_editor_delete_builds_command`: + +```python +def test_editor_heal_cut_builds_command(win, tmp_path, monkeypatch): + import main as m, pytest + src = tmp_path / "v0.wav" + src.write_bytes(b"") + dlg = m.AudioEditorDialog(str(src), parent=win) + dlg._clip_dur = 6.0 + dlg._wave.set_view(0.0, 6.0) + dlg._wave.set_selection(2.0, 3.0) + seen = {} + class _Stop(Exception): + pass + def fake(inp, s, e, out): + seen.update(inp=inp, s=s, e=e, out=out) + raise _Stop + monkeypatch.setattr(m, "build_audio_heal_delete_command", fake) + with pytest.raises(_Stop): + dlg._on_heal_cut() + assert seen["inp"] == str(src) + assert seen["s"] == 2.0 and seen["e"] == 3.0 + assert seen["out"].endswith(".wav") +``` + +- [ ] **Step 2: Verify RED** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "audio_editor_dialog_scaffold or editor_heal_cut" -q +``` + +Expected: failures because `_btn_heal_cut`, `_on_heal_cut`, and the import do not exist. + +- [ ] **Step 3: Implement minimal editor wiring** + +In `main.py`, update the `from core.ffmpeg import` block so this line: + +```python + build_audio_delete_command, build_audio_silence_command, + build_audio_reverse_command, +``` + +becomes: + +```python + build_audio_delete_command, build_audio_silence_command, + build_audio_reverse_command, build_audio_heal_delete_command, +``` + +In `AudioEditorDialog.__init__`, create the button before `_btn_delete`: + +```python + self._btn_heal_cut = QPushButton("Heal Cut") +``` + +Wire it in the button/slot tuple: + +```python + (self._btn_heal_cut, self._on_heal_cut), +``` + +Add it first in the ops row: + +```python + ops.addWidget(self._btn_heal_cut) + for b in (self._btn_delete, self._btn_silence, self._btn_reverse, + self._btn_trim): + ops.addWidget(b) +``` + +Add the handler next to `_on_delete`: + +```python + def _on_heal_cut(self): + self._apply_op(build_audio_heal_delete_command, whole_clip_ok=False, + after_success=self._prepare_join_preview) +``` + +Update `_apply_op` signature and its successful-render block. The function header becomes: + +```python + def _apply_op(self, build_fn, whole_clip_ok: bool = True, + after_success=None) -> None: +``` + +The successful-render block becomes: + +```python + if (proc is not None and proc.returncode == 0 + and os.path.exists(tmp) and os.path.getsize(tmp) > 0): + del self._versions[self._ver_idx + 1:] + self._versions.append(tmp) + self._ver_idx += 1 + self._temps.add(tmp) + self._reload() + if after_success is not None: + after_success(s) + self._set_status("") +``` + +Add a no-op preview method for this task: + +```python + def _prepare_join_preview(self, seam_t: float) -> None: + self._join_preview = (max(0.0, seam_t - 1.0), seam_t + 1.0) +``` + +Initialize before `_reload()`: + +```python + self._join_preview: tuple[float, float] | None = None +``` + +- [ ] **Step 4: Verify GREEN** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "audio_editor_dialog_scaffold or editor_heal_cut or editor_delete" -q +``` + +Expected: selected editor tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add main.py tests/test_ui_structure.py +git commit -m "feat: add heal cut editor action" +``` + +## Task 4: Join Loop Preview And Save To Library + +**Files:** +- Modify: `main.py` +- Modify: `tests/test_ui_structure.py` + +- [ ] **Step 1: Write failing tests** + +Add this test after `test_editor_play_stop_safe`: + +```python +def test_editor_loop_join_plays_preview_range(win, tmp_path, monkeypatch): + import main as m + src = tmp_path / "v0.wav" + src.write_bytes(b"") + dlg = m.AudioEditorDialog(str(src), parent=win) + dlg._join_preview = (1.0, 3.0) + seen = {} + monkeypatch.setattr(dlg, "_play_current", lambda start=None, end=None: seen.update(start=start, end=end)) + dlg._on_loop_join() + assert seen == {"start": 1.0, "end": 3.0} +``` + +Add this test after the library tests: + +```python +def test_editor_save_to_library_adds_current_version(win, tmp_path, monkeypatch): + import main as m + src = tmp_path / "v0.wav" + src.write_bytes(b"") + out_dir = tmp_path / "library" + win._settings.setValue("audio_library_dir", str(out_dir)) + dlg = m.AudioEditorDialog(str(src), parent=win) + dlg._clip_dur = 2.0 + saved = [] + monkeypatch.setattr(m, "build_audio_clip_command", + lambda inp, start, dur, out: ["ffmpeg", out]) + def fake_run(cmd, capture_output=True, timeout=300): + target = cmd[-1] + os.makedirs(os.path.dirname(target), exist_ok=True) + with open(target, "wb") as f: + f.write(b"x") + class Proc: + returncode = 0 + return Proc() + monkeypatch.setattr(m.subprocess, "run", fake_run) + win._scan_panel._library.add_clip = lambda p: saved.append(p) + dlg._on_save_to_library() + assert saved + assert saved[0].startswith(str(out_dir)) +``` + +- [ ] **Step 2: Verify RED** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "loop_join or save_to_library" -q +``` + +Expected: failures because `_on_loop_join`, `_play_current`, `_btn_loop_join`, `_btn_save_library`, and `_on_save_to_library` do not exist. + +- [ ] **Step 3: Implement playback helper and save button** + +In `AudioEditorDialog.__init__`, add: + +```python + self._btn_loop_join = QPushButton("Loop Join") + self._btn_save_library = QPushButton("Save to Library") +``` + +Wire buttons: + +```python + (self._btn_loop_join, self._on_loop_join), + (self._btn_save_library, self._on_save_to_library), +``` + +Add `_btn_loop_join` beside play and `_btn_save_library` before Save As. + +Replace `_on_play` playback start with a helper: + +```python + def _on_play(self, checked: bool) -> None: + if not checked: + self._stop_play() + return + self._play_current() + + def _play_current(self, start: float | None = None, + end: float | None = None) -> None: + from PyQt6.QtCore import QProcess + self._stop_play() + self._play_proc = QProcess(self) + self._play_proc.finished.connect(self._on_play_finished) + self._play_proc.errorOccurred.connect(self._on_play_error) + args = ["-autoexit", "-nodisp", "-loglevel", "error"] + if start is not None: + args += ["-ss", str(start)] + if end is not None and start is not None and end > start: + args += ["-t", str(end - start)] + args.append(self._current()) + self._play_proc.start(_bin("ffplay"), args) + self._btn_play.setText("■ Stop") + if not self._btn_play.isChecked(): + self._btn_play.blockSignals(True) + self._btn_play.setChecked(True) + self._btn_play.blockSignals(False) +``` + +Add loop handler: + +```python + def _on_loop_join(self) -> None: + if not self._join_preview: + self._set_status("No healed join to preview") + return + self._play_current(*self._join_preview) +``` + +Add save-to-library handler: + +```python + def _on_save_to_library(self) -> None: + parent = self.parent() + settings = getattr(parent, "_settings", None) + scan_panel = getattr(parent, "_scan_panel", None) + if settings is None or scan_panel is None: + self._set_status("Library unavailable") + return + base = settings.value("audio_library_dir", "") + if not base: + base = os.path.join(str(Path.home()), "8cut_audio_library") + settings.setValue("audio_library_dir", base) + try: + os.makedirs(base, exist_ok=True) + except OSError: + self._set_status("Could not create library folder") + return + stem = os.path.splitext(os.path.basename(self._current()))[0] or "clip" + out = os.path.join(base, f"{stem}_edited.wav") + i = 1 + while os.path.exists(out): + out = os.path.join(base, f"{stem}_edited_{i}.wav") + i += 1 + dur = probe_duration(self._current()) or self._clip_dur or 0.0 + cmd = build_audio_clip_command(self._current(), 0.0, dur, out) + QApplication.setOverrideCursor(Qt.CursorShape.WaitCursor) + try: + proc = subprocess.run(cmd, capture_output=True, timeout=300) + except Exception: + proc = None + finally: + QApplication.restoreOverrideCursor() + if proc is not None and proc.returncode == 0 and os.path.exists(out): + scan_panel._library.add_clip(out) + self._last_saved = out + self._set_status(f"Saved to library: {os.path.basename(out)}") + else: + self._set_status("Save to library failed") +``` + +- [ ] **Step 4: Verify GREEN** + +Run: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "audio_editor_dialog_scaffold or editor_play_stop_safe or loop_join or save_to_library" -q +``` + +Expected: selected tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add main.py tests/test_ui_structure.py +git commit -m "feat: preview healed joins and save editor clips to library" +``` + +## Task 5: Final Verification + +**Files:** +- Verify only; no file changes expected. + +- [ ] **Step 1: Run full UI tests** + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -q +``` + +Expected: all tests pass. + +- [ ] **Step 2: Run utility tests** + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -q +``` + +Expected: new Heal Cut tests pass; the same three baseline failures remain unless unrelated baseline fixes were made: + +- `test_ffmpeg_command_no_resize` +- `test_db_get_markers_returns_sorted` +- `test_audio_extract_timing` + +- [ ] **Step 3: Import smoke** + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -c "import main; print('OK')" +``` + +Expected: prints `OK`. + +- [ ] **Step 4: Review git diff** + +```bash +git status --short +git diff --stat HEAD~4..HEAD +``` + +Expected: commits touch only `core/ffmpeg.py`, `main.py`, `tests/test_utils.py`, and `tests/test_ui_structure.py`, plus this plan commit if it was committed separately.