diff --git a/docs/plans/2026-07-02-audio-merge-design.md b/docs/plans/2026-07-02-audio-merge-design.md new file mode 100644 index 0000000..a7cee7c --- /dev/null +++ b/docs/plans/2026-07-02-audio-merge-design.md @@ -0,0 +1,118 @@ +# Audio merge (crossfade) — Phase 2 design + +**Goal:** Let the user **easily assemble multiple audio clips into one output with +crossfades at the joins** — directly from the Audio tab — with room to grow into +fancier assembly (per-join curves, gain automation, effects). This is the +"audio editor" the user asked for; its north star is *crossfade merging*. + +**Runs in:** Python/Qt client (`main.py`) + `core/ffmpeg.py`. No server/DB change. + +**Builds on:** Phase 1 (the Audio QToolBox tab, `build_audio_clip_command`, +`audio_edit_filters`, the waveform strip, and ffplay audition). + +--- + +## Priority / ordering + +The stated goal is crossfade merging, so that lands first. The interactive +drag-select waveform (approved separately) is a *precision clip-picker* that +feeds the merge — valuable but not required for a working merge (whole +clips/files merge fine). Ordering: + +- **Phase 2 (this doc):** the **Merge pane** + crossfade engine. Clips are added + from the current extract selection (rendered with its edits) or from files on + disk. +- **Phase 2-later:** interactive waveform selection (drag in/out handles, zoom, + moving playhead) as the way to define each added clip precisely; then "fancier + stuff" (per-join crossfade curves, per-clip gain, reorder-by-drag). + +--- + +## Engine — `core/ffmpeg.py` + +`build_crossfade_merge_command(clips: list[str], crossfade: float, out_path: str) -> list[str]` + +- **0 clips** → `ValueError` (caller guards; never invoked empty). +- **1 clip** → straight re-encode to `out_path`'s format: `-i c0 -vn out`. +- **≥2 clips, crossfade > 0** → chain ffmpeg `acrossfade` (available; `N->A`): + ``` + -i c0 -i c1 ... -i c{n-1} + -filter_complex + "[0][1]acrossfade=d=D[a1];[a1][2]acrossfade=d=D[a2];…;[a{n-2}][{n-1}]acrossfade=d=D[out]" + -map "[out]" out + ``` + (single global crossfade `D` on every join in v1; per-join comes later). + For exactly 2 clips the label is `[out]` directly (no intermediate). +- **≥2 clips, crossfade == 0** → butt-join via the `concat` filter + (`concat=n=N:v=0:a=1`) so a zero-crossfade merge still works. + +Codec chosen by `out_path` extension via the existing `_AUDIO_CODEC_BY_EXT` +(reuse — same formats as extract: wav/mp3/flac/m4a/ogg/opus). + +Note: acrossfade needs each pair to share a sample format; ffmpeg auto-negotiates +via the filtergraph, but if mixed-rate inputs cause trouble we insert `aresample` +before each input. Start without it; add only if a real mismatch shows up. + +Pure function → fully unit-tested (no ffmpeg run needed). + +--- + +## UI — a third QToolBox pane: **Merge** + +Added to `_build_audio_tab` after "Extract & Edit" and "Scan / Classify". + +**Widgets (constructed in `__init__`):** +- `_merge_list` — a `QListWidget` (reorderable) of clips to merge, in order. Each + row shows the basename + duration (probed once on add). The clip's absolute + path is stored on the item (`Qt.UserRole`). +- `_spn_crossfade` — `QDoubleSpinBox`, 0.0–10.0 s, step 0.1, default 0.5 s, suffix + " s". One global crossfade applied to every join in v1. +- Buttons: **＋ Selection** (add the current extract area, rendered with the edit + chain, to a temp clip and append), **＋ File…** (`QFileDialog` multi-select audio + files), **▲ / ▼** (reorder selected row), **✕** (remove selected), + **▶ Preview** (merge to a temp file and audition via the Phase-1 ffplay path), + **Merge & Save…** (render + save-as). + +**Handlers:** +- **＋ Selection** — reuse `build_audio_clip_command(self._file_path, cursor, len, + tmp, filters=self._current_edit_filters() or None)` → temp wav in the app temp + dir (unique name per add) → append to `_merge_list`. Guards on a loaded file. +- **＋ File…** — append each chosen path. +- **▲/▼/✕** — list reorder/remove. +- **Merge & Save…** (`_on_merge_save`) — collect the ordered paths; if <1 clip, + status + return; build via `build_crossfade_merge_command(paths, + _spn_crossfade.value(), out)`; `QFileDialog.getSaveFileName` (same format + filter as extract, remembered dir); `subprocess.run` under a wait cursor + + status; report saved length via `probe_duration`, mirroring extract's success/ + error reporting. +- **▶ Preview** — same build to a temp file, then start the existing audition + QProcess(ffplay) on it (reuse `_stop_audition`/teardown machinery, or a small + shared `_play_file(path)`). + +**Temp files:** rendered selection-clips and preview output live in the system +temp dir with unique names; a session set tracks them and `closeEvent` best-effort +removes them (extends the Phase-1 audition teardown). + +--- + +## Persistence & migration +- QSettings gains `audio_crossfade` (last crossfade value). No DB/schema change. +- The merge sequence is **session-only** (not persisted) in v1 — it's a scratch + assembly surface, not a saved project. (Saved projects = "fancier stuff".) + +## Testing +- `tests/test_utils.py`: TDD `build_crossfade_merge_command` — 1-clip re-encode, + 2-clip acrossfade (`-filter_complex` contains `acrossfade=d=0.5`, maps `[out]`), + 3-clip chained (two acrossfade stages), crossfade==0 → `concat`, codec-by-ext, + 0-clip → ValueError. +- `tests/test_ui_structure.py`: Merge pane exists as a 3rd QToolBox page; + `_merge_list`/`_spn_crossfade`/buttons present; add-file appends a row; + remove/reorder mutate the list; `_on_merge_save` with an empty list is a safe + no-op. + +## What this does NOT do (v1) +- No per-join crossfade durations or curve selection (single global value). +- No interactive waveform drag-select yet (whole-clip/selection granularity). +- No saved/reloadable merge projects (session-only sequence). +- No multi-track mixing/overlap beyond the crossfade at joins. +- No DB/dataset wiring (that's the deferred Phase 2 "dataset" work, separate). diff --git a/docs/plans/2026-07-02-audio-merge-implementation.md b/docs/plans/2026-07-02-audio-merge-implementation.md new file mode 100644 index 0000000..69fded8 --- /dev/null +++ b/docs/plans/2026-07-02-audio-merge-implementation.md @@ -0,0 +1,224 @@ +# Audio Merge (Crossfade) — Phase 2 Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** A "Merge" pane in the Audio tab that assembles an ordered list of audio clips into one output with a crossfade at every join, rendered via ffmpeg `acrossfade`. Clips come from the current extract selection (with its edits) or from files on disk. + +**Architecture:** A pure `core/ffmpeg.build_crossfade_merge_command` (TDD) + a third `QToolBox` page in `_build_audio_tab`. Reuses Phase-1 pieces: `build_audio_clip_command`, `_current_edit_filters`, `probe_duration`, and the ffplay audition machinery (`_audition_proc`/`_stop_audition`/`_teardown_audition`). + +**Tech Stack:** Python 3, PyQt6, ffmpeg (`acrossfade`/`concat` filters), pytest. + +**Design doc:** `docs/plans/2026-07-02-audio-merge-design.md` + +--- + +## Conventions +- Branch `audio-tab` (continues Phase 1). New commit per task; don't amend across tasks. +- UI tests: `LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -v`. Run `test_ui_structure.py` and `test_utils.py` in SEPARATE processes (known segfault when combined). Pre-existing red in `test_utils.py`: `test_audio_extract_timing`, `test_ffmpeg_command_no_resize`, `test_db_get_markers_returns_sorted` — untouched by this work. + +--- + +## Task M1: `build_crossfade_merge_command` (pure, TDD) + +**Files:** Modify `core/ffmpeg.py`; test `tests/test_utils.py`. + +**Step 1 — failing tests:** +```python +def test_merge_single_clip_reencodes(): + cmd = build_crossfade_merge_command(["/a.wav"], 0.5, "/o/out.mp3") + assert cmd[0] == "ffmpeg" + assert cmd.count("-i") == 1 + assert "libmp3lame" in cmd # codec by out ext + assert "acrossfade" not in " ".join(cmd) + assert cmd[-1] == "/o/out.mp3" + +def test_merge_two_clips_acrossfade(): + cmd = build_crossfade_merge_command(["/a.wav", "/b.wav"], 0.5, "/o/out.wav") + assert cmd.count("-i") == 2 + fc = cmd[cmd.index("-filter_complex") + 1] + assert "[0][1]acrossfade=d=0.5" in fc + assert "[out]" in fc + assert cmd[cmd.index("-map") + 1] == "[out]" + +def test_merge_three_clips_chains(): + cmd = build_crossfade_merge_command(["/a.wav", "/b.wav", "/c.wav"], 1.0, "/o/o.wav") + fc = cmd[cmd.index("-filter_complex") + 1] + assert fc.count("acrossfade=d=1.0") == 2 # two joins + assert cmd[cmd.index("-map") + 1] == "[out]" + +def test_merge_zero_crossfade_uses_concat(): + cmd = build_crossfade_merge_command(["/a.wav", "/b.wav"], 0.0, "/o/o.wav") + fc = cmd[cmd.index("-filter_complex") + 1] + assert "concat=n=2:v=0:a=1" in fc + assert "acrossfade" not in fc + +def test_merge_empty_raises(): + import pytest + with pytest.raises(ValueError): + build_crossfade_merge_command([], 0.5, "/o/o.wav") +``` + +**Step 2** — run `...pytest tests/test_utils.py -k merge -v` → FAIL. + +**Step 3 — implement** in `core/ffmpeg.py` (near `build_audio_clip_command`): +```python +def build_crossfade_merge_command(clips: list[str], crossfade: float, + out_path: str) -> list[str]: + """ffmpeg command that concatenates *clips* in order into *out_path*, + crossfading each join by *crossfade* seconds (0 = butt-join). Re-encoded + per the output extension.""" + if not clips: + raise ValueError("no clips to merge") + ext = os.path.splitext(out_path)[1].lower() + codec = _AUDIO_CODEC_BY_EXT.get(ext, []) + cmd = [_bin("ffmpeg"), "-y"] + for c in clips: + cmd += ["-i", c] + if len(clips) == 1: + return cmd + ["-vn", *codec, out_path] + if crossfade > 0: + # Chain acrossfade: [0][1]->[a1]; [a1][2]->[a2]; … last label = [out]. + parts, prev = [], "0" + for i in range(1, len(clips)): + label = "out" if i == len(clips) - 1 else f"a{i}" + parts.append(f"[{prev}][{i}]acrossfade=d={crossfade}[{label}]") + prev = label + fc = ";".join(parts) + else: + inputs = "".join(f"[{i}]" for i in range(len(clips))) + fc = f"{inputs}concat=n={len(clips)}:v=0:a=1[out]" + return cmd + ["-filter_complex", fc, "-map", "[out]", *codec, out_path] +``` +Confirm `_bin` import in ffmpeg.py (it uses `_bin` already). + +**Step 4** — `...pytest tests/test_utils.py -k "merge or audio_clip" -v` → PASS (5 new + existing audio-clip tests). + +**Step 5 — commit:** `git add core/ffmpeg.py tests/test_utils.py && git commit -m "feat: build_crossfade_merge_command — chained acrossfade / concat"` + +--- + +## Task M2: Merge pane widgets + 3rd QToolBox page + +**Files:** Modify `main.py`; test `tests/test_ui_structure.py`. + +**Step 1 — failing test:** +```python +def test_merge_pane_present(win): + from PyQt6.QtWidgets import QToolBox, QListWidget, QDoubleSpinBox + tb = win._tab_audio.findChild(QToolBox) + titles = [tb.itemText(i) for i in range(tb.count())] + assert "Merge" in titles + assert isinstance(win._merge_list, QListWidget) + assert isinstance(win._spn_crossfade, QDoubleSpinBox) + assert win._spn_crossfade.value() == 0.5 +``` + +**Step 2** — run → FAIL. + +**Step 3 — implement:** +- In `__init__` (after the audio-edit widget block), construct: `self._merge_list = QListWidget()` (set `setSelectionMode(SingleSelection)`); `self._spn_crossfade = QDoubleSpinBox()` (range 0.0–10.0, decimals 2, step 0.1, suffix " s", value from QSettings `audio_crossfade` default 0.5, persist on change); the buttons `self._btn_merge_add_sel` ("＋ Selection"), `self._btn_merge_add_file` ("＋ File…"), `self._btn_merge_up` ("▲"), `self._btn_merge_down` ("▼"), `self._btn_merge_remove` ("✕"), `self._btn_merge_preview` ("▶ Preview"), `self._btn_merge_save` ("Merge & Save…"). Wire each `clicked` to its handler (defined in M3/M4; add stub methods now that `pass` — or define fully in later tasks and reference them). To keep TDD green, define minimal stub handlers now. +- In `_build_audio_tab`, add a third page: a `QWidget` with a `QVBoxLayout` holding the list, a crossfade row (`QLabel("Crossfade:")` + `_spn_crossfade`), a button grid (add/remove/reorder), and a bottom row (Preview, Merge & Save). `tb.addItem(merge_page, "Merge")`. + +**Step 4** — run full `tests/test_ui_structure.py` → new test passes, prior 27 still pass. + +**Step 5 — commit:** `feat: Merge pane (3rd Audio QToolBox page) — list + crossfade + buttons` + +--- + +## Task M3: Add-to-sequence + reorder/remove + +**Files:** `main.py`; test `tests/test_ui_structure.py`. + +**Step 1 — failing test** (drives the list ops without file dialogs): +```python +def test_merge_list_add_remove_reorder(win, tmp_path): + from PyQt6.QtCore import Qt + a = tmp_path / "a.wav"; b = tmp_path / "b.wav" + a.write_bytes(b""); b.write_bytes(b"") + win._merge_add_paths([str(a), str(b)]) + assert win._merge_list.count() == 2 + assert win._merge_list.item(0).data(Qt.ItemDataRole.UserRole) == str(a) + win._merge_list.setCurrentRow(1) + win._merge_move(-1) # move b up + assert win._merge_list.item(0).data(Qt.ItemDataRole.UserRole) == str(b) + win._merge_list.setCurrentRow(0) + win._merge_remove_selected() + assert win._merge_list.count() == 1 +``` + +**Step 2** — run → FAIL. + +**Step 3 — implement** on MainWindow: +- `_merge_add_paths(paths)` — for each path append a `QListWidgetItem` with text `basename (Ds)` (duration via `probe_duration`, blank if None) and `setData(UserRole, abspath)`. +- `_on_merge_add_file` — `QFileDialog.getOpenFileNames` (audio filter, remembered dir) → `_merge_add_paths`. +- `_on_merge_add_selection` — guard `_file_path`; render `[_cursor, _cursor+len]` with `_current_edit_filters()` to a unique temp wav (track it for cleanup) via `build_audio_clip_command` + `subprocess.run` under wait cursor; on success `_merge_add_paths([tmp])`; else status. +- `_merge_move(delta)` and `_merge_remove_selected()` — standard list row moves/removal. +- Replace the M2 stubs for add/file/selection/up/down/remove with these (up=`_merge_move(-1)`, down=`_merge_move(1)`). + +**Step 4** — run full UI file → new test + prior pass. + +**Step 5 — commit:** `feat: Merge sequence — add (file/selection), reorder, remove` + +--- + +## Task M4: Merge & Save + Preview + +**Files:** `main.py`; test `tests/test_ui_structure.py`. + +**Step 1 — failing test** (patches the builder to capture args; sentinel-raises to avoid running ffmpeg): +```python +def test_merge_save_builds_command(win, tmp_path, monkeypatch): + import main as m, pytest + from PyQt6.QtCore import Qt + a = tmp_path / "a.wav"; b = tmp_path / "b.wav"; a.write_bytes(b""); b.write_bytes(b"") + win._merge_add_paths([str(a), str(b)]) + win._spn_crossfade.setValue(0.75) + seen = {} + class _Stop(Exception): pass + def fake(clips, xf, out): + seen["clips"] = clips; seen["xf"] = xf; seen["out"] = out + raise _Stop + monkeypatch.setattr(m, "build_crossfade_merge_command", fake) + monkeypatch.setattr(m.QFileDialog, "getSaveFileName", + staticmethod(lambda *a, **k: (str(tmp_path / "m.wav"), ""))) + with pytest.raises(_Stop): + win._on_merge_save() + assert seen["clips"] == [str(a), str(b)] + assert seen["xf"] == 0.75 + assert seen["out"].endswith(".wav") + +def test_merge_save_empty_is_noop(win): + win._merge_list.clear() + win._on_merge_save() # must not raise / not open a dialog +``` + +**Step 2** — run → FAIL. + +**Step 3 — implement:** +- `_merge_paths()` helper → list of UserRole paths in order. +- `import build_crossfade_merge_command` at the top with the other ffmpeg imports. +- `_on_merge_save` — if `_merge_paths()` empty → status + return; save dialog (reuse the extract save-filter builder / remembered dir); `cmd = build_crossfade_merge_command(paths, self._spn_crossfade.value(), out)`; run under wait cursor + status; on success report `probe_duration`, mirror extract's success/error handling. +- `_play_file(path)` — extract the ffplay-start logic from `_on_audio_audition` into this shared helper (render→play stays in audition; `_play_file` just starts `QProcess(ffplay)` on an existing file and reuses `_stop_audition`/teardown). Have `_on_audio_audition` call `_play_file(tmp)` after its render. +- `_on_merge_preview` — build to a unique temp wav, run, then `_play_file(tmp)`; guard empty list. + +**Step 4** — run full UI file → new tests + prior pass; also re-run the audition test to confirm the `_play_file` refactor didn't break it. + +**Step 5 — commit:** `feat: merge save + preview (crossfade render, ffplay preview)` + +--- + +## Task M5: Temp cleanup, verify, docs + +**Files:** `main.py`, `README.md`; verify. + +- **Temp cleanup:** track rendered temp clips/preview outputs in a `self._merge_temps: set[str]`; best-effort `os.remove` them in `closeEvent` (add next to the audition teardown). Add `_stop_audition()` to `closeEvent` too (Phase-1 code-review follow-up). +- **Docs:** bump `APP_VERSION` "1.3"→"1.4"; add a `("1.4", [...])` changelog entry describing the Merge pane (assemble clips, crossfade at joins, add from selection/file, preview, save). README: add a bullet group under "Audio extraction & editing" for **Merge / crossfade**. +- **Verify:** run `tests/test_utils.py`, `tests/test_ui_structure.py`, `tests/test_audio_scan.py`, `tests/test_db.py` each in its own process; report counts; confirm the only failures are the 3 known pre-existing ones. `python -c "import main; print('OK')"` smoke. +- **Commit:** `docs: Merge pane (Phase 2) — v1.4 changelog + README + temp cleanup` + +--- + +## Deferred (Phase 2-later / "fancier stuff") +- Interactive waveform drag-select (in/out handles, zoom, moving playhead) to define each added clip precisely. +- Per-join crossfade durations + curve selection (`c1=/c2=`), per-clip gain, drag-to-reorder. +- Saved/reloadable merge projects.