docs: audio merge (crossfade) — Phase 2 design + implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,118 @@
|
||||
# Audio merge (crossfade) — Phase 2 design
|
||||
|
||||
**Goal:** Let the user **easily assemble multiple audio clips into one output with
|
||||
crossfades at the joins** — directly from the Audio tab — with room to grow into
|
||||
fancier assembly (per-join curves, gain automation, effects). This is the
|
||||
"audio editor" the user asked for; its north star is *crossfade merging*.
|
||||
|
||||
**Runs in:** Python/Qt client (`main.py`) + `core/ffmpeg.py`. No server/DB change.
|
||||
|
||||
**Builds on:** Phase 1 (the Audio QToolBox tab, `build_audio_clip_command`,
|
||||
`audio_edit_filters`, the waveform strip, and ffplay audition).
|
||||
|
||||
---
|
||||
|
||||
## Priority / ordering
|
||||
|
||||
The stated goal is crossfade merging, so that lands first. The interactive
|
||||
drag-select waveform (approved separately) is a *precision clip-picker* that
|
||||
feeds the merge — valuable but not required for a working merge (whole
|
||||
clips/files merge fine). Ordering:
|
||||
|
||||
- **Phase 2 (this doc):** the **Merge pane** + crossfade engine. Clips are added
|
||||
from the current extract selection (rendered with its edits) or from files on
|
||||
disk.
|
||||
- **Phase 2-later:** interactive waveform selection (drag in/out handles, zoom,
|
||||
moving playhead) as the way to define each added clip precisely; then "fancier
|
||||
stuff" (per-join crossfade curves, per-clip gain, reorder-by-drag).
|
||||
|
||||
---
|
||||
|
||||
## Engine — `core/ffmpeg.py`
|
||||
|
||||
`build_crossfade_merge_command(clips: list[str], crossfade: float, out_path: str) -> list[str]`
|
||||
|
||||
- **0 clips** → `ValueError` (caller guards; never invoked empty).
|
||||
- **1 clip** → straight re-encode to `out_path`'s format: `-i c0 -vn <codec> out`.
|
||||
- **≥2 clips, crossfade > 0** → chain ffmpeg `acrossfade` (available; `N->A`):
|
||||
```
|
||||
-i c0 -i c1 ... -i c{n-1}
|
||||
-filter_complex
|
||||
"[0][1]acrossfade=d=D[a1];[a1][2]acrossfade=d=D[a2];…;[a{n-2}][{n-1}]acrossfade=d=D[out]"
|
||||
-map "[out]" <codec> out
|
||||
```
|
||||
(single global crossfade `D` on every join in v1; per-join comes later).
|
||||
For exactly 2 clips the label is `[out]` directly (no intermediate).
|
||||
- **≥2 clips, crossfade == 0** → butt-join via the `concat` filter
|
||||
(`concat=n=N:v=0:a=1`) so a zero-crossfade merge still works.
|
||||
|
||||
Codec chosen by `out_path` extension via the existing `_AUDIO_CODEC_BY_EXT`
|
||||
(reuse — same formats as extract: wav/mp3/flac/m4a/ogg/opus).
|
||||
|
||||
Note: acrossfade needs each pair to share a sample format; ffmpeg auto-negotiates
|
||||
via the filtergraph, but if mixed-rate inputs cause trouble we insert `aresample`
|
||||
before each input. Start without it; add only if a real mismatch shows up.
|
||||
|
||||
Pure function → fully unit-tested (no ffmpeg run needed).
|
||||
|
||||
---
|
||||
|
||||
## UI — a third QToolBox pane: **Merge**
|
||||
|
||||
Added to `_build_audio_tab` after "Extract & Edit" and "Scan / Classify".
|
||||
|
||||
**Widgets (constructed in `__init__`):**
|
||||
- `_merge_list` — a `QListWidget` (reorderable) of clips to merge, in order. Each
|
||||
row shows the basename + duration (probed once on add). The clip's absolute
|
||||
path is stored on the item (`Qt.UserRole`).
|
||||
- `_spn_crossfade` — `QDoubleSpinBox`, 0.0–10.0 s, step 0.1, default 0.5 s, suffix
|
||||
" s". One global crossfade applied to every join in v1.
|
||||
- Buttons: **＋ Selection** (add the current extract area, rendered with the edit
|
||||
chain, to a temp clip and append), **＋ File…** (`QFileDialog` multi-select audio
|
||||
files), **▲ / ▼** (reorder selected row), **✕** (remove selected),
|
||||
**▶ Preview** (merge to a temp file and audition via the Phase-1 ffplay path),
|
||||
**Merge & Save…** (render + save-as).
|
||||
|
||||
**Handlers:**
|
||||
- **＋ Selection** — reuse `build_audio_clip_command(self._file_path, cursor, len,
|
||||
tmp, filters=self._current_edit_filters() or None)` → temp wav in the app temp
|
||||
dir (unique name per add) → append to `_merge_list`. Guards on a loaded file.
|
||||
- **＋ File…** — append each chosen path.
|
||||
- **▲/▼/✕** — list reorder/remove.
|
||||
- **Merge & Save…** (`_on_merge_save`) — collect the ordered paths; if <1 clip,
|
||||
status + return; build via `build_crossfade_merge_command(paths,
|
||||
_spn_crossfade.value(), out)`; `QFileDialog.getSaveFileName` (same format
|
||||
filter as extract, remembered dir); `subprocess.run` under a wait cursor +
|
||||
status; report saved length via `probe_duration`, mirroring extract's success/
|
||||
error reporting.
|
||||
- **▶ Preview** — same build to a temp file, then start the existing audition
|
||||
QProcess(ffplay) on it (reuse `_stop_audition`/teardown machinery, or a small
|
||||
shared `_play_file(path)`).
|
||||
|
||||
**Temp files:** rendered selection-clips and preview output live in the system
|
||||
temp dir with unique names; a session set tracks them and `closeEvent` best-effort
|
||||
removes them (extends the Phase-1 audition teardown).
|
||||
|
||||
---
|
||||
|
||||
## Persistence & migration
|
||||
- QSettings gains `audio_crossfade` (last crossfade value). No DB/schema change.
|
||||
- The merge sequence is **session-only** (not persisted) in v1 — it's a scratch
|
||||
assembly surface, not a saved project. (Saved projects = "fancier stuff".)
|
||||
|
||||
## Testing
|
||||
- `tests/test_utils.py`: TDD `build_crossfade_merge_command` — 1-clip re-encode,
|
||||
2-clip acrossfade (`-filter_complex` contains `acrossfade=d=0.5`, maps `[out]`),
|
||||
3-clip chained (two acrossfade stages), crossfade==0 → `concat`, codec-by-ext,
|
||||
0-clip → ValueError.
|
||||
- `tests/test_ui_structure.py`: Merge pane exists as a 3rd QToolBox page;
|
||||
`_merge_list`/`_spn_crossfade`/buttons present; add-file appends a row;
|
||||
remove/reorder mutate the list; `_on_merge_save` with an empty list is a safe
|
||||
no-op.
|
||||
|
||||
## What this does NOT do (v1)
|
||||
- No per-join crossfade durations or curve selection (single global value).
|
||||
- No interactive waveform drag-select yet (whole-clip/selection granularity).
|
||||
- No saved/reloadable merge projects (session-only sequence).
|
||||
- No multi-track mixing/overlap beyond the crossfade at joins.
|
||||
- No DB/dataset wiring (that's the deferred Phase 2 "dataset" work, separate).
|
||||
@@ -0,0 +1,224 @@
|
||||
# Audio Merge (Crossfade) — Phase 2 Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** A "Merge" pane in the Audio tab that assembles an ordered list of audio clips into one output with a crossfade at every join, rendered via ffmpeg `acrossfade`. Clips come from the current extract selection (with its edits) or from files on disk.
|
||||
|
||||
**Architecture:** A pure `core/ffmpeg.build_crossfade_merge_command` (TDD) + a third `QToolBox` page in `_build_audio_tab`. Reuses Phase-1 pieces: `build_audio_clip_command`, `_current_edit_filters`, `probe_duration`, and the ffplay audition machinery (`_audition_proc`/`_stop_audition`/`_teardown_audition`).
|
||||
|
||||
**Tech Stack:** Python 3, PyQt6, ffmpeg (`acrossfade`/`concat` filters), pytest.
|
||||
|
||||
**Design doc:** `docs/plans/2026-07-02-audio-merge-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Conventions
|
||||
- Branch `audio-tab` (continues Phase 1). New commit per task; don't amend across tasks.
|
||||
- UI tests: `LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -v`. Run `test_ui_structure.py` and `test_utils.py` in SEPARATE processes (known segfault when combined). Pre-existing red in `test_utils.py`: `test_audio_extract_timing`, `test_ffmpeg_command_no_resize`, `test_db_get_markers_returns_sorted` — untouched by this work.
|
||||
|
||||
---
|
||||
|
||||
## Task M1: `build_crossfade_merge_command` (pure, TDD)
|
||||
|
||||
**Files:** Modify `core/ffmpeg.py`; test `tests/test_utils.py`.
|
||||
|
||||
**Step 1 — failing tests:**
|
||||
```python
|
||||
def test_merge_single_clip_reencodes():
|
||||
cmd = build_crossfade_merge_command(["/a.wav"], 0.5, "/o/out.mp3")
|
||||
assert cmd[0] == "ffmpeg"
|
||||
assert cmd.count("-i") == 1
|
||||
assert "libmp3lame" in cmd # codec by out ext
|
||||
assert "acrossfade" not in " ".join(cmd)
|
||||
assert cmd[-1] == "/o/out.mp3"
|
||||
|
||||
def test_merge_two_clips_acrossfade():
|
||||
cmd = build_crossfade_merge_command(["/a.wav", "/b.wav"], 0.5, "/o/out.wav")
|
||||
assert cmd.count("-i") == 2
|
||||
fc = cmd[cmd.index("-filter_complex") + 1]
|
||||
assert "[0][1]acrossfade=d=0.5" in fc
|
||||
assert "[out]" in fc
|
||||
assert cmd[cmd.index("-map") + 1] == "[out]"
|
||||
|
||||
def test_merge_three_clips_chains():
|
||||
cmd = build_crossfade_merge_command(["/a.wav", "/b.wav", "/c.wav"], 1.0, "/o/o.wav")
|
||||
fc = cmd[cmd.index("-filter_complex") + 1]
|
||||
assert fc.count("acrossfade=d=1.0") == 2 # two joins
|
||||
assert cmd[cmd.index("-map") + 1] == "[out]"
|
||||
|
||||
def test_merge_zero_crossfade_uses_concat():
|
||||
cmd = build_crossfade_merge_command(["/a.wav", "/b.wav"], 0.0, "/o/o.wav")
|
||||
fc = cmd[cmd.index("-filter_complex") + 1]
|
||||
assert "concat=n=2:v=0:a=1" in fc
|
||||
assert "acrossfade" not in fc
|
||||
|
||||
def test_merge_empty_raises():
|
||||
import pytest
|
||||
with pytest.raises(ValueError):
|
||||
build_crossfade_merge_command([], 0.5, "/o/o.wav")
|
||||
```
|
||||
|
||||
**Step 2** — run `...pytest tests/test_utils.py -k merge -v` → FAIL.
|
||||
|
||||
**Step 3 — implement** in `core/ffmpeg.py` (near `build_audio_clip_command`):
|
||||
```python
|
||||
def build_crossfade_merge_command(clips: list[str], crossfade: float,
|
||||
out_path: str) -> list[str]:
|
||||
"""ffmpeg command that concatenates *clips* in order into *out_path*,
|
||||
crossfading each join by *crossfade* seconds (0 = butt-join). Re-encoded
|
||||
per the output extension."""
|
||||
if not clips:
|
||||
raise ValueError("no clips to merge")
|
||||
ext = os.path.splitext(out_path)[1].lower()
|
||||
codec = _AUDIO_CODEC_BY_EXT.get(ext, [])
|
||||
cmd = [_bin("ffmpeg"), "-y"]
|
||||
for c in clips:
|
||||
cmd += ["-i", c]
|
||||
if len(clips) == 1:
|
||||
return cmd + ["-vn", *codec, out_path]
|
||||
if crossfade > 0:
|
||||
# Chain acrossfade: [0][1]->[a1]; [a1][2]->[a2]; … last label = [out].
|
||||
parts, prev = [], "0"
|
||||
for i in range(1, len(clips)):
|
||||
label = "out" if i == len(clips) - 1 else f"a{i}"
|
||||
parts.append(f"[{prev}][{i}]acrossfade=d={crossfade}[{label}]")
|
||||
prev = label
|
||||
fc = ";".join(parts)
|
||||
else:
|
||||
inputs = "".join(f"[{i}]" for i in range(len(clips)))
|
||||
fc = f"{inputs}concat=n={len(clips)}:v=0:a=1[out]"
|
||||
return cmd + ["-filter_complex", fc, "-map", "[out]", *codec, out_path]
|
||||
```
|
||||
Confirm `_bin` import in ffmpeg.py (it uses `_bin` already).
|
||||
|
||||
**Step 4** — `...pytest tests/test_utils.py -k "merge or audio_clip" -v` → PASS (5 new + existing audio-clip tests).
|
||||
|
||||
**Step 5 — commit:** `git add core/ffmpeg.py tests/test_utils.py && git commit -m "feat: build_crossfade_merge_command — chained acrossfade / concat"`
|
||||
|
||||
---
|
||||
|
||||
## Task M2: Merge pane widgets + 3rd QToolBox page
|
||||
|
||||
**Files:** Modify `main.py`; test `tests/test_ui_structure.py`.
|
||||
|
||||
**Step 1 — failing test:**
|
||||
```python
|
||||
def test_merge_pane_present(win):
|
||||
from PyQt6.QtWidgets import QToolBox, QListWidget, QDoubleSpinBox
|
||||
tb = win._tab_audio.findChild(QToolBox)
|
||||
titles = [tb.itemText(i) for i in range(tb.count())]
|
||||
assert "Merge" in titles
|
||||
assert isinstance(win._merge_list, QListWidget)
|
||||
assert isinstance(win._spn_crossfade, QDoubleSpinBox)
|
||||
assert win._spn_crossfade.value() == 0.5
|
||||
```
|
||||
|
||||
**Step 2** — run → FAIL.
|
||||
|
||||
**Step 3 — implement:**
|
||||
- In `__init__` (after the audio-edit widget block), construct: `self._merge_list = QListWidget()` (set `setSelectionMode(SingleSelection)`); `self._spn_crossfade = QDoubleSpinBox()` (range 0.0–10.0, decimals 2, step 0.1, suffix " s", value from QSettings `audio_crossfade` default 0.5, persist on change); the buttons `self._btn_merge_add_sel` ("＋ Selection"), `self._btn_merge_add_file` ("＋ File…"), `self._btn_merge_up` ("▲"), `self._btn_merge_down` ("▼"), `self._btn_merge_remove` ("✕"), `self._btn_merge_preview` ("▶ Preview"), `self._btn_merge_save` ("Merge & Save…"). Wire each `clicked` to its handler (defined in M3/M4; add stub methods now that `pass` — or define fully in later tasks and reference them). To keep TDD green, define minimal stub handlers now.
|
||||
- In `_build_audio_tab`, add a third page: a `QWidget` with a `QVBoxLayout` holding the list, a crossfade row (`QLabel("Crossfade:")` + `_spn_crossfade`), a button grid (add/remove/reorder), and a bottom row (Preview, Merge & Save). `tb.addItem(merge_page, "Merge")`.
|
||||
|
||||
**Step 4** — run full `tests/test_ui_structure.py` → new test passes, prior 27 still pass.
|
||||
|
||||
**Step 5 — commit:** `feat: Merge pane (3rd Audio QToolBox page) — list + crossfade + buttons`
|
||||
|
||||
---
|
||||
|
||||
## Task M3: Add-to-sequence + reorder/remove
|
||||
|
||||
**Files:** `main.py`; test `tests/test_ui_structure.py`.
|
||||
|
||||
**Step 1 — failing test** (drives the list ops without file dialogs):
|
||||
```python
|
||||
def test_merge_list_add_remove_reorder(win, tmp_path):
|
||||
from PyQt6.QtCore import Qt
|
||||
a = tmp_path / "a.wav"; b = tmp_path / "b.wav"
|
||||
a.write_bytes(b""); b.write_bytes(b"")
|
||||
win._merge_add_paths([str(a), str(b)])
|
||||
assert win._merge_list.count() == 2
|
||||
assert win._merge_list.item(0).data(Qt.ItemDataRole.UserRole) == str(a)
|
||||
win._merge_list.setCurrentRow(1)
|
||||
win._merge_move(-1) # move b up
|
||||
assert win._merge_list.item(0).data(Qt.ItemDataRole.UserRole) == str(b)
|
||||
win._merge_list.setCurrentRow(0)
|
||||
win._merge_remove_selected()
|
||||
assert win._merge_list.count() == 1
|
||||
```
|
||||
|
||||
**Step 2** — run → FAIL.
|
||||
|
||||
**Step 3 — implement** on MainWindow:
|
||||
- `_merge_add_paths(paths)` — for each path append a `QListWidgetItem` with text `basename (Ds)` (duration via `probe_duration`, blank if None) and `setData(UserRole, abspath)`.
|
||||
- `_on_merge_add_file` — `QFileDialog.getOpenFileNames` (audio filter, remembered dir) → `_merge_add_paths`.
|
||||
- `_on_merge_add_selection` — guard `_file_path`; render `[_cursor, _cursor+len]` with `_current_edit_filters()` to a unique temp wav (track it for cleanup) via `build_audio_clip_command` + `subprocess.run` under wait cursor; on success `_merge_add_paths([tmp])`; else status.
|
||||
- `_merge_move(delta)` and `_merge_remove_selected()` — standard list row moves/removal.
|
||||
- Replace the M2 stubs for add/file/selection/up/down/remove with these (up=`_merge_move(-1)`, down=`_merge_move(1)`).
|
||||
|
||||
**Step 4** — run full UI file → new test + prior pass.
|
||||
|
||||
**Step 5 — commit:** `feat: Merge sequence — add (file/selection), reorder, remove`
|
||||
|
||||
---
|
||||
|
||||
## Task M4: Merge & Save + Preview
|
||||
|
||||
**Files:** `main.py`; test `tests/test_ui_structure.py`.
|
||||
|
||||
**Step 1 — failing test** (patches the builder to capture args; sentinel-raises to avoid running ffmpeg):
|
||||
```python
|
||||
def test_merge_save_builds_command(win, tmp_path, monkeypatch):
|
||||
import main as m, pytest
|
||||
from PyQt6.QtCore import Qt
|
||||
a = tmp_path / "a.wav"; b = tmp_path / "b.wav"; a.write_bytes(b""); b.write_bytes(b"")
|
||||
win._merge_add_paths([str(a), str(b)])
|
||||
win._spn_crossfade.setValue(0.75)
|
||||
seen = {}
|
||||
class _Stop(Exception): pass
|
||||
def fake(clips, xf, out):
|
||||
seen["clips"] = clips; seen["xf"] = xf; seen["out"] = out
|
||||
raise _Stop
|
||||
monkeypatch.setattr(m, "build_crossfade_merge_command", fake)
|
||||
monkeypatch.setattr(m.QFileDialog, "getSaveFileName",
|
||||
staticmethod(lambda *a, **k: (str(tmp_path / "m.wav"), "")))
|
||||
with pytest.raises(_Stop):
|
||||
win._on_merge_save()
|
||||
assert seen["clips"] == [str(a), str(b)]
|
||||
assert seen["xf"] == 0.75
|
||||
assert seen["out"].endswith(".wav")
|
||||
|
||||
def test_merge_save_empty_is_noop(win):
|
||||
win._merge_list.clear()
|
||||
win._on_merge_save() # must not raise / not open a dialog
|
||||
```
|
||||
|
||||
**Step 2** — run → FAIL.
|
||||
|
||||
**Step 3 — implement:**
|
||||
- `_merge_paths()` helper → list of UserRole paths in order.
|
||||
- `import build_crossfade_merge_command` at the top with the other ffmpeg imports.
|
||||
- `_on_merge_save` — if `_merge_paths()` empty → status + return; save dialog (reuse the extract save-filter builder / remembered dir); `cmd = build_crossfade_merge_command(paths, self._spn_crossfade.value(), out)`; run under wait cursor + status; on success report `probe_duration`, mirror extract's success/error handling.
|
||||
- `_play_file(path)` — extract the ffplay-start logic from `_on_audio_audition` into this shared helper (render→play stays in audition; `_play_file` just starts `QProcess(ffplay)` on an existing file and reuses `_stop_audition`/teardown). Have `_on_audio_audition` call `_play_file(tmp)` after its render.
|
||||
- `_on_merge_preview` — build to a unique temp wav, run, then `_play_file(tmp)`; guard empty list.
|
||||
|
||||
**Step 4** — run full UI file → new tests + prior pass; also re-run the audition test to confirm the `_play_file` refactor didn't break it.
|
||||
|
||||
**Step 5 — commit:** `feat: merge save + preview (crossfade render, ffplay preview)`
|
||||
|
||||
---
|
||||
|
||||
## Task M5: Temp cleanup, verify, docs
|
||||
|
||||
**Files:** `main.py`, `README.md`; verify.
|
||||
|
||||
- **Temp cleanup:** track rendered temp clips/preview outputs in a `self._merge_temps: set[str]`; best-effort `os.remove` them in `closeEvent` (add next to the audition teardown). Add `_stop_audition()` to `closeEvent` too (Phase-1 code-review follow-up).
|
||||
- **Docs:** bump `APP_VERSION` "1.3"→"1.4"; add a `("1.4", [...])` changelog entry describing the Merge pane (assemble clips, crossfade at joins, add from selection/file, preview, save). README: add a bullet group under "Audio extraction & editing" for **Merge / crossfade**.
|
||||
- **Verify:** run `tests/test_utils.py`, `tests/test_ui_structure.py`, `tests/test_audio_scan.py`, `tests/test_db.py` each in its own process; report counts; confirm the only failures are the 3 known pre-existing ones. `python -c "import main; print('OK')"` smoke.
|
||||
- **Commit:** `docs: Merge pane (Phase 2) — v1.4 changelog + README + temp cleanup`
|
||||
|
||||
---
|
||||
|
||||
## Deferred (Phase 2-later / "fancier stuff")
|
||||
- Interactive waveform drag-select (in/out handles, zoom, moving playhead) to define each added clip precisely.
|
||||
- Per-join crossfade durations + curve selection (`c1=/c2=`), per-clip gain, drag-to-reorder.
|
||||
- Saved/reloadable merge projects.
|
||||
Reference in New Issue
Block a user