Ethanfel/8-cut
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: plan audio heal cut implementation

Browse Source
This commit is contained in:
Ethanfel
2026-07-04 20:13:21 +02:00
parent 7ff870ee88
commit fec08cfcb3
1 changed files with 677 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/plans/2026-07-04-audio-timeline-picker-heal-cut-implementation.md
+677
View File
@@ -0,0 +1,677 @@
# Audio Timeline Picker And Heal Cut Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make the existing audio timeline band practical for picking regions and add a one-click Heal Cut editor workflow for removing short artifacts and rejoining the audio smoothly.
**Architecture:** Keep the current single-lane `TimelineWidget` as the source-region picker and keep `MainWindow` as the synchronization owner between timeline, cursor, length spinbox, and waveform. Add a pure `core.ffmpeg.build_audio_heal_delete_command()` command builder, then wire it into `AudioEditorDialog` as the primary destructive operation using the existing rendered-version undo/redo stack.
**Tech Stack:** Python 3, PyQt6, ffmpeg filtergraphs, pytest. Run `tests/test_ui_structure.py` and `tests/test_utils.py` in separate processes with `LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen`.
---
## Baseline
Run before implementation:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -q
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -q
```
Expected baseline:
- `tests/test_ui_structure.py`: 70 passed.
- `tests/test_utils.py`: 90 passed, 3 known pre-existing failures:
- `test_ffmpeg_command_no_resize`
- `test_db_get_markers_returns_sorted`
- `test_audio_extract_timing`
## File Structure
- `core/ffmpeg.py`: add pure Heal Cut command builder and small helper for automatic fade duration.
- `tests/test_utils.py`: add pure tests for Heal Cut command generation.
- `main.py`: import builder, improve `TimelineWidget` audio interactions, add editor Heal Cut UI, join preview range, loop playback, and Save to Library.
- `tests/test_ui_structure.py`: add focused Qt structure/behavior tests for timeline picking and editor integration.
## Task 1: Pure Heal Cut ffmpeg Builder
**Files:**
- Modify: `core/ffmpeg.py`
- Modify: `tests/test_utils.py`
- [ ] **Step 1: Write failing tests**
Append these tests after `test_audio_delete_empty_head` in `tests/test_utils.py`:
```python
def test_audio_heal_delete_command_crossfades_join():
from core.ffmpeg import build_audio_heal_delete_command
cmd = build_audio_heal_delete_command("/in.wav", 2.0, 4.0, "/o/o.wav", crossfade=0.1)
assert cmd[0] == "ffmpeg"
assert cmd.count("-i") == 1
fc = cmd[cmd.index("-filter_complex") + 1]
assert "atrim=end=2.0" in fc
assert "atrim=start=4.0" in fc
assert "acrossfade=d=0.1:c1=qsin:c2=qsin[out]" in fc
assert cmd[cmd.index("-map") + 1] == "[out]"
assert "pcm_s16le" in cmd
assert cmd[-1] == "/o/o.wav"
def test_audio_heal_delete_command_auto_crossfade_clamped():
from core.ffmpeg import build_audio_heal_delete_command
cmd = build_audio_heal_delete_command("/in.wav", 10.0, 12.0, "/o/o.mp3")
fc = cmd[cmd.index("-filter_complex") + 1]
assert "acrossfade=d=0.25:c1=qsin:c2=qsin[out]" in fc
assert "libmp3lame" in cmd
def test_audio_heal_delete_command_near_start_shortens_crossfade():
from core.ffmpeg import build_audio_heal_delete_command
cmd = build_audio_heal_delete_command("/in.wav", 0.03, 1.0, "/o/o.wav")
fc = cmd[cmd.index("-filter_complex") + 1]
assert "acrossfade=d=0.03:c1=qsin:c2=qsin[out]" in fc
def test_audio_heal_delete_command_rejects_invalid_region():
import pytest
from core.ffmpeg import build_audio_heal_delete_command
with pytest.raises(ValueError):
build_audio_heal_delete_command("/in.wav", 3.0, 3.0, "/o/o.wav")
```
- [ ] **Step 2: Verify RED**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -k "audio_heal_delete" -q
```
Expected: collection/import failure or test failure because `build_audio_heal_delete_command` does not exist.
- [ ] **Step 3: Implement minimal builder**
In `core/ffmpeg.py`, add this helper and command builder after `build_audio_delete_command`:
```python
def _auto_heal_crossfade(start: float, end: float,
requested: float | None = None) -> float:
if end <= start:
raise ValueError("heal delete end must be greater than start")
if requested is not None:
fade = max(0.0, float(requested))
else:
fade = min(0.25, max(0.04, (end - start) * 0.25))
# Without knowing total duration, clamp only to available pre-roll.
fade = min(fade, max(0.0, float(start)))
return round(fade, 3)
def build_audio_heal_delete_command(input_path: str, start: float, end: float,
out_path: str,
crossfade: float | None = None) -> list[str]:
"""Remove [start, end] and heal the join with a short equal-power crossfade."""
if end <= start:
raise ValueError("heal delete end must be greater than start")
s, e = round(start, 3), round(end, 3)
xf = _auto_heal_crossfade(s, e, crossfade)
ext = os.path.splitext(out_path)[1].lower()
codec = _AUDIO_CODEC_BY_EXT.get(ext, [])
if xf <= 0:
fc = (f"[0]atrim=end={s},asetpts=PTS-STARTPTS[a];"
f"[0]atrim=start={e},asetpts=PTS-STARTPTS[b];"
f"[a][b]concat=n=2:v=0:a=1[out]")
else:
fc = (f"[0]atrim=end={s},asetpts=PTS-STARTPTS[a];"
f"[0]atrim=start={e},asetpts=PTS-STARTPTS[b];"
f"[a][b]acrossfade=d={xf}:c1=qsin:c2=qsin[out]")
return [_bin("ffmpeg"), "-y", "-i", input_path,
"-filter_complex", fc, "-map", "[out]", *codec, out_path]
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -k "audio_heal_delete or audio_delete or audio_silence or audio_reverse" -q
```
Expected: all selected tests pass.
- [ ] **Step 5: Commit**
```bash
git add core/ffmpeg.py tests/test_utils.py
git commit -m "feat: add heal cut ffmpeg command"
```
## Task 2: Timeline Audio Band Interaction Upgrade
**Files:**
- Modify: `main.py`
- Modify: `tests/test_ui_structure.py`
- [ ] **Step 1: Write failing tests**
Add these tests after `test_timeline_audio_band_resize_right` in `tests/test_ui_structure.py`:
```python
def test_timeline_audio_click_outside_moves_region_start(win):
tl = win._timeline
tl._duration = 20.0
tl._view_start = 0.0
tl._view_span = 20.0
tl.resize(400, tl.height() or 80)
tl.set_audio_mode(True)
tl.set_audio_region(4.0, 8.0)
got = []
tl.audio_region_changed.connect(lambda s, e: got.append((s, e)))
tl._audio_begin_drag_at_x(tl._time_to_x(12.0))
tl._audio_end_drag()
assert got[-1] == tl._audio_region
s, e = tl._audio_region
assert abs(s - 12.0) < 0.1
assert abs((e - s) - 4.0) < 0.1
def test_timeline_audio_drag_empty_creates_region(win):
tl = win._timeline
tl._duration = 20.0
tl._view_start = 0.0
tl._view_span = 20.0
tl.resize(400, tl.height() or 80)
tl.set_audio_mode(True)
tl.set_audio_region(4.0, 8.0)
tl._audio_begin_drag_at_x(tl._time_to_x(12.0))
tl._audio_drag_to_x(tl._time_to_x(15.0))
tl._audio_end_drag()
s, e = tl._audio_region
assert abs(s - 12.0) < 0.1
assert abs(e - 15.0) < 0.1
def test_timeline_audio_hover_cursor_state(win):
tl = win._timeline
tl._duration = 20.0
tl._view_start = 0.0
tl._view_span = 20.0
tl.resize(400, tl.height() or 80)
tl.set_audio_mode(True)
tl.set_audio_region(4.0, 8.0)
assert tl._audio_hit_at_x(tl._time_to_x(4.0)) == "left"
assert tl._audio_hit_at_x(tl._time_to_x(8.0)) == "right"
assert tl._audio_hit_at_x(tl._time_to_x(6.0)) == "move"
assert tl._audio_hit_at_x(tl._time_to_x(12.0)) == "create"
```
- [ ] **Step 2: Verify RED**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "timeline_audio" -q
```
Expected: failures because `_audio_hit_at_x` does not exist and empty-space click/drag does not create/move the region.
- [ ] **Step 3: Implement minimal timeline changes**
In `TimelineWidget.__init__`, change `_AUDIO_EDGE_PX = 6` to `_AUDIO_EDGE_PX = 10`.
Add a helper near `_audio_begin_drag_at_x`:
```python
def _audio_hit_at_x(self, x: float) -> str | None:
if not self._audio_mode or self._audio_region is None:
return None
a0, a1 = self._audio_region
ax0 = self._time_to_x(a0)
ax1 = self._time_to_x(a1)
if abs(x - ax0) <= self._AUDIO_EDGE_PX:
return "left"
if abs(x - ax1) <= self._AUDIO_EDGE_PX:
return "right"
if ax0 < x < ax1:
return "move"
return "create"
```
Update `_audio_begin_drag_at_x` so `"create"` preserves current width until the pointer moves:
```python
def _audio_begin_drag_at_x(self, x: float) -> None:
if not self._audio_mode or self._audio_region is None:
return
hit = self._audio_hit_at_x(x)
a0, a1 = self._audio_region
t = max(0.0, min(self._pos_to_time(int(x)), self._duration))
self._audio_drag = hit
if hit == "move":
self._audio_drag_anchor = t - a0
elif hit == "create":
width = max(self._AUDIO_MIN_W, a1 - a0)
a0 = max(0.0, min(t, max(0.0, self._duration - width)))
self._audio_region = (a0, a0 + width)
self.update()
if self._audio_drag is not None:
self._audio_drag_orig = self._audio_region
```
Update `_audio_drag_to_x` with a create branch:
```python
elif self._audio_drag == "create":
anchor = self._audio_drag_orig[0] if self._audio_drag_orig else a0
a0 = min(anchor, t)
a1 = max(anchor + self._AUDIO_MIN_W, max(anchor, t))
```
Update `mouseMoveEvent` hover cursor logic before marker hover:
```python
if self._audio_mode and self._audio_region is not None:
hit = self._audio_hit_at_x(x)
if hit in ("left", "right"):
self.setCursor(Qt.CursorShape.SizeHorCursor)
elif hit == "move":
self.setCursor(Qt.CursorShape.OpenHandCursor)
elif hit == "create":
self.setCursor(Qt.CursorShape.CrossCursor)
return
```
In `paintEvent`, draw handle rectangles in Audio mode after the audio band edge lines:
```python
p.fillRect(ax1 - 4, rh, 8, th, QColor(0, 220, 190, 120))
p.fillRect(ax2 - 4, rh, 8, th, QColor(0, 220, 190, 120))
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "timeline_audio" -q
```
Expected: all selected timeline audio tests pass.
- [ ] **Step 5: Commit**
```bash
git add main.py tests/test_ui_structure.py
git commit -m "feat: improve audio region timeline picking"
```
## Task 3: Editor Heal Cut UI And Undo Integration
**Files:**
- Modify: `main.py`
- Modify: `tests/test_ui_structure.py`
- [ ] **Step 1: Write failing tests**
Update `test_audio_editor_dialog_scaffold` so the button list includes `_btn_heal_cut`:
```python
for name in ("_btn_heal_cut", "_btn_delete", "_btn_silence", "_btn_reverse",
"_btn_trim", "_btn_undo", "_btn_redo", "_btn_save_as"):
assert isinstance(getattr(dlg, name), QPushButton)
```
Add this test after `test_editor_delete_builds_command`:
```python
def test_editor_heal_cut_builds_command(win, tmp_path, monkeypatch):
import main as m, pytest
src = tmp_path / "v0.wav"
src.write_bytes(b"")
dlg = m.AudioEditorDialog(str(src), parent=win)
dlg._clip_dur = 6.0
dlg._wave.set_view(0.0, 6.0)
dlg._wave.set_selection(2.0, 3.0)
seen = {}
class _Stop(Exception):
pass
def fake(inp, s, e, out):
seen.update(inp=inp, s=s, e=e, out=out)
raise _Stop
monkeypatch.setattr(m, "build_audio_heal_delete_command", fake)
with pytest.raises(_Stop):
dlg._on_heal_cut()
assert seen["inp"] == str(src)
assert seen["s"] == 2.0 and seen["e"] == 3.0
assert seen["out"].endswith(".wav")
```
- [ ] **Step 2: Verify RED**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "audio_editor_dialog_scaffold or editor_heal_cut" -q
```
Expected: failures because `_btn_heal_cut`, `_on_heal_cut`, and the import do not exist.
- [ ] **Step 3: Implement minimal editor wiring**
In `main.py`, update the `from core.ffmpeg import` block so this line:
```python
build_audio_delete_command, build_audio_silence_command,
build_audio_reverse_command,
```
becomes:
```python
build_audio_delete_command, build_audio_silence_command,
build_audio_reverse_command, build_audio_heal_delete_command,
```
In `AudioEditorDialog.__init__`, create the button before `_btn_delete`:
```python
self._btn_heal_cut = QPushButton("Heal Cut")
```
Wire it in the button/slot tuple:
```python
(self._btn_heal_cut, self._on_heal_cut),
```
Add it first in the ops row:
```python
ops.addWidget(self._btn_heal_cut)
for b in (self._btn_delete, self._btn_silence, self._btn_reverse,
self._btn_trim):
ops.addWidget(b)
```
Add the handler next to `_on_delete`:
```python
def _on_heal_cut(self):
self._apply_op(build_audio_heal_delete_command, whole_clip_ok=False,
after_success=self._prepare_join_preview)
```
Update `_apply_op` signature and its successful-render block. The function header becomes:
```python
def _apply_op(self, build_fn, whole_clip_ok: bool = True,
after_success=None) -> None:
```
The successful-render block becomes:
```python
if (proc is not None and proc.returncode == 0
and os.path.exists(tmp) and os.path.getsize(tmp) > 0):
del self._versions[self._ver_idx + 1:]
self._versions.append(tmp)
self._ver_idx += 1
self._temps.add(tmp)
self._reload()
if after_success is not None:
after_success(s)
self._set_status("")
```
Add a no-op preview method for this task:
```python
def _prepare_join_preview(self, seam_t: float) -> None:
self._join_preview = (max(0.0, seam_t - 1.0), seam_t + 1.0)
```
Initialize before `_reload()`:
```python
self._join_preview: tuple[float, float] | None = None
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "audio_editor_dialog_scaffold or editor_heal_cut or editor_delete" -q
```
Expected: selected editor tests pass.
- [ ] **Step 5: Commit**
```bash
git add main.py tests/test_ui_structure.py
git commit -m "feat: add heal cut editor action"
```
## Task 4: Join Loop Preview And Save To Library
**Files:**
- Modify: `main.py`
- Modify: `tests/test_ui_structure.py`
- [ ] **Step 1: Write failing tests**
Add this test after `test_editor_play_stop_safe`:
```python
def test_editor_loop_join_plays_preview_range(win, tmp_path, monkeypatch):
import main as m
src = tmp_path / "v0.wav"
src.write_bytes(b"")
dlg = m.AudioEditorDialog(str(src), parent=win)
dlg._join_preview = (1.0, 3.0)
seen = {}
monkeypatch.setattr(dlg, "_play_current", lambda start=None, end=None: seen.update(start=start, end=end))
dlg._on_loop_join()
assert seen == {"start": 1.0, "end": 3.0}
```
Add this test after the library tests:
```python
def test_editor_save_to_library_adds_current_version(win, tmp_path, monkeypatch):
import main as m
src = tmp_path / "v0.wav"
src.write_bytes(b"")
out_dir = tmp_path / "library"
win._settings.setValue("audio_library_dir", str(out_dir))
dlg = m.AudioEditorDialog(str(src), parent=win)
dlg._clip_dur = 2.0
saved = []
monkeypatch.setattr(m, "build_audio_clip_command",
lambda inp, start, dur, out: ["ffmpeg", out])
def fake_run(cmd, capture_output=True, timeout=300):
target = cmd[-1]
os.makedirs(os.path.dirname(target), exist_ok=True)
with open(target, "wb") as f:
f.write(b"x")
class Proc:
returncode = 0
return Proc()
monkeypatch.setattr(m.subprocess, "run", fake_run)
win._scan_panel._library.add_clip = lambda p: saved.append(p)
dlg._on_save_to_library()
assert saved
assert saved[0].startswith(str(out_dir))
```
- [ ] **Step 2: Verify RED**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "loop_join or save_to_library" -q
```
Expected: failures because `_on_loop_join`, `_play_current`, `_btn_loop_join`, `_btn_save_library`, and `_on_save_to_library` do not exist.
- [ ] **Step 3: Implement playback helper and save button**
In `AudioEditorDialog.__init__`, add:
```python
self._btn_loop_join = QPushButton("Loop Join")
self._btn_save_library = QPushButton("Save to Library")
```
Wire buttons:
```python
(self._btn_loop_join, self._on_loop_join),
(self._btn_save_library, self._on_save_to_library),
```
Add `_btn_loop_join` beside play and `_btn_save_library` before Save As.
Replace `_on_play` playback start with a helper:
```python
def _on_play(self, checked: bool) -> None:
if not checked:
self._stop_play()
return
self._play_current()
def _play_current(self, start: float | None = None,
end: float | None = None) -> None:
from PyQt6.QtCore import QProcess
self._stop_play()
self._play_proc = QProcess(self)
self._play_proc.finished.connect(self._on_play_finished)
self._play_proc.errorOccurred.connect(self._on_play_error)
args = ["-autoexit", "-nodisp", "-loglevel", "error"]
if start is not None:
args += ["-ss", str(start)]
if end is not None and start is not None and end > start:
args += ["-t", str(end - start)]
args.append(self._current())
self._play_proc.start(_bin("ffplay"), args)
self._btn_play.setText("■ Stop")
if not self._btn_play.isChecked():
self._btn_play.blockSignals(True)
self._btn_play.setChecked(True)
self._btn_play.blockSignals(False)
```
Add loop handler:
```python
def _on_loop_join(self) -> None:
if not self._join_preview:
self._set_status("No healed join to preview")
return
self._play_current(*self._join_preview)
```
Add save-to-library handler:
```python
def _on_save_to_library(self) -> None:
parent = self.parent()
settings = getattr(parent, "_settings", None)
scan_panel = getattr(parent, "_scan_panel", None)
if settings is None or scan_panel is None:
self._set_status("Library unavailable")
return
base = settings.value("audio_library_dir", "")
if not base:
base = os.path.join(str(Path.home()), "8cut_audio_library")
settings.setValue("audio_library_dir", base)
try:
os.makedirs(base, exist_ok=True)
except OSError:
self._set_status("Could not create library folder")
return
stem = os.path.splitext(os.path.basename(self._current()))[0] or "clip"
out = os.path.join(base, f"{stem}_edited.wav")
i = 1
while os.path.exists(out):
out = os.path.join(base, f"{stem}_edited_{i}.wav")
i += 1
dur = probe_duration(self._current()) or self._clip_dur or 0.0
cmd = build_audio_clip_command(self._current(), 0.0, dur, out)
QApplication.setOverrideCursor(Qt.CursorShape.WaitCursor)
try:
proc = subprocess.run(cmd, capture_output=True, timeout=300)
except Exception:
proc = None
finally:
QApplication.restoreOverrideCursor()
if proc is not None and proc.returncode == 0 and os.path.exists(out):
scan_panel._library.add_clip(out)
self._last_saved = out
self._set_status(f"Saved to library: {os.path.basename(out)}")
else:
self._set_status("Save to library failed")
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -k "audio_editor_dialog_scaffold or editor_play_stop_safe or loop_join or save_to_library" -q
```
Expected: selected tests pass.
- [ ] **Step 5: Commit**
```bash
git add main.py tests/test_ui_structure.py
git commit -m "feat: preview healed joins and save editor clips to library"
```
## Task 5: Final Verification
**Files:**
- Verify only; no file changes expected.
- [ ] **Step 1: Run full UI tests**
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -q
```
Expected: all tests pass.
- [ ] **Step 2: Run utility tests**
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -q
```
Expected: new Heal Cut tests pass; the same three baseline failures remain unless unrelated baseline fixes were made:
- `test_ffmpeg_command_no_resize`
- `test_db_get_markers_returns_sorted`
- `test_audio_extract_timing`
- [ ] **Step 3: Import smoke**
```bash
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -c "import main; print('OK')"
```
Expected: prints `OK`.
- [ ] **Step 4: Review git diff**
```bash
git status --short
git diff --stat HEAD~4..HEAD
```
Expected: commits touch only `core/ffmpeg.py`, `main.py`, `tests/test_utils.py`, and `tests/test_ui_structure.py`, plus this plan commit if it was committed separately.