From 7ff870ee88134dccae0c3125f940f0def71ae9db Mon Sep 17 00:00:00 2001 From: Ethanfel Date: Sat, 4 Jul 2026 19:42:17 +0200 Subject: [PATCH] docs: design audio timeline picker and heal cut --- .gitignore | 1 + ...4-audio-timeline-picker-heal-cut-design.md | 204 ++++++++++++++++++ 2 files changed, 205 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-04-audio-timeline-picker-heal-cut-design.md diff --git a/.gitignore b/.gitignore index e0f8ad3..75e11bc 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,7 @@ __pycache__/ *.pyo .pytest_cache/ .worktrees/ +.superpowers/ .venv/ models/ cache/ diff --git a/docs/superpowers/specs/2026-07-04-audio-timeline-picker-heal-cut-design.md b/docs/superpowers/specs/2026-07-04-audio-timeline-picker-heal-cut-design.md new file mode 100644 index 0000000..e8eb245 --- /dev/null +++ b/docs/superpowers/specs/2026-07-04-audio-timeline-picker-heal-cut-design.md @@ -0,0 +1,204 @@ +# Audio Timeline Picker And Heal Cut Editor Design + +## Goal + +Make audio editing practical by splitting the workflow across two clear surfaces: + +- The main timeline is the fast, precise audio region picker. +- The clip editor is the finishing surface for one selected clip, centered on removing short noise artifacts and rejoining the audio cleanly. + +This pass focuses on the common workflow: select a bad 1-2 second artifact, remove it, heal the join, audition the seam, then save the result. + +## Current Context + +Recent audio work made the Audio tab first-class and added an audio region band, waveform selection, a clip library, merge support, and a destructive clip editor. The existing implementation is functionally broad, but the interaction model is still weak: + +- The timeline audio band has small edge hit targets and poor discoverability. +- The timeline has a playhead, but in Audio mode the relationship between the yellow cursor and teal audio region is not clear enough. +- The editor supports Delete, Silence, Reverse, Trim, undo/redo, playback, and Save As, but it lacks a purpose-built artifact-removal operation. +- The merge/crossfade pane is useful, but the immediate need is better one-clip cleanup rather than multi-clip assembly. + +## Approved Scope + +In scope: + +- Upgrade the existing single-lane timeline audio band rather than adding a separate audio lane. +- Make the teal audio band easy to move, resize, and create with the mouse. +- Keep the yellow playhead/cursor visible and visually distinct in Audio mode. +- Keep the timeline band, length spinbox, and waveform selection synchronized. +- Add a primary `Heal Cut` operation to `AudioEditorDialog`. +- Keep existing editor operations as secondary tools: Hard Delete, Silence, Reverse, Trim. +- Add join-focused loop audition after Heal Cut. +- Add clearer save actions: Save to Library and Save As. +- Store Heal Cut outputs in the existing temp-version undo/redo stack. +- Add a pure ffmpeg command builder in `core/ffmpeg.py` so heal behavior is testable outside Qt. + +Out of scope for this pass: + +- A separate timeline audio lane. +- A full multitrack editor. +- Saved editor projects. +- Spectral repair or ML audio inpainting. +- Major changes to the merge/crossfade pane. +- Database or schema changes. + +## Timeline Picker Design + +`TimelineWidget` remains the audio picking surface in Audio mode. + +Interaction changes: + +- Increase the audio band edge hit target and draw larger visible handles. +- Change the cursor on hover: + - left/right edge: horizontal resize + - inside band: move + - outside band: set/create region +- Drag an edge to resize the region. +- Drag inside the region to move it while preserving length. +- Click outside the region to set the audio region start at that time while preserving length. +- Drag from empty timeline space to create a new region. +- Emit `audio_region_changed(start, end)` on completed user edits, as today. + +Visual changes: + +- Keep the teal band as the audio region. +- Use brighter/solid handle styling in Audio mode. +- Keep the yellow cursor/playhead visible at the current start/play position. +- Avoid showing the normal blue export clip span in Audio mode. +- Preserve scan-region edge behavior and normal non-audio export behavior outside Audio mode. + +Sync behavior: + +- `MainWindow` remains the owner of synchronization between: + - `TimelineWidget._audio_region` + - `_cursor` + - `_spn_audio_len` + - `AudioWaveform` selection +- A timeline audio edit updates `_cursor` and audio length. +- A length spinbox edit updates the timeline audio band and waveform selection. +- A waveform selection edit updates `_cursor`, the length spinbox, and the timeline audio band. + +## Editor UI Design + +`AudioEditorDialog` becomes a focused one-clip cleanup tool. + +Layout: + +- Large waveform area with selection handles. +- Primary action panel with `Heal Cut`. +- Secondary destructive tools: + - Hard Delete + - Silence + - Reverse + - Trim +- Playback controls: + - Play clip + - Play selection + - Loop Join +- History: + - Undo + - Redo +- Save: + - Save to Library + - Save As + +Behavior: + +- Heal Cut is the recommended/default artifact-removal action. +- Hard Delete remains available for cases where no smoothing is wanted. +- After a successful Heal Cut, the editor sets a join preview range and starts or enables looping around the healed seam. +- Save to Library writes the current edited version into the managed audio library folder and adds it to `AudioLibraryTab`. +- Save As keeps the existing file-dialog path. + +## Heal Cut Behavior + +Primary workflow: + +1. User selects the artifact region in the editor waveform. +2. User clicks `Heal Cut`. +3. The editor renders a new temp version. +4. The selected artifact is removed. +5. The before/after audio is rejoined with an automatic short equal-power crossfade. +6. The editor reloads the new version, pushes it onto the undo stack, and prepares a loop preview around the seam. + +Command builder: + +Add this pure function in `core/ffmpeg.py`: + +```python +def build_audio_heal_delete_command( + input_path: str, + start: float, + end: float, + out_path: str, + crossfade: float | None = None, +) -> list[str]: +``` + +Automatic crossfade: + +- If `crossfade` is omitted, choose it automatically. +- Use a small practical range: 40-250 ms. +- Base the chosen duration on the selected artifact length and nearby available material. +- Clamp crossfade so it fits before `start` and after `end`. +- If no crossfade can fit, fall back to the shortest valid fade or hard delete and report that in editor status. + +Implementation shape: + +- Build the output from the kept region before the artifact and the kept region after the artifact. +- Overlap the join by the chosen crossfade duration. +- Use ffmpeg `acrossfade` with `qsin` for both sides (`c1=qsin:c2=qsin`), matching the existing curve list in the merge code. +- Keep output codec behavior aligned with the existing audio command helpers. + +Zero-crossing: + +- Zero-crossing edge adjustment is desirable, but should not block the first useful version. +- First pass should deliver automatic equal-power Heal Cut via ffmpeg. +- Treat sample-level zero-crossing adjustment as a deferred enhancement after the ffmpeg-based Heal Cut is working and tested. + +## Error Handling + +Heal Cut guardrails: + +- If no selection exists, show `Select an artifact first`. +- If the selection is too short, show a status message and do not edit. +- If the selection would remove the whole clip, block it. +- If the selected region is too close to the start or end for the preferred crossfade, use a shorter valid fade. +- If ffmpeg fails, keep the current version unchanged and remove the temp output. +- If the output is empty or missing, treat it as failure. + +Timeline guardrails: + +- Audio-mode mouse behavior must not affect non-audio export mode. +- Audio-band interactions must not interfere with Shift-drag scan-region resizing. +- Re-entrant synchronization between timeline, waveform, and spinbox must stay guarded. + +## Testing Plan + +Unit tests in `tests/test_utils.py`: + +- `build_audio_heal_delete_command` normal case uses a crossfade join. +- Explicit crossfade duration is respected. +- Automatic crossfade is clamped to a practical range. +- Near-start and near-end selections do not produce negative trim times. +- Whole-clip or invalid selections are rejected by the caller or produce a clear error path. +- Existing Delete/Silence/Reverse command tests continue to pass. + +UI tests in `tests/test_ui_structure.py`: + +- Timeline audio band exposes larger handle behavior through direct helper tests. +- Clicking/dragging empty audio-mode timeline space can create or reset an audio region. +- Timeline audio edit still syncs `_cursor` and `_spn_audio_len`. +- `AudioEditorDialog` has a primary Heal Cut button. +- Heal Cut invokes the new command builder and pushes a new undo version on success. +- Heal Cut failure leaves the current version unchanged. +- Save to Library adds the saved current version to the library. + +Verification commands should keep the repo's existing isolation rule: + +```bash +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -v +LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -v +``` + +Do not combine the Qt structure tests and utility tests in one process.