docs: design audio timeline picker and heal cut
This commit is contained in:
@@ -0,0 +1,204 @@
|
||||
# Audio Timeline Picker And Heal Cut Editor Design
|
||||
|
||||
## Goal
|
||||
|
||||
Make audio editing practical by splitting the workflow across two clear surfaces:
|
||||
|
||||
- The main timeline is the fast, precise audio region picker.
|
||||
- The clip editor is the finishing surface for one selected clip, centered on removing short noise artifacts and rejoining the audio cleanly.
|
||||
|
||||
This pass focuses on the common workflow: select a bad 1-2 second artifact, remove it, heal the join, audition the seam, then save the result.
|
||||
|
||||
## Current Context
|
||||
|
||||
Recent audio work made the Audio tab first-class and added an audio region band, waveform selection, a clip library, merge support, and a destructive clip editor. The existing implementation is functionally broad, but the interaction model is still weak:
|
||||
|
||||
- The timeline audio band has small edge hit targets and poor discoverability.
|
||||
- The timeline has a playhead, but in Audio mode the relationship between the yellow cursor and teal audio region is not clear enough.
|
||||
- The editor supports Delete, Silence, Reverse, Trim, undo/redo, playback, and Save As, but it lacks a purpose-built artifact-removal operation.
|
||||
- The merge/crossfade pane is useful, but the immediate need is better one-clip cleanup rather than multi-clip assembly.
|
||||
|
||||
## Approved Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- Upgrade the existing single-lane timeline audio band rather than adding a separate audio lane.
|
||||
- Make the teal audio band easy to move, resize, and create with the mouse.
|
||||
- Keep the yellow playhead/cursor visible and visually distinct in Audio mode.
|
||||
- Keep the timeline band, length spinbox, and waveform selection synchronized.
|
||||
- Add a primary `Heal Cut` operation to `AudioEditorDialog`.
|
||||
- Keep existing editor operations as secondary tools: Hard Delete, Silence, Reverse, Trim.
|
||||
- Add join-focused loop audition after Heal Cut.
|
||||
- Add clearer save actions: Save to Library and Save As.
|
||||
- Store Heal Cut outputs in the existing temp-version undo/redo stack.
|
||||
- Add a pure ffmpeg command builder in `core/ffmpeg.py` so heal behavior is testable outside Qt.
|
||||
|
||||
Out of scope for this pass:
|
||||
|
||||
- A separate timeline audio lane.
|
||||
- A full multitrack editor.
|
||||
- Saved editor projects.
|
||||
- Spectral repair or ML audio inpainting.
|
||||
- Major changes to the merge/crossfade pane.
|
||||
- Database or schema changes.
|
||||
|
||||
## Timeline Picker Design
|
||||
|
||||
`TimelineWidget` remains the audio picking surface in Audio mode.
|
||||
|
||||
Interaction changes:
|
||||
|
||||
- Increase the audio band edge hit target and draw larger visible handles.
|
||||
- Change the cursor on hover:
|
||||
- left/right edge: horizontal resize
|
||||
- inside band: move
|
||||
- outside band: set/create region
|
||||
- Drag an edge to resize the region.
|
||||
- Drag inside the region to move it while preserving length.
|
||||
- Click outside the region to set the audio region start at that time while preserving length.
|
||||
- Drag from empty timeline space to create a new region.
|
||||
- Emit `audio_region_changed(start, end)` on completed user edits, as today.
|
||||
|
||||
Visual changes:
|
||||
|
||||
- Keep the teal band as the audio region.
|
||||
- Use brighter/solid handle styling in Audio mode.
|
||||
- Keep the yellow cursor/playhead visible at the current start/play position.
|
||||
- Avoid showing the normal blue export clip span in Audio mode.
|
||||
- Preserve scan-region edge behavior and normal non-audio export behavior outside Audio mode.
|
||||
|
||||
Sync behavior:
|
||||
|
||||
- `MainWindow` remains the owner of synchronization between:
|
||||
- `TimelineWidget._audio_region`
|
||||
- `_cursor`
|
||||
- `_spn_audio_len`
|
||||
- `AudioWaveform` selection
|
||||
- A timeline audio edit updates `_cursor` and audio length.
|
||||
- A length spinbox edit updates the timeline audio band and waveform selection.
|
||||
- A waveform selection edit updates `_cursor`, the length spinbox, and the timeline audio band.
|
||||
|
||||
## Editor UI Design
|
||||
|
||||
`AudioEditorDialog` becomes a focused one-clip cleanup tool.
|
||||
|
||||
Layout:
|
||||
|
||||
- Large waveform area with selection handles.
|
||||
- Primary action panel with `Heal Cut`.
|
||||
- Secondary destructive tools:
|
||||
- Hard Delete
|
||||
- Silence
|
||||
- Reverse
|
||||
- Trim
|
||||
- Playback controls:
|
||||
- Play clip
|
||||
- Play selection
|
||||
- Loop Join
|
||||
- History:
|
||||
- Undo
|
||||
- Redo
|
||||
- Save:
|
||||
- Save to Library
|
||||
- Save As
|
||||
|
||||
Behavior:
|
||||
|
||||
- Heal Cut is the recommended/default artifact-removal action.
|
||||
- Hard Delete remains available for cases where no smoothing is wanted.
|
||||
- After a successful Heal Cut, the editor sets a join preview range and starts or enables looping around the healed seam.
|
||||
- Save to Library writes the current edited version into the managed audio library folder and adds it to `AudioLibraryTab`.
|
||||
- Save As keeps the existing file-dialog path.
|
||||
|
||||
## Heal Cut Behavior
|
||||
|
||||
Primary workflow:
|
||||
|
||||
1. User selects the artifact region in the editor waveform.
|
||||
2. User clicks `Heal Cut`.
|
||||
3. The editor renders a new temp version.
|
||||
4. The selected artifact is removed.
|
||||
5. The before/after audio is rejoined with an automatic short equal-power crossfade.
|
||||
6. The editor reloads the new version, pushes it onto the undo stack, and prepares a loop preview around the seam.
|
||||
|
||||
Command builder:
|
||||
|
||||
Add this pure function in `core/ffmpeg.py`:
|
||||
|
||||
```python
|
||||
def build_audio_heal_delete_command(
|
||||
input_path: str,
|
||||
start: float,
|
||||
end: float,
|
||||
out_path: str,
|
||||
crossfade: float | None = None,
|
||||
) -> list[str]:
|
||||
```
|
||||
|
||||
Automatic crossfade:
|
||||
|
||||
- If `crossfade` is omitted, choose it automatically.
|
||||
- Use a small practical range: 40-250 ms.
|
||||
- Base the chosen duration on the selected artifact length and nearby available material.
|
||||
- Clamp crossfade so it fits before `start` and after `end`.
|
||||
- If no crossfade can fit, fall back to the shortest valid fade or hard delete and report that in editor status.
|
||||
|
||||
Implementation shape:
|
||||
|
||||
- Build the output from the kept region before the artifact and the kept region after the artifact.
|
||||
- Overlap the join by the chosen crossfade duration.
|
||||
- Use ffmpeg `acrossfade` with `qsin` for both sides (`c1=qsin:c2=qsin`), matching the existing curve list in the merge code.
|
||||
- Keep output codec behavior aligned with the existing audio command helpers.
|
||||
|
||||
Zero-crossing:
|
||||
|
||||
- Zero-crossing edge adjustment is desirable, but should not block the first useful version.
|
||||
- First pass should deliver automatic equal-power Heal Cut via ffmpeg.
|
||||
- Treat sample-level zero-crossing adjustment as a deferred enhancement after the ffmpeg-based Heal Cut is working and tested.
|
||||
|
||||
## Error Handling
|
||||
|
||||
Heal Cut guardrails:
|
||||
|
||||
- If no selection exists, show `Select an artifact first`.
|
||||
- If the selection is too short, show a status message and do not edit.
|
||||
- If the selection would remove the whole clip, block it.
|
||||
- If the selected region is too close to the start or end for the preferred crossfade, use a shorter valid fade.
|
||||
- If ffmpeg fails, keep the current version unchanged and remove the temp output.
|
||||
- If the output is empty or missing, treat it as failure.
|
||||
|
||||
Timeline guardrails:
|
||||
|
||||
- Audio-mode mouse behavior must not affect non-audio export mode.
|
||||
- Audio-band interactions must not interfere with Shift-drag scan-region resizing.
|
||||
- Re-entrant synchronization between timeline, waveform, and spinbox must stay guarded.
|
||||
|
||||
## Testing Plan
|
||||
|
||||
Unit tests in `tests/test_utils.py`:
|
||||
|
||||
- `build_audio_heal_delete_command` normal case uses a crossfade join.
|
||||
- Explicit crossfade duration is respected.
|
||||
- Automatic crossfade is clamped to a practical range.
|
||||
- Near-start and near-end selections do not produce negative trim times.
|
||||
- Whole-clip or invalid selections are rejected by the caller or produce a clear error path.
|
||||
- Existing Delete/Silence/Reverse command tests continue to pass.
|
||||
|
||||
UI tests in `tests/test_ui_structure.py`:
|
||||
|
||||
- Timeline audio band exposes larger handle behavior through direct helper tests.
|
||||
- Clicking/dragging empty audio-mode timeline space can create or reset an audio region.
|
||||
- Timeline audio edit still syncs `_cursor` and `_spn_audio_len`.
|
||||
- `AudioEditorDialog` has a primary Heal Cut button.
|
||||
- Heal Cut invokes the new command builder and pushes a new undo version on success.
|
||||
- Heal Cut failure leaves the current version unchanged.
|
||||
- Save to Library adds the saved current version to the library.
|
||||
|
||||
Verification commands should keep the repo's existing isolation rule:
|
||||
|
||||
```bash
|
||||
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -v
|
||||
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -v
|
||||
```
|
||||
|
||||
Do not combine the Qt structure tests and utility tests in one process.
|
||||
Reference in New Issue
Block a user