205 lines
8.1 KiB
Markdown
205 lines
8.1 KiB
Markdown
# Audio Timeline Picker And Heal Cut Editor Design
|
|
|
|
## Goal
|
|
|
|
Make audio editing practical by splitting the workflow across two clear surfaces:
|
|
|
|
- The main timeline is the fast, precise audio region picker.
|
|
- The clip editor is the finishing surface for one selected clip, centered on removing short noise artifacts and rejoining the audio cleanly.
|
|
|
|
This pass focuses on the common workflow: select a bad 1-2 second artifact, remove it, heal the join, audition the seam, then save the result.
|
|
|
|
## Current Context
|
|
|
|
Recent audio work made the Audio tab first-class and added an audio region band, waveform selection, a clip library, merge support, and a destructive clip editor. The existing implementation is functionally broad, but the interaction model is still weak:
|
|
|
|
- The timeline audio band has small edge hit targets and poor discoverability.
|
|
- The timeline has a playhead, but in Audio mode the relationship between the yellow cursor and teal audio region is not clear enough.
|
|
- The editor supports Delete, Silence, Reverse, Trim, undo/redo, playback, and Save As, but it lacks a purpose-built artifact-removal operation.
|
|
- The merge/crossfade pane is useful, but the immediate need is better one-clip cleanup rather than multi-clip assembly.
|
|
|
|
## Approved Scope
|
|
|
|
In scope:
|
|
|
|
- Upgrade the existing single-lane timeline audio band rather than adding a separate audio lane.
|
|
- Make the teal audio band easy to move, resize, and create with the mouse.
|
|
- Keep the yellow playhead/cursor visible and visually distinct in Audio mode.
|
|
- Keep the timeline band, length spinbox, and waveform selection synchronized.
|
|
- Add a primary `Heal Cut` operation to `AudioEditorDialog`.
|
|
- Keep existing editor operations as secondary tools: Hard Delete, Silence, Reverse, Trim.
|
|
- Add join-focused loop audition after Heal Cut.
|
|
- Add clearer save actions: Save to Library and Save As.
|
|
- Store Heal Cut outputs in the existing temp-version undo/redo stack.
|
|
- Add a pure ffmpeg command builder in `core/ffmpeg.py` so heal behavior is testable outside Qt.
|
|
|
|
Out of scope for this pass:
|
|
|
|
- A separate timeline audio lane.
|
|
- A full multitrack editor.
|
|
- Saved editor projects.
|
|
- Spectral repair or ML audio inpainting.
|
|
- Major changes to the merge/crossfade pane.
|
|
- Database or schema changes.
|
|
|
|
## Timeline Picker Design
|
|
|
|
`TimelineWidget` remains the audio picking surface in Audio mode.
|
|
|
|
Interaction changes:
|
|
|
|
- Increase the audio band edge hit target and draw larger visible handles.
|
|
- Change the cursor on hover:
|
|
- left/right edge: horizontal resize
|
|
- inside band: move
|
|
- outside band: set/create region
|
|
- Drag an edge to resize the region.
|
|
- Drag inside the region to move it while preserving length.
|
|
- Click outside the region to set the audio region start at that time while preserving length.
|
|
- Drag from empty timeline space to create a new region.
|
|
- Emit `audio_region_changed(start, end)` on completed user edits, as today.
|
|
|
|
Visual changes:
|
|
|
|
- Keep the teal band as the audio region.
|
|
- Use brighter/solid handle styling in Audio mode.
|
|
- Keep the yellow cursor/playhead visible at the current start/play position.
|
|
- Avoid showing the normal blue export clip span in Audio mode.
|
|
- Preserve scan-region edge behavior and normal non-audio export behavior outside Audio mode.
|
|
|
|
Sync behavior:
|
|
|
|
- `MainWindow` remains the owner of synchronization between:
|
|
- `TimelineWidget._audio_region`
|
|
- `_cursor`
|
|
- `_spn_audio_len`
|
|
- `AudioWaveform` selection
|
|
- A timeline audio edit updates `_cursor` and audio length.
|
|
- A length spinbox edit updates the timeline audio band and waveform selection.
|
|
- A waveform selection edit updates `_cursor`, the length spinbox, and the timeline audio band.
|
|
|
|
## Editor UI Design
|
|
|
|
`AudioEditorDialog` becomes a focused one-clip cleanup tool.
|
|
|
|
Layout:
|
|
|
|
- Large waveform area with selection handles.
|
|
- Primary action panel with `Heal Cut`.
|
|
- Secondary destructive tools:
|
|
- Hard Delete
|
|
- Silence
|
|
- Reverse
|
|
- Trim
|
|
- Playback controls:
|
|
- Play clip
|
|
- Play selection
|
|
- Loop Join
|
|
- History:
|
|
- Undo
|
|
- Redo
|
|
- Save:
|
|
- Save to Library
|
|
- Save As
|
|
|
|
Behavior:
|
|
|
|
- Heal Cut is the recommended/default artifact-removal action.
|
|
- Hard Delete remains available for cases where no smoothing is wanted.
|
|
- After a successful Heal Cut, the editor sets a join preview range and starts or enables looping around the healed seam.
|
|
- Save to Library writes the current edited version into the managed audio library folder and adds it to `AudioLibraryTab`.
|
|
- Save As keeps the existing file-dialog path.
|
|
|
|
## Heal Cut Behavior
|
|
|
|
Primary workflow:
|
|
|
|
1. User selects the artifact region in the editor waveform.
|
|
2. User clicks `Heal Cut`.
|
|
3. The editor renders a new temp version.
|
|
4. The selected artifact is removed.
|
|
5. The before/after audio is rejoined with an automatic short equal-power crossfade.
|
|
6. The editor reloads the new version, pushes it onto the undo stack, and prepares a loop preview around the seam.
|
|
|
|
Command builder:
|
|
|
|
Add this pure function in `core/ffmpeg.py`:
|
|
|
|
```python
|
|
def build_audio_heal_delete_command(
|
|
input_path: str,
|
|
start: float,
|
|
end: float,
|
|
out_path: str,
|
|
crossfade: float | None = None,
|
|
) -> list[str]:
|
|
```
|
|
|
|
Automatic crossfade:
|
|
|
|
- If `crossfade` is omitted, choose it automatically.
|
|
- Use a small practical range: 40-250 ms.
|
|
- Base the chosen duration on the selected artifact length and nearby available material.
|
|
- Clamp crossfade so it fits before `start` and after `end`.
|
|
- If no crossfade can fit, fall back to the shortest valid fade or hard delete and report that in editor status.
|
|
|
|
Implementation shape:
|
|
|
|
- Build the output from the kept region before the artifact and the kept region after the artifact.
|
|
- Overlap the join by the chosen crossfade duration.
|
|
- Use ffmpeg `acrossfade` with `qsin` for both sides (`c1=qsin:c2=qsin`), matching the existing curve list in the merge code.
|
|
- Keep output codec behavior aligned with the existing audio command helpers.
|
|
|
|
Zero-crossing:
|
|
|
|
- Zero-crossing edge adjustment is desirable, but should not block the first useful version.
|
|
- First pass should deliver automatic equal-power Heal Cut via ffmpeg.
|
|
- Treat sample-level zero-crossing adjustment as a deferred enhancement after the ffmpeg-based Heal Cut is working and tested.
|
|
|
|
## Error Handling
|
|
|
|
Heal Cut guardrails:
|
|
|
|
- If no selection exists, show `Select an artifact first`.
|
|
- If the selection is too short, show a status message and do not edit.
|
|
- If the selection would remove the whole clip, block it.
|
|
- If the selected region is too close to the start or end for the preferred crossfade, use a shorter valid fade.
|
|
- If ffmpeg fails, keep the current version unchanged and remove the temp output.
|
|
- If the output is empty or missing, treat it as failure.
|
|
|
|
Timeline guardrails:
|
|
|
|
- Audio-mode mouse behavior must not affect non-audio export mode.
|
|
- Audio-band interactions must not interfere with Shift-drag scan-region resizing.
|
|
- Re-entrant synchronization between timeline, waveform, and spinbox must stay guarded.
|
|
|
|
## Testing Plan
|
|
|
|
Unit tests in `tests/test_utils.py`:
|
|
|
|
- `build_audio_heal_delete_command` normal case uses a crossfade join.
|
|
- Explicit crossfade duration is respected.
|
|
- Automatic crossfade is clamped to a practical range.
|
|
- Near-start and near-end selections do not produce negative trim times.
|
|
- Whole-clip or invalid selections are rejected by the caller or produce a clear error path.
|
|
- Existing Delete/Silence/Reverse command tests continue to pass.
|
|
|
|
UI tests in `tests/test_ui_structure.py`:
|
|
|
|
- Timeline audio band exposes larger handle behavior through direct helper tests.
|
|
- Clicking/dragging empty audio-mode timeline space can create or reset an audio region.
|
|
- Timeline audio edit still syncs `_cursor` and `_spn_audio_len`.
|
|
- `AudioEditorDialog` has a primary Heal Cut button.
|
|
- Heal Cut invokes the new command builder and pushes a new undo version on success.
|
|
- Heal Cut failure leaves the current version unchanged.
|
|
- Save to Library adds the saved current version to the library.
|
|
|
|
Verification commands should keep the repo's existing isolation rule:
|
|
|
|
```bash
|
|
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_ui_structure.py -v
|
|
LD_PRELOAD=/usr/lib/libstdc++.so.6 QT_QPA_PLATFORM=offscreen python -m pytest tests/test_utils.py -v
|
|
```
|
|
|
|
Do not combine the Qt structure tests and utility tests in one process.
|