_on_extract_audio clamped the duration to (timeline_duration - cursor), so with
the playhead within the requested length of the end (or any under-reported
duration) a 30s request was silently truncated to whatever remained — the user
asked for 30s and got 3s with no indication why.
Drop the clamp: pass the requested length straight to ffmpeg, which stops
cleanly at end-of-file if the source is shorter. Then ffprobe the result and,
when it comes up short, say so ("Saved 3.0s — source ended before 30.0s
requested") instead of silently shrinking. When there's room, 30s now yields
exactly 30s.
Adds core.ffmpeg.probe_duration(). Verified end-to-end: a fitting request
returns the exact length; a genuine near-end request returns the available
audio (rc=0) and is reported as truncated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A dedicated "♪ Extract audio" button on the transport row grabs an exact
length of audio (set via the adjacent length box, from the playhead) and opens
a Save As dialog. Output format follows the chosen extension — WAV (pcm_s16le),
MP3 (libmp3lame), FLAC, m4a/aac, ogg/opus — re-encoding as needed; unknown
extensions let ffmpeg pick from the container.
- core.ffmpeg.build_audio_clip_command(input, start, duration, out_path):
fast-seek + exact -t duration + -vn, codec by extension. Verified end-to-end
(wav/mp3/flac all land at exactly the requested duration).
- Timeline shows the audio area as a distinct teal dashed band spanning
[cursor, cursor+length], updated live as the playhead or length changes, so
you see exactly what will be extracted.
- Length + last save dir persist in QSettings; button enabled once a file loads.
Tests: 3 core (codec-by-extension, exact length, case-insensitive) + 2 GUI
(controls exist, band tracks cursor/length).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
load_for_file no longer runs three DB queries on the UI thread during file
load. A _ScanLoadWorker reads the bundle (hard negatives, scan-export times,
latest scan results) via its own short-lived connection — safe alongside the
main connection now that WAL is on. The table rebuild stays on the UI thread
in _on_scan_bundle_loaded; the timeline scan regions are synced from the new
loaded(filename) signal. Stale results from rapid file switches are ignored,
and the worker is drained on shutdown.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Group clips by export folder in one scan instead of re-scanning every row for
each folder; also drops the extra get_export_folders() query. Speeds up the
train-dialog stats with many subcategories.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Database:
- Enable WAL + synchronous=NORMAL + bigger cache pragmas
- Add (profile, filename) index covering the hot queries
- _refresh_playlist_checks: one get_clip_counts_grouped() scan for the whole
profile instead of one query per file (was O(N) full scans per keystroke/
tab switch/file load)
Timeline (60fps playback):
- set_play_position only repaints when the playhead moves a whole pixel or the
view scrolls (≈30x fewer full repaints in non-zoomed playback)
- Cache all per-paint QColor/QPen objects and the other-folder color table in
__init__ instead of allocating them every frame; drop the per-paint
visible-markers list comprehension
File load / startup:
- PlaylistWidget stats files for the missing-set only when paths change, not on
every filter keystroke
- Cache the vid-folder lookup (DB + os.listdir) per (file, folder) so spinner
ticks don't repeat it; m-counter still recomputed so it stays correct
- Swap the waveform worker without blocking the UI thread (no wait(1000))
- Defer the changelog modal so the window is interactive first
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Sub menu now has per-folder "Disable all" / "Enable all" buttons with live counts
- relocate_video_clips accepts filename=None to move every video's clips in a folder
- get_all_folder_counts returns profile-wide per-folder counts (incl _disabled)
- Disable-all confirms before moving; both refresh markers + playlist counts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Train dialog: multi-select positive subcategories via checkbox list, optional model name suffix ({profile}_{model}_{name}.joblib)
- list_trained_models recognizes named model variants
- Disable a video per-subcategory: moves its clips to a sibling {subcat}_disabled folder, rewrites DB output_path, migrates dataset.json, marks the name red
- Disabled clips excluded from training, stats, timeline, and playlist counts
- Playlist per-video count reflects only visible, non-disabled subcategories
- Persist subcategory show/hide visibility per profile across restarts
- Add/remove playlist separator rows (right-click) to mark batches, persisted per profile
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ProcessedDB.update_source_paths() to re-resolve missing or stale
source_path entries by matching filenames against a directory listing
and the current playlist. Exposed as "Update paths" button in the
train dialog next to the video dir field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract the manual export counter (m1, m2, ...) from the output path
so timeline markers match their filenames. Falls back to sequential
numbering for old-format paths without m-prefix.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Subprofile/subfolder exports now appear as colored markers (yellow,
green, blue, purple, orange) with their own numbering, separate from
the main folder's red markers. Each folder gets its own color and
independent sequence numbers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Subprofile exports (folder_suffix) created markers that interleaved
with main folder markers, shifting their numbering. Now get_markers
and _get_markers_for accept an export_folder parameter and use
SQL LIKE to only return markers whose output_path is in that folder.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Per-profile playlist persistence (session_files/{profile} in QSettings)
- Training data resolves source videos via playlist paths before fallback dir
- Guard against deleted video files in _load_file
- Fix marker double-click to seek to exact marker time instead of click pixel
- Show manual clip spans as light amber areas on the timeline
- Extend marker tuples with clip_span from DB (clip_duration + overlap)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add clip duration spinner (2–30s, default 8s) replacing all hardcoded
8.0 references. Store clip_duration in DB for accurate re-export span
calculations. Add x2/x4 playback speed toggle buttons. On Windows, mpv
renders directly into the widget's native window handle (WId embedding)
instead of slow FBO readback; crop overlays use a transparent child
widget. Fix _poll_render crash when player is None after closeEvent.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-export dialog now offers two modes: keep section length (adjust clip
count) or keep clip count (adjust section length). Files shared with
other profiles are preserved during re-export. Vid folder is resolved
before DB deletions to reuse existing folders. Add delete profile option
with confirmation dialog. Profile duplication now copies all tables
including processed exports.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-export button (next to Spread spinner) re-exports all manual clips
for the current file into the current folder with the new spread value.
Old files are deleted from their original locations first.
Duplicate profile option in the profile dropdown copies scan_results,
hard_negatives, and hidden_files to a new profile name (exports are not
copied since they reference file paths tied to the source profile).
Also widened get_profiles() to include profiles that only have
scan_results or hard_negatives, not just exports.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add "Merge N rows" context-menu option that combines selected scan rows
into one (min start, max end, max score), with full undo support.
Ctrl+Z is now an application-wide shortcut so it works regardless of
which widget has focus. Negatives undo now respects the exported-green
row color instead of reverting to default.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Right-click on exported (green) rows shows "Delete export" to wipe
associated clip files, annotations, DB rows and empty vid folders;
scan panel, markers and playlist badge refresh afterwards.
- Exporting with rows selected in the scan panel now runs a partial
export: prior scan exports are preserved, and the area index for new
clip filenames is offset past existing a-suffixes in the vid folder
to avoid collisions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Scan panel rows whose range contains an exported clip's start time
are colored green. Priority: disabled > negative > exported > default.
Exported state refreshes automatically after an auto-export batch
completes on the current file.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When starting a scan export batch, delete old scan_export entries for
the same file+profile before writing new ones. Logs a warning when
replacing. Prevents stale entry buildup from repeated scan exports.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two bugs caused vid number collisions (multiple files sharing a vid_NNN):
1. "First gap" assignment (n=1; while vid_n in existing: n++) would
reuse deleted vid numbers. Changed to max(existing) + 1 so numbers
always increase.
2. LIMIT 1 without ORDER BY returned arbitrary rows when a file had
entries in multiple vid folders. Added ORDER BY rowid DESC for
deterministic latest-wins behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Set CQ/QP rate control (quality 28) for NVENC, VAAPI, QSV, and AMF
hardware encoders instead of relying on encoder defaults which
produce unnecessarily large files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Manual clips now follow the same pattern as scan exports:
clip_003_m1_0.mp4 (manual) vs clip_003_a1_0.mp4 (auto-scan).
The clip number matches the vid folder number.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Export layout changed from clip_NNN group dirs to vid_NNN per-video folders
- Automatic DB migration rewrites old paths and moves files on startup
- Per-video counter with DB cross-check to prevent overwrites
- Changelog popup on version bump with "don't show again" checkbox
- Scan region resize now requires Shift+drag to prevent accidental edits
- Recalculate vid folder and counter on file load
- Add EAT_LARGE embedding model variant
- Update tests for new flat export path structure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use microsecond-precision timestamps to prevent version merging on
sub-second scans
- Clear undo stack when switching scan versions (stale row references)
- Parse timestamp labels robustly instead of hard-coded string slicing
- "Clear All" in hard negatives dialog respects active model filter
- Remove time.sleep from tests (no longer needed with microsecond timestamps)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add source_model column to hard_negatives table with migration. New
get_hard_negatives() returns full rows, delete_hard_negatives_by_ids()
for bulk deletion. get_training_data() gains use_hard_negatives param.
TrainDialog has "Use hard negatives" checkbox. Scan panel passes current
model name when marking negatives.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add scan_timestamp column to scan_results. save_scan_results now inserts
with a timestamp and prunes versions beyond max_versions (default 5).
get_scan_results returns only the latest version by default, with optional
scan_timestamp parameter for loading specific versions. New get_scan_versions
method returns available versions for a (file, profile, model) tuple.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ghost folders (scan-export-only) no longer appear in training dropdowns.
Also filters out 0-clip folders from get_training_stats.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move waveforms creation inside the else branch so AST and EAT
models (which have their own preprocessing) don't waste GPU memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- WaveformWorker extracts low-res audio envelope via ffmpeg, drawn as
green polygon on timeline track
- _safe_disconnect() replaces bare TypeError catches for signal cleanup
- Train button toggles to Cancel during training, calls worker.cancel()
- Dynamic GPU batch sizing: 64 for ≥16GB VRAM, 32 for ≥8GB, 16 default
- Overlap warning before exporting clips that intersect existing markers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fresh databases were missing scan_export column — broke first export
- Threshold slider now filters existing scan results without rescanning
- N key toggles hard negative on selected scan regions
- All 59 tests passing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Prefetch next video's audio while GPU processes current embeddings
- Don't cancel Scan All when switching files in playlist
- Windows setup script now creates venv, installs PyTorch + requirements
- 8cut.bat auto-detects venv
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Show bright green playback position line in review mode
- Model history button next to scan model dropdown
- Skip backup on restore if identical timestamped copy already exists
- Auto-rescan when restoring a model version
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Scan regions can be disabled (Del/Backspace) instead of deleted, shown greyed out
- Resize scan regions by dragging timeline edges or editing table cells
- Grey ghost overlay shows trimmed portions of resized regions
- Ctrl+Z undo for disable, resize, drag, and negative toggle actions
- Fix training stats including scan-exported clips when checkbox unchecked
- Switch classifier to HistGradientBoostingClassifier (multi-threaded)
- Timestamped model saves with latest copy at base path
- Fix next-folder counter not detecting scan export folders
- Each scan area exports to its own numbered clip folder
- Platform-aware HW encoder detection (Linux/Windows/macOS)
- Auto-detect VAAPI render device instead of hardcoding
- Use shutil.move for cross-drive safety on Windows
- Comprehensive README rewrite with scan workflow documentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fuse overlapping scan regions before display (merge adjacent 1s-hop windows)
- Hard negatives: mark false positives from scan panel for training feedback
- Toggle with "Add to Negatives" button, red text + red timeline regions
- Stored in dedicated hard_negatives table, always included in training
- Model versioning: auto-backup on retrain, right-click model combo to rollback
- Scan review mode: "Review" toggle hides spread/markers for free navigation
- Scan exports: saved to DB with scan_export flag, no timeline markers
- Training dialog checkbox to optionally include scan exports
- Single group folder per batch with area numbering (clip_042_a1_0.mp4)
- Export scan results: skip negatives, skip regions < 8s, respect spread
- Button shows estimated clip count, updates on spread/fuse/negative changes
- Timeline: reload scan regions on file load, "Clear all markers" context menu
- Default training model changed to HUBERT_XLARGE
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Stash profile and crop_center at export start for async safety
- Scope get_group/delete_group by profile to prevent cross-profile leaks
- Guard auto-negative sampling when no markers exist (prevents flood)
- Wrap ffmpeg subprocess with clean timeout error message
- Fix scan-all panel reload to use stashed profile, not live value
- Remove dead warnings import
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace librosa with direct ffmpeg subprocess for 10x faster audio loading
- Add ScanResultsPanel with per-model tabs, seek-on-click, delete, and export
- Persist scan results in DB per (filename, profile, model)
- Add model selector dropdown to switch between trained embedding models
- Add "Scan All" button for batch scanning playlist videos
- Support manual negative examples via negative class folder
- Configurable auto-negative margin (default 30s, 0 = disabled)
- Deduplicate nearby training markers (8s min gap)
- Parallel audio loading with ThreadPoolExecutor during training
- Progress callbacks from training for UI status updates
- Cache bypass in scan_video (skip audio loading when embeddings cached)
- Move all caches (models, embeddings, downloads) into project directory
- Add 8cut.sh launcher script with auto venv/conda detection
- Fix 11 bugs across thread safety, signal handling, and state management
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Models now live in <project>/models/ instead of ~/.8cut_models/ so
everything stays self-contained. Updated .gitignore to exclude
models/, .venv/, *.joblib, and *.pt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix test_utils.py importing build_annotation_json_path from main
instead of core.annotations (all 59 tests pass now)
- Fix get_training_data double-counting clips at same start_time
in both positive and soft sets — subtract positive from soft
- Add cancel_flag to train_classifier so training can be interrupted
between videos (TrainWorker passes self as cancel_flag)
- Remove orphaned core/export.py (was for deleted server API)
- Remove stale Dockerfile and docker-compose.yml (referenced server)
- Clean up leftover server/__pycache__ and client/ build artifacts
- Add torch to requirements.txt (was only mentioned in comments)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove legacy distance-mode scanning (build_profile, _similarity, etc.)
and hand-crafted intensity features — pipeline is now embedding-only
- Integrate Microsoft BEATs as embedding option alongside wav2vec2/HuBERT
- Add TrainDialog with positive class selector, model picker, video dir
fallback, and live training stats
- Add TrainWorker QThread with cancel support and proper lifecycle cleanup
- Add source_path column to DB for robust source video tracking
- Add get_export_folders/get_training_data/get_training_stats to DB
- Wire source_path in all export DB writes (_on_clip_done, _on_auto_clip_done)
- Cancel scan/train workers in closeEvent to prevent use-after-free crashes
- Add setup_env.sh supporting both conda and python venv (CUDA 12.8)
- Update requirements.txt with all actual dependencies
- Update 8cut_train.py with --positive flag for new DB-driven training
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause of poor discrimination: MFCC[0] (energy) dominated the
feature vector, making cosine similarity see all audio as similar.
Changes:
- Skip MFCC[0], use 12 coefficients instead of 20
- Add delta MFCCs for temporal dynamics
- Add 7-band spectral contrast for tonal vs noise quality
- Switch from cosine similarity to euclidean-distance-based score
- Pre-compute STFT once for whole file (10-20x faster)
- Vectorized sliding window via cumulative sums (no Python loop)
- Lower sample rate 22050→16000 Hz (faster, no quality loss)
- 62-dim feature vector (was 40-dim mean+std of raw MFCCs)
- Default threshold 0.05 (new similarity scale)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mean-only vectors were too similar across different audio segments,
causing everything to match even at threshold 0.99. Adding std
captures temporal dynamics and makes the similarity scores much
more spread out.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ethanfel