Add source_model column to hard_negatives table with migration. New
get_hard_negatives() returns full rows, delete_hard_negatives_by_ids()
for bulk deletion. get_training_data() gains use_hard_negatives param.
TrainDialog has "Use hard negatives" checkbox. Scan panel passes current
model name when marking negatives.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each model tab now has a version combo showing scan history. When multiple
versions exist for a (file, model), users can switch between them to
compare results across training iterations. Added _current_table() and
_tab_table() helpers to unwrap the new container→table widget hierarchy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add scan_timestamp column to scan_results. save_scan_results now inserts
with a timestamp and prunes versions beyond max_versions (default 5).
get_scan_results returns only the latest version by default, with optional
scan_timestamp parameter for loading specific versions. New get_scan_versions
method returns available versions for a (file, profile, model) tuple.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ghost folders (scan-export-only) no longer appear in training dropdowns.
Also filters out 0-clip folders from get_training_stats.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move waveforms creation inside the else branch so AST and EAT
models (which have their own preprocessing) don't waste GPU memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers scan result versioning per model, hard negative management
dialog with training toggle, and ghost folder fix.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Calibration: cv=min(3,n_pos,n_neg_sample) could yield cv=1 (ValueError);
replaced with min_class >= 6 guard to skip calibration for tiny datasets
- AST: clarified chunks are already numpy arrays, use list(chunks) directly
- EAT: noted extract_features returns plain tensor (not tuple)
- Multi-layer: explicit notes on _w2v_model_name storing base name,
ml_cfg needed in _extract_w2v_targeted, embeddings_list vs embeddings
- Added AST to _ml_config layer_counts upfront in Task 2
- Added integration test for model switching (no-reload verification)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Training cancel: connect finished signal to re-enable button (was stuck disabled)
- Waveform worker: disconnect stale signal and wait on file switch, clean up on close
- DatasetStatsDialog: numeric sort via DisplayRole, remove dead widget allocation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Details button in Train dialog opens a stats view showing:
- Class totals (positive/soft/negative) with colored balance bar
- Per-video table sortable by column
- Warnings for low clip counts, class imbalance, negative-only videos
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- WaveformWorker extracts low-res audio envelope via ffmpeg, drawn as
green polygon on timeline track
- _safe_disconnect() replaces bare TypeError catches for signal cleanup
- Train button toggles to Cancel during training, calls worker.cancel()
- Dynamic GPU batch sizing: 64 for ≥16GB VRAM, 32 for ≥8GB, 16 default
- Overlap warning before exporting clips that intersect existing markers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fresh databases were missing scan_export column — broke first export
- Threshold slider now filters existing scan results without rescanning
- N key toggles hard negative on selected scan regions
- All 59 tests passing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- setup-windows.ps1 and setup_env.sh detect nvidia-smi for CUDA vs CPU PyTorch
- Startup logs Python version, venv path, PyTorch/CUDA/GPU, scikit-learn, librosa
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Prefetch next video's audio while GPU processes current embeddings
- Don't cancel Scan All when switching files in playlist
- Windows setup script now creates venv, installs PyTorch + requirements
- 8cut.bat auto-detects venv
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Show bright green playback position line in review mode
- Model history button next to scan model dropdown
- Skip backup on restore if identical timestamped copy already exists
- Auto-rescan when restoring a model version
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Scan regions can be disabled (Del/Backspace) instead of deleted, shown greyed out
- Resize scan regions by dragging timeline edges or editing table cells
- Grey ghost overlay shows trimmed portions of resized regions
- Ctrl+Z undo for disable, resize, drag, and negative toggle actions
- Fix training stats including scan-exported clips when checkbox unchecked
- Switch classifier to HistGradientBoostingClassifier (multi-threaded)
- Timestamped model saves with latest copy at base path
- Fix next-folder counter not detecting scan export folders
- Each scan area exports to its own numbered clip folder
- Platform-aware HW encoder detection (Linux/Windows/macOS)
- Auto-detect VAAPI render device instead of hardcoding
- Use shutil.move for cross-drive safety on Windows
- Comprehensive README rewrite with scan workflow documentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fuse overlapping scan regions before display (merge adjacent 1s-hop windows)
- Hard negatives: mark false positives from scan panel for training feedback
- Toggle with "Add to Negatives" button, red text + red timeline regions
- Stored in dedicated hard_negatives table, always included in training
- Model versioning: auto-backup on retrain, right-click model combo to rollback
- Scan review mode: "Review" toggle hides spread/markers for free navigation
- Scan exports: saved to DB with scan_export flag, no timeline markers
- Training dialog checkbox to optionally include scan exports
- Single group folder per batch with area numbering (clip_042_a1_0.mp4)
- Export scan results: skip negatives, skip regions < 8s, respect spread
- Button shows estimated clip count, updates on spread/fuse/negative changes
- Timeline: reload scan regions on file load, "Clear all markers" context menu
- Default training model changed to HUBERT_XLARGE
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Stash profile and crop_center at export start for async safety
- Scope get_group/delete_group by profile to prevent cross-profile leaks
- Guard auto-negative sampling when no markers exist (prevents flood)
- Wrap ffmpeg subprocess with clean timeout error message
- Fix scan-all panel reload to use stashed profile, not live value
- Remove dead warnings import
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace librosa with direct ffmpeg subprocess for 10x faster audio loading
- Add ScanResultsPanel with per-model tabs, seek-on-click, delete, and export
- Persist scan results in DB per (filename, profile, model)
- Add model selector dropdown to switch between trained embedding models
- Add "Scan All" button for batch scanning playlist videos
- Support manual negative examples via negative class folder
- Configurable auto-negative margin (default 30s, 0 = disabled)
- Deduplicate nearby training markers (8s min gap)
- Parallel audio loading with ThreadPoolExecutor during training
- Progress callbacks from training for UI status updates
- Cache bypass in scan_video (skip audio loading when embeddings cached)
- Move all caches (models, embeddings, downloads) into project directory
- Add 8cut.sh launcher script with auto venv/conda detection
- Fix 11 bugs across thread safety, signal handling, and state management
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Models now live in <project>/models/ instead of ~/.8cut_models/ so
everything stays self-contained. Updated .gitignore to exclude
models/, .venv/, *.joblib, and *.pt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix test_utils.py importing build_annotation_json_path from main
instead of core.annotations (all 59 tests pass now)
- Fix get_training_data double-counting clips at same start_time
in both positive and soft sets — subtract positive from soft
- Add cancel_flag to train_classifier so training can be interrupted
between videos (TrainWorker passes self as cancel_flag)
- Remove orphaned core/export.py (was for deleted server API)
- Remove stale Dockerfile and docker-compose.yml (referenced server)
- Clean up leftover server/__pycache__ and client/ build artifacts
- Add torch to requirements.txt (was only mentioned in comments)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove legacy distance-mode scanning (build_profile, _similarity, etc.)
and hand-crafted intensity features — pipeline is now embedding-only
- Integrate Microsoft BEATs as embedding option alongside wav2vec2/HuBERT
- Add TrainDialog with positive class selector, model picker, video dir
fallback, and live training stats
- Add TrainWorker QThread with cancel support and proper lifecycle cleanup
- Add source_path column to DB for robust source video tracking
- Add get_export_folders/get_training_data/get_training_stats to DB
- Wire source_path in all export DB writes (_on_clip_done, _on_auto_clip_done)
- Cancel scan/train workers in closeEvent to prevent use-after-free crashes
- Add setup_env.sh supporting both conda and python venv (CUDA 12.8)
- Update requirements.txt with all actual dependencies
- Update 8cut_train.py with --positive flag for new DB-driven training
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause of poor discrimination: MFCC[0] (energy) dominated the
feature vector, making cosine similarity see all audio as similar.
Changes:
- Skip MFCC[0], use 12 coefficients instead of 20
- Add delta MFCCs for temporal dynamics
- Add 7-band spectral contrast for tonal vs noise quality
- Switch from cosine similarity to euclidean-distance-based score
- Pre-compute STFT once for whole file (10-20x faster)
- Vectorized sliding window via cumulative sums (no Python loop)
- Lower sample rate 22050→16000 Hz (faster, no quality loss)
- 62-dim feature vector (was 40-dim mean+std of raw MFCCs)
- Default threshold 0.05 (new similarity scale)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mean-only vectors were too similar across different audio segments,
causing everything to match even at threshold 0.99. Adding std
captures temporal dynamics and makes the similarity scores much
more spread out.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Rename ScanWorker.finished → scan_done to stop shadowing
QThread.finished. Previously, cancelled scans leaked the QThread
because the custom signal was never emitted.
2. Block signals on combobox reset in _on_scan_ref_changed to
prevent re-entrant call when user cancels folder dialog.
3. Merge overlapping scan regions into clusters before S-key
navigation so it jumps to the next distinct match, not 1s forward
through overlapping windows.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When cancelling a scan during file change, connect finished signal
to deleteLater instead of calling it immediately on a running thread.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Scan button, threshold spinner, mode combobox, and reference source
combobox to the settings row. Implement handler methods for starting scans,
handling results/errors, cleanup of workers, and reference folder selection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add scan region storage and rendering to TimelineWidget:
- _scan_regions list in __init__ for (start, end, score) tuples
- set_scan_regions() and clear_scan_regions() methods
- paintEvent draws semi-transparent blue rectangles with score-based opacity
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Swap Task 5/6 order so get_all_export_paths exists before UI uses it
- Remove cosine similarity clamping to preserve anti-correlation signal
- Use os.path.exists instead of os.path.isfile (handles image sequences)
- Add worker cleanup to disconnect stale signals before new scan
- Remove lock from get_all_export_paths (matches read-only convention)
- Always use get_all_export_paths for Current Profile (not current-file-first)
- Filter export paths with os.path.exists for deleted files
- Use abs() for float comparison in tests instead of ==
- Add cancel_flag to ScanWorker and scan_video for interruptible scans
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ethanfel