Commit Graph

239 Commits

Author SHA1 Message Date
Ethanfel edc5784ba6 feat: hard negative source_model tracking, training toggle
Add source_model column to hard_negatives table with migration. New
get_hard_negatives() returns full rows, delete_hard_negatives_by_ids()
for bulk deletion. get_training_data() gains use_hard_negatives param.
TrainDialog has "Use hard negatives" checkbox. Scan panel passes current
model name when marking negatives.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 15:27:11 +02:00
Ethanfel 8ed9fbf557 feat: scan version selector in results panel
Each model tab now has a version combo showing scan history. When multiple
versions exist for a (file, model), users can switch between them to
compare results across training iterations. Added _current_table() and
_tab_table() helpers to unwrap the new container→table widget hierarchy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 15:22:46 +02:00
Ethanfel 7855ea62c2 fix: training cancel button re-enable, waveform worker cleanup, stats table sort
- Training cancel: connect finished signal to re-enable button (was stuck disabled)
- Waveform worker: disconnect stale signal and wait on file switch, clean up on close
- DatasetStatsDialog: numeric sort via DisplayRole, remove dead widget allocation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 13:03:14 +02:00
Ethanfel 70be5974cf feat: dataset statistics dialog with per-video breakdown and class balance
Details button in Train dialog opens a stats view showing:
- Class totals (positive/soft/negative) with colored balance bar
- Per-video table sortable by column
- Warnings for low clip counts, class imbalance, negative-only videos

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 12:55:42 +02:00
Ethanfel a0286d5cf9 feat: waveform overlay, signal safety, training cancel, dynamic batch size, duplicate detection
- WaveformWorker extracts low-res audio envelope via ffmpeg, drawn as
  green polygon on timeline track
- _safe_disconnect() replaces bare TypeError catches for signal cleanup
- Train button toggles to Cancel during training, calls worker.cancel()
- Dynamic GPU batch sizing: 64 for ≥16GB VRAM, 32 for ≥8GB, 16 default
- Overlap warning before exporting clips that intersect existing markers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 12:53:48 +02:00
Ethanfel 2b7dfb330d fix: DB schema missing scan_export column, add threshold filter and N hotkey
- Fresh databases were missing scan_export column — broke first export
- Threshold slider now filters existing scan results without rescanning
- N key toggles hard negative on selected scan regions
- All 59 tests passing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-19 12:45:14 +02:00
Ethanfel 282156e8ed feat: auto-detect GPU in setup scripts, log environment at startup
- setup-windows.ps1 and setup_env.sh detect nvidia-smi for CUDA vs CPU PyTorch
- Startup logs Python version, venv path, PyTorch/CUDA/GPU, scikit-learn, librosa

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 22:12:45 +02:00
Ethanfel 3417a0f603 fix: crash when switching folder in train dialog (signal recursion)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 22:00:23 +02:00
Ethanfel cd0552197f feat: prefetch audio during Scan All, fix file-switch interruption, fix Windows setup
- Prefetch next video's audio while GPU processes current embeddings
- Don't cancel Scan All when switching files in playlist
- Windows setup script now creates venv, installs PyTorch + requirements
- 8cut.bat auto-detects venv

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 21:50:33 +02:00
Ethanfel 7dffcb08eb feat: interruptible Scan All — stop after current video, resume later
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 21:37:47 +02:00
Ethanfel eda7826a40 fix: safe PATH fallback for Windows DLL loading, deduplicate model restore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 21:14:44 +02:00
Ethanfel e7e20b0fe6 fix: review mode playback line, model restore dedup, auto-rescan on rollback
- Show bright green playback position line in review mode
- Model history button next to scan model dropdown
- Skip backup on restore if identical timestamped copy already exists
- Auto-rescan when restoring a model version

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 21:05:40 +02:00
Ethanfel 814ef946eb fix: add missing shortcuts to help dialog (disable, undo, drag resize)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 20:47:52 +02:00
Ethanfel 6ddfcde8ee feat: disable/resize scan regions, undo, training fixes, cross-platform cleanup
- Scan regions can be disabled (Del/Backspace) instead of deleted, shown greyed out
- Resize scan regions by dragging timeline edges or editing table cells
- Grey ghost overlay shows trimmed portions of resized regions
- Ctrl+Z undo for disable, resize, drag, and negative toggle actions
- Fix training stats including scan-exported clips when checkbox unchecked
- Switch classifier to HistGradientBoostingClassifier (multi-threaded)
- Timestamped model saves with latest copy at base path
- Fix next-folder counter not detecting scan export folders
- Each scan area exports to its own numbered clip folder
- Platform-aware HW encoder detection (Linux/Windows/macOS)
- Auto-detect VAAPI render device instead of hardcoding
- Use shutil.move for cross-drive safety on Windows
- Comprehensive README rewrite with scan workflow documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 20:34:56 +02:00
Ethanfel b161412d94 feat: scan workflow — region fusion, hard negatives, review mode, versioned models
- Fuse overlapping scan regions before display (merge adjacent 1s-hop windows)
- Hard negatives: mark false positives from scan panel for training feedback
  - Toggle with "Add to Negatives" button, red text + red timeline regions
  - Stored in dedicated hard_negatives table, always included in training
- Model versioning: auto-backup on retrain, right-click model combo to rollback
- Scan review mode: "Review" toggle hides spread/markers for free navigation
- Scan exports: saved to DB with scan_export flag, no timeline markers
  - Training dialog checkbox to optionally include scan exports
  - Single group folder per batch with area numbering (clip_042_a1_0.mp4)
- Export scan results: skip negatives, skip regions < 8s, respect spread
  - Button shows estimated clip count, updates on spread/fuse/negative changes
- Timeline: reload scan regions on file load, "Clear all markers" context menu
- Default training model changed to HUBERT_XLARGE

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 18:43:05 +02:00
Ethanfel 5a9e068903 fix: 6 bugs — profile isolation, export stashing, auto-negative guard
- Stash profile and crop_center at export start for async safety
- Scope get_group/delete_group by profile to prevent cross-profile leaks
- Guard auto-negative sampling when no markers exist (prevents flood)
- Wrap ffmpeg subprocess with clean timeout error message
- Fix scan-all panel reload to use stashed profile, not live value
- Remove dead warnings import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 16:28:51 +02:00
Ethanfel 6870e5aaf3 feat: scan results panel, model switching, batch scan, and training improvements
- Replace librosa with direct ffmpeg subprocess for 10x faster audio loading
- Add ScanResultsPanel with per-model tabs, seek-on-click, delete, and export
- Persist scan results in DB per (filename, profile, model)
- Add model selector dropdown to switch between trained embedding models
- Add "Scan All" button for batch scanning playlist videos
- Support manual negative examples via negative class folder
- Configurable auto-negative margin (default 30s, 0 = disabled)
- Deduplicate nearby training markers (8s min gap)
- Parallel audio loading with ThreadPoolExecutor during training
- Progress callbacks from training for UI status updates
- Cache bypass in scan_video (skip audio loading when embeddings cached)
- Move all caches (models, embeddings, downloads) into project directory
- Add 8cut.sh launcher script with auto venv/conda detection
- Fix 11 bugs across thread safety, signal handling, and state management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 16:12:52 +02:00
Ethanfel e1789d4e71 fix: bug audit — broken test imports, training data overlap, cleanup
- Fix test_utils.py importing build_annotation_json_path from main
  instead of core.annotations (all 59 tests pass now)
- Fix get_training_data double-counting clips at same start_time
  in both positive and soft sets — subtract positive from soft
- Add cancel_flag to train_classifier so training can be interrupted
  between videos (TrainWorker passes self as cancel_flag)
- Remove orphaned core/export.py (was for deleted server API)
- Remove stale Dockerfile and docker-compose.yml (referenced server)
- Clean up leftover server/__pycache__ and client/ build artifacts
- Add torch to requirements.txt (was only mentioned in comments)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 12:55:58 +02:00
Ethanfel 12ed183f1b feat: integrate training UI, BEATs model, and clean up legacy code
- Remove legacy distance-mode scanning (build_profile, _similarity, etc.)
  and hand-crafted intensity features — pipeline is now embedding-only
- Integrate Microsoft BEATs as embedding option alongside wav2vec2/HuBERT
- Add TrainDialog with positive class selector, model picker, video dir
  fallback, and live training stats
- Add TrainWorker QThread with cancel support and proper lifecycle cleanup
- Add source_path column to DB for robust source video tracking
- Add get_export_folders/get_training_data/get_training_stats to DB
- Wire source_path in all export DB writes (_on_clip_done, _on_auto_clip_done)
- Cancel scan/train workers in closeEvent to prevent use-after-free crashes
- Add setup_env.sh supporting both conda and python venv (CUDA 12.8)
- Update requirements.txt with all actual dependencies
- Update 8cut_train.py with --positive flag for new DB-driven training

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 11:52:27 +02:00
Ethanfel f2c38aee79 feat: rewrite audio scan with MFCC+delta+spectral contrast pipeline
Root cause of poor discrimination: MFCC[0] (energy) dominated the
feature vector, making cosine similarity see all audio as similar.

Changes:
- Skip MFCC[0], use 12 coefficients instead of 20
- Add delta MFCCs for temporal dynamics
- Add 7-band spectral contrast for tonal vs noise quality
- Switch from cosine similarity to euclidean-distance-based score
- Pre-compute STFT once for whole file (10-20x faster)
- Vectorized sliding window via cumulative sums (no Python loop)
- Lower sample rate 22050→16000 Hz (faster, no quality loss)
- 62-dim feature vector (was 40-dim mean+std of raw MFCCs)
- Default threshold 0.05 (new similarity scale)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 15:28:44 +02:00
Ethanfel c6c5934fe8 fix: threshold step 0.05 → 0.01 for finer control
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 09:21:14 +02:00
Ethanfel 73d5367424 fix: three audio scan bugs — signal shadow, re-entrancy, S-key jump
1. Rename ScanWorker.finished → scan_done to stop shadowing
   QThread.finished. Previously, cancelled scans leaked the QThread
   because the custom signal was never emitted.

2. Block signals on combobox reset in _on_scan_ref_changed to
   prevent re-entrant call when user cancels folder dialog.

3. Merge overlapping scan regions into clusters before S-key
   navigation so it jumps to the next distinct match, not 1s forward
   through overlapping windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 09:12:24 +02:00
Ethanfel 1e2cebd424 fix: prevent deleteLater on still-running ScanWorker QThread
When cancelling a scan during file change, connect finished signal
to deleteLater instead of calling it immediately on a running thread.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 09:02:35 +02:00
Ethanfel c439aca9b9 feat: add S shortcut and clear scan on file change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 08:59:47 +02:00
Ethanfel afda9b2d9f feat: add scan UI controls and start_scan handler
Add Scan button, threshold spinner, mode combobox, and reference source
combobox to the settings row. Implement handler methods for starting scans,
handling results/errors, cleanup of workers, and reference folder selection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 08:57:56 +02:00
Ethanfel 4cf54f2642 feat: add ScanWorker QThread for background scanning
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 08:54:20 +02:00
Ethanfel e7f4de9ec1 feat: timeline scan region rendering
Add scan region storage and rendering to TimelineWidget:
- _scan_regions list in __init__ for (start, end, score) tuples
- set_scan_regions() and clear_scan_regions() methods
- paintEvent draws semi-transparent blue rectangles with score-based opacity

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-17 08:53:18 +02:00
Ethanfel b09ba3fa9e fix: third-pass review bugs
- Switch DELETE /export to query param (path param strips leading /)
- Add CropKeyframe Pydantic model for typed keyframe validation
- Convert keyframes to tuples before passing to apply_keyframes_to_jobs
- Remove dead QFrame import from main.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 14:20:20 +02:00
Ethanfel fbbfa6fdce refactor: import shared logic from core/ instead of inline definitions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 13:43:44 +02:00
Ethanfel 7abf0b4d4c feat: autoclip, play/pause improvements, number key exports, focus fix
- Autoclip (A): adjusts clip count to fit current pause position
- Pause no longer resets playback position — stays where paused
- Play resumes from pause point instead of restarting
- Spread/clips changes update loop end without restarting playback
- Number keys 1-9 export to subprofiles
- Click-away clears focus from spinboxes so hotkeys work again
- Lock mode: double-click marker jumps cursor to end of clip span

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 13:19:21 +02:00
Ethanfel 9e5bd4a8ec feat: add subprofiles, live play loop update, fix lock mode scrub
- Subprofiles: lightweight export variants that append a suffix to the
  export folder (e.g. _soft, _intense). Each gets its own export button
  in the transport row. Managed via "+" menu, persisted in QSettings.
- Play loop now updates immediately when spread/clips spinboxes change.
- Lock mode: ignore stale mpv position updates while user is dragging
  to prevent the play position from jumping back.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:58:26 +02:00
Ethanfel 34d8ad1dc7 feat: add Windows setup script and launcher for running from source
- setup-windows.ps1: downloads libmpv DLL and ffmpeg, installs pip deps
- 8cut.bat: double-click launcher
- main.py: add_dll_directory for libmpv next to script (not just frozen)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 23:08:17 +02:00
Ethanfel 5b4e4bf818 feat: add PyInstaller spec and GitHub Actions release workflow
Build & Release / build (8cut-macos-arm64, macos-latest) (push) Has been cancelled
Build & Release / build (8cut-macos-x86_64, macos-13) (push) Has been cancelled
Build & Release / build (8cut-windows, windows-latest) (push) Has been cancelled
Build & Release / release (push) Has been cancelled
Enables cross-platform builds for Windows and macOS. Adds _bin() helper
to resolve bundled ffmpeg in frozen builds, and configures ctypes library
path for bundled libmpv.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 22:40:44 +02:00
Ethanfel bd4e97c45a fix: lock mode seek falls back to cursor instead of jumping to start
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 19:15:01 +02:00
Ethanfel 1aeaad7f6d fix: skip keyframe creation at frame 0 where base state applies
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:56:48 +02:00
Ethanfel 874632dffa fix: keep export complete message visible until next action
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:52:37 +02:00
Ethanfel 86055f2072 fix: defer preview follow so geometry is up-to-date after main window move
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:51:27 +02:00
Ethanfel 5fddb06354 fix: add right margin to panel, make Hide exported a QPushButton
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:48:51 +02:00
Ethanfel e60263548d feat: move status messages to inline label on settings row
Replace the bottom status bar with a right-aligned label on the
settings row, saving vertical space. Add "Export complete" message
when a batch finishes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:45:10 +02:00
Ethanfel 86f447f3d6 feat: add Show Hidden button to reveal and unhide playlist files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:39:22 +02:00
Ethanfel 1d5b8023a2 feat: auto-create/remove keyframes when toggling random crop in lock mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:29:11 +02:00
Ethanfel cb4392125d fix: scrub preview fallback before first keyframe + document overwrite behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:08:35 +02:00
Ethanfel 328c800d60 feat: apply keyframe crop modes in overwrite exports too
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:05:20 +02:00
Ethanfel 7a35e8268b feat: preview effective keyframe crop state during lock-mode scrub
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:05:08 +02:00
Ethanfel 465894af51 feat: color-code keyframe diamonds by crop mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 16:04:53 +02:00
Ethanfel 1004bd0a28 feat: rewrite export to use per-keyframe crop modes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:58:01 +02:00
Ethanfel 279aee14cb feat: add apply_keyframes_to_jobs helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:57:27 +02:00
Ethanfel 4f15f77175 feat: snapshot ratio and random flags into crop keyframes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:52:25 +02:00
Ethanfel 17e42c44b3 refactor: widen keyframe tuple to carry ratio and random flags
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:52:04 +02:00
Ethanfel 8e8c8b9774 feat: add resolve_keyframe helper to extract sorted-keyframe lookup
Adds a pure function that returns the latest keyframe at or before a
given time (with tolerance), replacing the inline lookup pattern that
appears multiple times in main.py. Includes 6 tests covering empty
list, before-first, exact match, between, after-last, and tolerance.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-14 15:48:00 +02:00