Fix caption cache timing, commit path tracking, and clean up start script

- Move caption cache refresh before UI render so indicators show on load
- Return actual dest paths from commit_batch/commit_global to fix
  caption-on-apply silently failing when files are renamed on collision
- Simplify start.sh to only run NiceGUI (remove Streamlit)
- Add requests to requirements.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

CLAUDE.md

@@ -0,0 +1,61 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Turbo Sorter Pro v12.5 - A dual-interface image organization tool combining Streamlit (admin dashboard) and NiceGUI (gallery interface) for managing large image collections through time-sync matching, ID collision resolution, category-based sorting, and gallery tagging with pairing capabilities.
+
+## Commands
+
+```bash
+# Install dependencies
+pip install -r requirements.txt
+
+# Run Streamlit dashboard (port 8501)
+streamlit run app.py --server.port=8501 --server.address=0.0.0.0
+
+# Run NiceGUI gallery (port 8080)
+python3 gallery_app.py
+
+# Both services (container startup)
+./start.sh
+
+# Syntax check all Python files
+python3 -m py_compile *.py
+```
+
+## Architecture
+
+### Dual-Framework Design
+- **Streamlit (app.py, port 8501)**: Administrative dashboard with 5 modular tabs for management workflows
+- **NiceGUI (gallery_app.py, port 8080)**: Modern gallery interface for image tagging and pairing operations
+- **Shared Backend**: Both UIs use `SorterEngine` (engine.py) and the same SQLite database
+
+### Core Components
+
+| File | Purpose |
+|------|---------|
+| `engine.py` | Static `SorterEngine` class - all DB operations, file handling, image compression |
+| `gallery_app.py` | NiceGUI gallery with `AppState` class for centralized state management |
+| `app.py` | Streamlit entry point, loads tab modules |
+| `tab_*.py` | Independent tab modules for each workflow |
+
+### Database
+SQLite at `/app/sorter_database.db` with tables: profiles, folder_ids, categories, staging_area, processed_log, folder_tags, profile_categories, pairing_settings.
+
+### Tab Workflows
+1. **Time-Sync Discovery** - Match images by timestamp across folders
+2. **ID Review** - Resolve ID collisions between target/control folders
+3. **Unused Archive** - Manage rejected image pairs
+4. **Category Sorter** - One-to-many categorization
+5. **Gallery Staged** - Grid-based tagging with Gallery/Pairing dual modes
+
+## Key Patterns
+
+- **ID Format**: `id001_`, `id002_` (zero-padded 3-digit prefix)
+- **Staging Pattern**: Two-phase commit (stage → commit) with undo support
+- **Image Formats**: .jpg, .jpeg, .png, .webp, .bmp, .tiff
+- **Compression**: WebP with ThreadPoolExecutor (8 workers)
+- **Permissions**: chmod 0o777 applied to committed files
+- **Default Paths**: `/storage` when not configured

engine.py

@@ -1,12 +1,31 @@
 import os
 import shutil
 import sqlite3
+import base64
+import requests
+from datetime import datetime
+from contextlib import contextmanager
 from PIL import Image
 from io import BytesIO
 
 class SorterEngine:
     DB_PATH = "/app/sorter_database.db"
 
+    @staticmethod
+    @contextmanager
+    def get_db():
+        """Context manager for database connections.
+        Ensures proper commit/rollback and always closes connection."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        try:
+            yield conn
+            conn.commit()
+        except Exception:
+            conn.rollback()
+            raise
+        finally:
+            conn.close()
+
     # --- 1. DATABASE INITIALIZATION ---
     @staticmethod
     def init_db():
@@ -27,21 +46,85 @@ class SorterEngine:
         cursor.execute('''CREATE TABLE IF NOT EXISTS processed_log 
             (source_path TEXT PRIMARY KEY, category TEXT, action_type TEXT)''')
         
-        # Seed categories if empty
+        # --- NEW: FOLDER TAGS TABLE (persists tags by folder) ---
+        # Check if old schema exists (without profile column) and migrate
+        cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='folder_tags'")
+        if cursor.fetchone():
+            cursor.execute("PRAGMA table_info(folder_tags)")
+            columns = [row[1] for row in cursor.fetchall()]
+            if 'profile' not in columns:
+                # Migrate: drop old table and recreate with profile column
+                cursor.execute("DROP TABLE folder_tags")
+                conn.commit()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS folder_tags
+            (profile TEXT, folder_path TEXT, filename TEXT, category TEXT, tag_index INTEGER,
+             PRIMARY KEY (profile, folder_path, filename))''')
+        
+        # --- NEW: PROFILE CATEGORIES TABLE (each profile has its own categories) ---
+        cursor.execute('''CREATE TABLE IF NOT EXISTS profile_categories 
+            (profile TEXT, category TEXT, PRIMARY KEY (profile, category))''')
+        
+        # --- NEW: PAIRING SETTINGS TABLE ---
+        cursor.execute('''CREATE TABLE IF NOT EXISTS pairing_settings 
+            (profile TEXT PRIMARY KEY, 
+             adjacent_folder TEXT, 
+             main_category TEXT, 
+             adj_category TEXT, 
+             main_output TEXT, 
+             adj_output TEXT, 
+             time_window INTEGER)''')
+        
+        # Seed categories if empty (legacy table)
         cursor.execute("SELECT COUNT(*) FROM categories")
         if cursor.fetchone()[0] == 0:
-            for cat in ["_TRASH", "Default", "Action", "Solo"]:
+            for cat in ["_TRASH", "control", "Default", "Action", "Solo"]:
                 cursor.execute("INSERT OR IGNORE INTO categories VALUES (?)", (cat,))
 
+        # --- CAPTION TABLES ---
+        # Per-category prompt templates
+        cursor.execute('''CREATE TABLE IF NOT EXISTS category_prompts
+            (profile TEXT, category TEXT, prompt_template TEXT,
+             PRIMARY KEY (profile, category))''')
+
+        # Stored captions
+        cursor.execute('''CREATE TABLE IF NOT EXISTS image_captions
+            (image_path TEXT PRIMARY KEY, caption TEXT, model TEXT,
+             generated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)''')
+
+        # Caption API settings per profile
+        cursor.execute('''CREATE TABLE IF NOT EXISTS caption_settings
+            (profile TEXT PRIMARY KEY,
+             api_endpoint TEXT DEFAULT 'http://localhost:8080/v1/chat/completions',
+             model_name TEXT DEFAULT 'local-model',
+             max_tokens INTEGER DEFAULT 300,
+             temperature REAL DEFAULT 0.7,
+             timeout_seconds INTEGER DEFAULT 60,
+             batch_size INTEGER DEFAULT 4)''')
+
+        # --- PERFORMANCE INDEXES ---
+        # Index for staging_area queries filtered by category
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_staging_category ON staging_area(target_category)")
+        # Index for folder_tags queries filtered by profile and folder_path
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_folder_tags_profile ON folder_tags(profile, folder_path)")
+        # Index for profile_categories lookups
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_profile_categories ON profile_categories(profile)")
+        # Index for caption lookups by image path
+        cursor.execute("CREATE INDEX IF NOT EXISTS idx_image_captions ON image_captions(image_path)")
+
         conn.commit()
         conn.close()
 
     # --- 2. PROFILE & PATH MANAGEMENT ---
     @staticmethod
-    def save_tab_paths(profile_name, t1_t=None, t2_t=None, t2_c=None, t4_s=None, t4_o=None, mode=None, t5_s=None, t5_o=None):
+    def save_tab_paths(profile_name, t1_t=None, t2_t=None, t2_c=None, t4_s=None, t4_o=None, mode=None, t5_s=None, t5_o=None,
+                       pair_adjacent_folder=None, pair_main_category=None, pair_adj_category=None, 
+                       pair_main_output=None, pair_adj_output=None, pair_time_window=None):
         """Updates specific tab paths in the database while preserving others."""
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
+        
+        # Save main profile settings
         cursor.execute("SELECT * FROM profiles WHERE name = ?", (profile_name,))
         row = cursor.fetchone()
         
@@ -61,6 +144,38 @@ class SorterEngine:
             t5_o if t5_o is not None else row[8]
         )
         cursor.execute("INSERT OR REPLACE INTO profiles VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)", new_values)
+        
+        # Save pairing settings if any are provided
+        if any(x is not None for x in [pair_adjacent_folder, pair_main_category, pair_adj_category, 
+                                        pair_main_output, pair_adj_output, pair_time_window]):
+            # Ensure table exists
+            cursor.execute('''CREATE TABLE IF NOT EXISTS pairing_settings 
+                (profile TEXT PRIMARY KEY, 
+                 adjacent_folder TEXT, 
+                 main_category TEXT, 
+                 adj_category TEXT, 
+                 main_output TEXT, 
+                 adj_output TEXT, 
+                 time_window INTEGER)''')
+            
+            # Get existing values
+            cursor.execute("SELECT * FROM pairing_settings WHERE profile = ?", (profile_name,))
+            pair_row = cursor.fetchone()
+            
+            if not pair_row:
+                pair_row = (profile_name, "", "control", "control", "/storage", "/storage", 60)
+            
+            pair_values = (
+                profile_name,
+                pair_adjacent_folder if pair_adjacent_folder is not None else pair_row[1],
+                pair_main_category if pair_main_category is not None else pair_row[2],
+                pair_adj_category if pair_adj_category is not None else pair_row[3],
+                pair_main_output if pair_main_output is not None else pair_row[4],
+                pair_adj_output if pair_adj_output is not None else pair_row[5],
+                pair_time_window if pair_time_window is not None else pair_row[6]
+            )
+            cursor.execute("INSERT OR REPLACE INTO pairing_settings VALUES (?, ?, ?, ?, ?, ?, ?)", pair_values)
+        
         conn.commit()
         conn.close()
     @staticmethod
@@ -91,33 +206,90 @@ class SorterEngine:
 
     @staticmethod
     def load_profiles():
-        """Loads all workspace presets."""
+        """Loads all workspace presets including pairing settings.
+        Uses LEFT JOIN to fetch all data in a single query (fixes N+1 problem)."""
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
-        cursor.execute("SELECT * FROM profiles")
-        rows = cursor.fetchall()
-        conn.close()
-        return {r[0]: {
-            "tab1_target": r[1], "tab2_target": r[2], "tab2_control": r[3], 
-            "tab4_source": r[4], "tab4_out": r[5], "mode": r[6],
-            "tab5_source": r[7], "tab5_out": r[8]
-        } for r in rows}
 
-    # --- 3. CATEGORY MANAGEMENT (Sorted A-Z) ---
+        # Ensure pairing_settings table exists before JOIN
+        cursor.execute('''CREATE TABLE IF NOT EXISTS pairing_settings
+            (profile TEXT PRIMARY KEY,
+             adjacent_folder TEXT,
+             main_category TEXT,
+             adj_category TEXT,
+             main_output TEXT,
+             adj_output TEXT,
+             time_window INTEGER)''')
+
+        # Single query with LEFT JOIN - eliminates N+1 queries
+        cursor.execute('''
+            SELECT p.name, p.tab1_target, p.tab2_target, p.tab2_control,
+                   p.tab4_source, p.tab4_out, p.mode, p.tab5_source, p.tab5_out,
+                   ps.adjacent_folder, ps.main_category, ps.adj_category,
+                   ps.main_output, ps.adj_output, ps.time_window
+            FROM profiles p
+            LEFT JOIN pairing_settings ps ON p.name = ps.profile
+        ''')
+        rows = cursor.fetchall()
+
+        profiles = {}
+        for r in rows:
+            profile_name = r[0]
+            profiles[profile_name] = {
+                "tab1_target": r[1], "tab2_target": r[2], "tab2_control": r[3],
+                "tab4_source": r[4], "tab4_out": r[5], "mode": r[6],
+                "tab5_source": r[7], "tab5_out": r[8],
+                # Pairing settings from JOIN (with defaults for NULL)
+                "pair_adjacent_folder": r[9] or "",
+                "pair_main_category": r[10] or "control",
+                "pair_adj_category": r[11] or "control",
+                "pair_main_output": r[12] or "/storage",
+                "pair_adj_output": r[13] or "/storage",
+                "pair_time_window": r[14] or 60
+            }
+
+        conn.close()
+        return profiles
+
+    # --- 3. CATEGORY MANAGEMENT (Profile-based) ---
     @staticmethod
-    def get_categories():
+    def get_categories(profile=None):
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
-        cursor.execute("SELECT name FROM categories ORDER BY name COLLATE NOCASE ASC")
-        cats = [r[0] for r in cursor.fetchall()]
+        
+        # Ensure table exists
+        cursor.execute('''CREATE TABLE IF NOT EXISTS profile_categories 
+            (profile TEXT, category TEXT, PRIMARY KEY (profile, category))''')
+        
+        if profile:
+            cursor.execute("SELECT category FROM profile_categories WHERE profile = ? ORDER BY category COLLATE NOCASE ASC", (profile,))
+            cats = [r[0] for r in cursor.fetchall()]
+            # If no categories for this profile, seed with defaults
+            if not cats:
+                for cat in ["_TRASH", "control"]:
+                    cursor.execute("INSERT OR IGNORE INTO profile_categories VALUES (?, ?)", (profile, cat))
+                conn.commit()
+                cats = ["_TRASH", "control"]
+        else:
+            # Fallback to legacy table
+            cursor.execute("SELECT name FROM categories ORDER BY name COLLATE NOCASE ASC")
+            cats = [r[0] for r in cursor.fetchall()]
+        
         conn.close()
         return cats
 
     @staticmethod
-    def add_category(name):
+    def add_category(name, profile=None):
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
-        cursor.execute("INSERT OR IGNORE INTO categories VALUES (?)", (name,))
+        
+        if profile:
+            cursor.execute('''CREATE TABLE IF NOT EXISTS profile_categories 
+                (profile TEXT, category TEXT, PRIMARY KEY (profile, category))''')
+            cursor.execute("INSERT OR IGNORE INTO profile_categories VALUES (?, ?)", (profile, name))
+        else:
+            cursor.execute("INSERT OR IGNORE INTO categories VALUES (?)", (name,))
+        
         conn.commit()
         conn.close()
 
@@ -153,20 +325,42 @@ class SorterEngine:
 
     # --- 4. IMAGE & ID OPERATIONS ---
     @staticmethod
-    def get_images(path, recursive=False):
-        """Image scanner with optional recursive subfolder support."""
+    def get_images(path, recursive=False, exclude_paths=None):
+        """Image scanner with optional recursive subfolder support.
+        
+        Args:
+            path: Directory to scan
+            recursive: Whether to scan subdirectories
+            exclude_paths: List of paths to exclude from scanning
+        """
         exts = ('.jpg', '.jpeg', '.png', '.webp', '.bmp', '.tiff')
         if not path or not os.path.exists(path): return []
+        
+        exclude_paths = exclude_paths or []
+        # Normalize exclude paths
+        exclude_normalized = [os.path.normpath(os.path.abspath(p)) for p in exclude_paths]
+        
         image_list = []
         if recursive:
-            for root, _, files in os.walk(path):
+            for root, dirs, files in os.walk(path):
                 # Skip the trash folder from scanning
                 if "_DELETED" in root: continue
+                
+                # Skip excluded paths
+                root_normalized = os.path.normpath(os.path.abspath(root))
+                if any(root_normalized.startswith(exc) or exc.startswith(root_normalized) for exc in exclude_normalized):
+                    # Remove excluded dirs from dirs to prevent descending into them
+                    dirs[:] = [d for d in dirs if os.path.normpath(os.path.abspath(os.path.join(root, d))) not in exclude_normalized]
+                    if root_normalized in exclude_normalized:
+                        continue
+                
                 for f in files:
-                    if f.lower().endswith(exts): image_list.append(os.path.join(root, f))
+                    if f.lower().endswith(exts): 
+                        image_list.append(os.path.join(root, f))
         else:
             for f in os.listdir(path):
-                if f.lower().endswith(exts): image_list.append(os.path.join(path, f))
+                if f.lower().endswith(exts): 
+                    image_list.append(os.path.join(path, f))
         return sorted(image_list)
 
     @staticmethod
@@ -226,36 +420,45 @@ class SorterEngine:
     @staticmethod
     def stage_image(original_path, category, new_name):
         """Records a pending rename/move in the database."""
-        conn = sqlite3.connect(SorterEngine.DB_PATH)
-        cursor = conn.cursor()
-        cursor.execute("INSERT OR REPLACE INTO staging_area VALUES (?, ?, ?, 1)", (original_path, category, new_name))
-        conn.commit()
-        conn.close()
+        with SorterEngine.get_db() as conn:
+            cursor = conn.cursor()
+            cursor.execute("INSERT OR REPLACE INTO staging_area VALUES (?, ?, ?, 1)", (original_path, category, new_name))
 
     @staticmethod
     def clear_staged_item(original_path):
         """Removes an item from the pending staging area."""
-        conn = sqlite3.connect(SorterEngine.DB_PATH)
-        cursor = conn.cursor()
-        cursor.execute("DELETE FROM staging_area WHERE original_path = ?", (original_path,))
-        conn.commit()
-        conn.close()
+        with SorterEngine.get_db() as conn:
+            cursor = conn.cursor()
+            cursor.execute("DELETE FROM staging_area WHERE original_path = ?", (original_path,))
+
+    @staticmethod
+    def clear_staging_area():
+        """Clears all items from the staging area."""
+        with SorterEngine.get_db() as conn:
+            cursor = conn.cursor()
+            cursor.execute("DELETE FROM staging_area")
 
     @staticmethod
     def get_staged_data():
         """Retrieves current tagged/staged images."""
-        conn = sqlite3.connect(SorterEngine.DB_PATH)
-        cursor = conn.cursor()
-        cursor.execute("SELECT * FROM staging_area")
-        rows = cursor.fetchall()
-        conn.close()
-        # FIXED: Added "marked": r[3] to the dictionary
-        return {r[0]: {"cat": r[1], "name": r[2], "marked": r[3]} for r in rows}
+        with SorterEngine.get_db() as conn:
+            cursor = conn.cursor()
+            cursor.execute("SELECT * FROM staging_area")
+            rows = cursor.fetchall()
+            # FIXED: Added "marked": r[3] to the dictionary
+            return {r[0]: {"cat": r[1], "name": r[2], "marked": r[3]} for r in rows}
         
     @staticmethod
-    def commit_global(output_root, cleanup_mode, operation="Copy", source_root=None):
-        """Commits ALL staged files and fixes permissions."""
+    def commit_global(output_root, cleanup_mode, operation="Copy", source_root=None, profile=None):
+        """Commits ALL staged files and fixes permissions.
+        Returns dict mapping original_path -> {dest, cat} for committed files."""
         data = SorterEngine.get_staged_data()
+        committed = {}
+
+        # Save folder tags BEFORE processing (so we can restore them later)
+        if source_root:
+            SorterEngine.save_folder_tags(source_root, profile)
+
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
 
@@ -281,6 +484,9 @@ class SorterEngine:
                 # --- FIX PERMISSIONS ---
                 SorterEngine.fix_permissions(final_dst)
 
+                # Track actual destination
+                committed[old_p] = {"dest": final_dst, "cat": info['cat']}
+
                 # Log History
                 cursor.execute("INSERT OR REPLACE INTO processed_log VALUES (?, ?, ?)",
                                (old_p, info['cat'], operation))
@@ -304,6 +510,7 @@ class SorterEngine:
         cursor.execute("DELETE FROM staging_area")
         conn.commit()
         conn.close()
+        return committed
 
     # --- 6. CORE UTILITIES (SYNC & UNDO) ---
     @staticmethod
@@ -415,10 +622,12 @@ class SorterEngine:
 
     @staticmethod
     def commit_batch(file_list, output_root, cleanup_mode, operation="Copy"):
-        """Commits files and fixes permissions."""
+        """Commits files and fixes permissions.
+        Returns dict mapping original_path -> actual_dest_path for committed files."""
         data = SorterEngine.get_staged_data()
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
+        committed = {}
 
         if not os.path.exists(output_root): os.makedirs(output_root, exist_ok=True)
 
@@ -447,6 +656,9 @@ class SorterEngine:
                 # --- FIX PERMISSIONS ---
                 SorterEngine.fix_permissions(final_dst)
 
+                # Track actual destination
+                committed[file_path] = {"dest": final_dst, "cat": info['cat']}
+
                 # Update DB
                 cursor.execute("DELETE FROM staging_area WHERE original_path = ?", (file_path,))
                 cursor.execute("INSERT OR REPLACE INTO processed_log VALUES (?, ?, ?)",
@@ -467,6 +679,7 @@ class SorterEngine:
 
         conn.commit()
         conn.close()
+        return committed
 
     @staticmethod
     def rename_category(old_name, new_name):
@@ -494,11 +707,16 @@ class SorterEngine:
             conn.close()
 
     @staticmethod
-    def delete_category(name):
+    def delete_category(name, profile=None):
         """Deletes a category and clears any staged tags associated with it."""
         conn = sqlite3.connect(SorterEngine.DB_PATH)
         cursor = conn.cursor()
-        cursor.execute("DELETE FROM categories WHERE name = ?", (name,))
+        
+        if profile:
+            cursor.execute("DELETE FROM profile_categories WHERE profile = ? AND category = ?", (profile, name))
+        else:
+            cursor.execute("DELETE FROM categories WHERE name = ?", (name,))
+        
         cursor.execute("DELETE FROM staging_area WHERE target_category = ?", (name,))
         conn.commit()
         conn.close()
@@ -514,3 +732,528 @@ class SorterEngine:
             if img_path in staged_keys:
                 tagged_pages.add(idx // page_size)
         return tagged_pages
+
+    # --- 7. FOLDER TAG PERSISTENCE ---
+    @staticmethod
+    def save_folder_tags(folder_path, profile=None):
+        """
+        Saves current staging data associated with a folder for later restoration.
+        Call this BEFORE clearing the staging area.
+        """
+        import re
+        staged = SorterEngine.get_staged_data()
+        if not staged:
+            return 0
+        
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+        
+        # Ensure table exists with profile column
+        cursor.execute('''CREATE TABLE IF NOT EXISTS folder_tags 
+            (profile TEXT, folder_path TEXT, filename TEXT, category TEXT, tag_index INTEGER,
+             PRIMARY KEY (profile, folder_path, filename))''')
+        
+        # Check if old schema (without profile) - migrate if needed
+        cursor.execute("PRAGMA table_info(folder_tags)")
+        columns = [row[1] for row in cursor.fetchall()]
+        if 'profile' not in columns:
+            cursor.execute("DROP TABLE folder_tags")
+            cursor.execute('''CREATE TABLE folder_tags 
+                (profile TEXT, folder_path TEXT, filename TEXT, category TEXT, tag_index INTEGER,
+                 PRIMARY KEY (profile, folder_path, filename))''')
+            conn.commit()
+        
+        profile = profile or "Default"
+        saved_count = 0
+        for orig_path, info in staged.items():
+            # Only save tags for files that are in this folder (or subfolders)
+            if orig_path.startswith(folder_path):
+                filename = os.path.basename(orig_path)
+                category = info['cat']
+                
+                # Extract index from the new_name (e.g., "Action_042.jpg" -> 42)
+                new_name = info['name']
+                match = re.search(r'_(\d+)', new_name)
+                tag_index = int(match.group(1)) if match else 0
+                
+                cursor.execute(
+                    "INSERT OR REPLACE INTO folder_tags VALUES (?, ?, ?, ?, ?)",
+                    (profile, folder_path, filename, category, tag_index)
+                )
+                saved_count += 1
+        
+        conn.commit()
+        conn.close()
+        return saved_count
+
+    @staticmethod
+    def restore_folder_tags(folder_path, all_images, profile=None):
+        """
+        Restores previously saved tags for a folder back into the staging area.
+        Call this when loading/reloading a folder.
+        Returns the number of tags restored.
+        """
+        try:
+            conn = sqlite3.connect(SorterEngine.DB_PATH)
+            cursor = conn.cursor()
+            
+            # Ensure table exists with profile column
+            cursor.execute('''CREATE TABLE IF NOT EXISTS folder_tags 
+                (profile TEXT, folder_path TEXT, filename TEXT, category TEXT, tag_index INTEGER,
+                 PRIMARY KEY (profile, folder_path, filename))''')
+            
+            # Check if old schema (without profile) - migrate if needed
+            cursor.execute("PRAGMA table_info(folder_tags)")
+            columns = [row[1] for row in cursor.fetchall()]
+            if 'profile' not in columns:
+                cursor.execute("DROP TABLE folder_tags")
+                cursor.execute('''CREATE TABLE folder_tags 
+                    (profile TEXT, folder_path TEXT, filename TEXT, category TEXT, tag_index INTEGER,
+                     PRIMARY KEY (profile, folder_path, filename))''')
+                conn.commit()
+            
+            profile = profile or "Default"
+            
+            # Get saved tags for this folder and profile
+            cursor.execute(
+                "SELECT filename, category, tag_index FROM folder_tags WHERE profile = ? AND folder_path = ?",
+                (profile, folder_path)
+            )
+            saved_tags = {row[0]: {"cat": row[1], "index": row[2]} for row in cursor.fetchall()}
+            
+            if not saved_tags:
+                conn.close()
+                return 0
+            
+            # Build a map of filename -> full path from current images
+            filename_to_path = {}
+            for img_path in all_images:
+                fname = os.path.basename(img_path)
+                if fname not in filename_to_path:
+                    filename_to_path[fname] = img_path
+            
+            # Restore tags to staging area
+            restored = 0
+            for filename, tag_info in saved_tags.items():
+                if filename in filename_to_path:
+                    full_path = filename_to_path[filename]
+                    cursor.execute("SELECT 1 FROM staging_area WHERE original_path = ?", (full_path,))
+                    if not cursor.fetchone():
+                        ext = os.path.splitext(filename)[1]
+                        new_name = f"{tag_info['cat']}_{tag_info['index']:03d}{ext}"
+                        cursor.execute(
+                            "INSERT OR REPLACE INTO staging_area VALUES (?, ?, ?, 1)",
+                            (full_path, tag_info['cat'], new_name)
+                        )
+                        restored += 1
+            
+            conn.commit()
+            conn.close()
+            return restored
+        except Exception as e:
+            print(f"Error restoring folder tags: {e}")
+            return 0
+
+    @staticmethod
+    def clear_folder_tags(folder_path):
+        """Clears saved tags for a specific folder."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+        cursor.execute("DELETE FROM folder_tags WHERE folder_path = ?", (folder_path,))
+        conn.commit()
+        conn.close()
+
+    @staticmethod
+    def get_saved_folder_tags(folder_path):
+        """Returns saved tags for a folder (for debugging/display)."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+        cursor.execute(
+            "SELECT filename, category, tag_index FROM folder_tags WHERE folder_path = ?",
+            (folder_path,)
+        )
+        result = {row[0]: {"cat": row[1], "index": row[2]} for row in cursor.fetchall()}
+        conn.close()
+        return result
+
+    # --- 8. CAPTION SETTINGS & PROMPTS ---
+    @staticmethod
+    def get_caption_settings(profile):
+        """Get caption API settings for a profile."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        # Ensure table exists
+        cursor.execute('''CREATE TABLE IF NOT EXISTS caption_settings
+            (profile TEXT PRIMARY KEY,
+             api_endpoint TEXT DEFAULT 'http://localhost:8080/v1/chat/completions',
+             model_name TEXT DEFAULT 'local-model',
+             max_tokens INTEGER DEFAULT 300,
+             temperature REAL DEFAULT 0.7,
+             timeout_seconds INTEGER DEFAULT 60,
+             batch_size INTEGER DEFAULT 4)''')
+
+        cursor.execute("SELECT * FROM caption_settings WHERE profile = ?", (profile,))
+        row = cursor.fetchone()
+        conn.close()
+
+        if row:
+            return {
+                "profile": row[0],
+                "api_endpoint": row[1],
+                "model_name": row[2],
+                "max_tokens": row[3],
+                "temperature": row[4],
+                "timeout_seconds": row[5],
+                "batch_size": row[6]
+            }
+        else:
+            # Return defaults
+            return {
+                "profile": profile,
+                "api_endpoint": "http://localhost:8080/v1/chat/completions",
+                "model_name": "local-model",
+                "max_tokens": 300,
+                "temperature": 0.7,
+                "timeout_seconds": 60,
+                "batch_size": 4
+            }
+
+    @staticmethod
+    def save_caption_settings(profile, api_endpoint=None, model_name=None, max_tokens=None,
+                              temperature=None, timeout_seconds=None, batch_size=None):
+        """Save caption API settings for a profile."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        # Ensure table exists
+        cursor.execute('''CREATE TABLE IF NOT EXISTS caption_settings
+            (profile TEXT PRIMARY KEY,
+             api_endpoint TEXT DEFAULT 'http://localhost:8080/v1/chat/completions',
+             model_name TEXT DEFAULT 'local-model',
+             max_tokens INTEGER DEFAULT 300,
+             temperature REAL DEFAULT 0.7,
+             timeout_seconds INTEGER DEFAULT 60,
+             batch_size INTEGER DEFAULT 4)''')
+
+        # Get existing values
+        cursor.execute("SELECT * FROM caption_settings WHERE profile = ?", (profile,))
+        row = cursor.fetchone()
+
+        if not row:
+            row = (profile, "http://localhost:8080/v1/chat/completions", "local-model", 300, 0.7, 60, 4)
+
+        new_values = (
+            profile,
+            api_endpoint if api_endpoint is not None else row[1],
+            model_name if model_name is not None else row[2],
+            max_tokens if max_tokens is not None else row[3],
+            temperature if temperature is not None else row[4],
+            timeout_seconds if timeout_seconds is not None else row[5],
+            batch_size if batch_size is not None else row[6]
+        )
+
+        cursor.execute("INSERT OR REPLACE INTO caption_settings VALUES (?, ?, ?, ?, ?, ?, ?)", new_values)
+        conn.commit()
+        conn.close()
+
+    @staticmethod
+    def get_category_prompt(profile, category):
+        """Get prompt template for a category."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS category_prompts
+            (profile TEXT, category TEXT, prompt_template TEXT,
+             PRIMARY KEY (profile, category))''')
+
+        cursor.execute(
+            "SELECT prompt_template FROM category_prompts WHERE profile = ? AND category = ?",
+            (profile, category)
+        )
+        row = cursor.fetchone()
+        conn.close()
+
+        if row and row[0]:
+            return row[0]
+        else:
+            # Default prompt
+            return "Describe this image in detail for training purposes. Include subjects, actions, setting, colors, and composition."
+
+    @staticmethod
+    def save_category_prompt(profile, category, prompt):
+        """Save prompt template for a category."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS category_prompts
+            (profile TEXT, category TEXT, prompt_template TEXT,
+             PRIMARY KEY (profile, category))''')
+
+        cursor.execute(
+            "INSERT OR REPLACE INTO category_prompts VALUES (?, ?, ?)",
+            (profile, category, prompt)
+        )
+        conn.commit()
+        conn.close()
+
+    @staticmethod
+    def get_all_category_prompts(profile):
+        """Get all prompt templates for a profile."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS category_prompts
+            (profile TEXT, category TEXT, prompt_template TEXT,
+             PRIMARY KEY (profile, category))''')
+
+        cursor.execute(
+            "SELECT category, prompt_template FROM category_prompts WHERE profile = ?",
+            (profile,)
+        )
+        result = {row[0]: row[1] for row in cursor.fetchall()}
+        conn.close()
+        return result
+
+    # --- 9. CAPTION STORAGE ---
+    @staticmethod
+    def save_caption(image_path, caption, model):
+        """Save a generated caption to the database."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS image_captions
+            (image_path TEXT PRIMARY KEY, caption TEXT, model TEXT,
+             generated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)''')
+
+        cursor.execute(
+            "INSERT OR REPLACE INTO image_captions VALUES (?, ?, ?, ?)",
+            (image_path, caption, model, datetime.now().isoformat())
+        )
+        conn.commit()
+        conn.close()
+
+    @staticmethod
+    def get_caption(image_path):
+        """Get caption for an image."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS image_captions
+            (image_path TEXT PRIMARY KEY, caption TEXT, model TEXT,
+             generated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)''')
+
+        cursor.execute(
+            "SELECT caption, model, generated_at FROM image_captions WHERE image_path = ?",
+            (image_path,)
+        )
+        row = cursor.fetchone()
+        conn.close()
+
+        if row:
+            return {"caption": row[0], "model": row[1], "generated_at": row[2]}
+        return None
+
+    @staticmethod
+    def get_captions_batch(image_paths):
+        """Get captions for multiple images."""
+        if not image_paths:
+            return {}
+
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS image_captions
+            (image_path TEXT PRIMARY KEY, caption TEXT, model TEXT,
+             generated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)''')
+
+        placeholders = ','.join('?' * len(image_paths))
+        cursor.execute(
+            f"SELECT image_path, caption, model, generated_at FROM image_captions WHERE image_path IN ({placeholders})",
+            image_paths
+        )
+        result = {row[0]: {"caption": row[1], "model": row[2], "generated_at": row[3]} for row in cursor.fetchall()}
+        conn.close()
+        return result
+
+    @staticmethod
+    def delete_caption(image_path):
+        """Delete caption for an image."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+        cursor.execute("DELETE FROM image_captions WHERE image_path = ?", (image_path,))
+        conn.commit()
+        conn.close()
+
+    @staticmethod
+    def get_all_caption_paths():
+        """Get set of all image paths that have captions."""
+        conn = sqlite3.connect(SorterEngine.DB_PATH)
+        cursor = conn.cursor()
+
+        cursor.execute('''CREATE TABLE IF NOT EXISTS image_captions
+            (image_path TEXT PRIMARY KEY, caption TEXT, model TEXT,
+             generated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)''')
+
+        cursor.execute("SELECT image_path FROM image_captions")
+        result = {row[0] for row in cursor.fetchall()}
+        conn.close()
+        return result
+
+    # --- 10. VLLM API CAPTIONING ---
+    @staticmethod
+    def caption_image_vllm(image_path, prompt, settings):
+        """
+        Generate caption for an image using VLLM API.
+
+        Args:
+            image_path: Path to the image file
+            prompt: Text prompt for captioning
+            settings: Dict with api_endpoint, model_name, max_tokens, temperature, timeout_seconds
+
+        Returns:
+            Tuple of (caption_text, error_message). If successful, error is None.
+        """
+        try:
+            # Read and encode image
+            with open(image_path, 'rb') as f:
+                img_bytes = f.read()
+            b64_image = base64.b64encode(img_bytes).decode('utf-8')
+
+            # Determine MIME type
+            ext = os.path.splitext(image_path)[1].lower()
+            mime_types = {
+                '.jpg': 'image/jpeg',
+                '.jpeg': 'image/jpeg',
+                '.png': 'image/png',
+                '.webp': 'image/webp',
+                '.bmp': 'image/bmp',
+                '.tiff': 'image/tiff'
+            }
+            mime_type = mime_types.get(ext, 'image/jpeg')
+
+            # Build request payload (OpenAI-compatible format)
+            payload = {
+                "model": settings.get('model_name', 'local-model'),
+                "messages": [{
+                    "role": "user",
+                    "content": [
+                        {"type": "text", "text": prompt},
+                        {"type": "image_url", "image_url": {"url": f"data:{mime_type};base64,{b64_image}"}}
+                    ]
+                }],
+                "max_tokens": settings.get('max_tokens', 300),
+                "temperature": settings.get('temperature', 0.7)
+            }
+
+            # Make API request
+            response = requests.post(
+                settings.get('api_endpoint', 'http://localhost:8080/v1/chat/completions'),
+                json=payload,
+                timeout=settings.get('timeout_seconds', 60)
+            )
+            response.raise_for_status()
+
+            result = response.json()
+            caption = result['choices'][0]['message']['content']
+            return caption.strip(), None
+
+        except requests.Timeout:
+            return None, f"API timeout after {settings.get('timeout_seconds', 60)}s"
+        except requests.RequestException as e:
+            return None, f"API error: {str(e)}"
+        except KeyError as e:
+            return None, f"Invalid API response: missing {str(e)}"
+        except Exception as e:
+            return None, f"Error: {str(e)}"
+
+    @staticmethod
+    def caption_batch_vllm(image_paths, get_prompt_fn, settings, progress_cb=None):
+        """
+        Caption multiple images using VLLM API.
+
+        Args:
+            image_paths: List of (image_path, category) tuples
+            get_prompt_fn: Function(category) -> prompt string
+            settings: Caption settings dict
+            progress_cb: Optional callback(current, total, status_msg) for progress updates
+
+        Returns:
+            Dict with results: {"success": count, "failed": count, "captions": {path: caption}}
+        """
+        results = {"success": 0, "failed": 0, "captions": {}, "errors": {}}
+        total = len(image_paths)
+
+        for i, (image_path, category) in enumerate(image_paths):
+            if progress_cb:
+                progress_cb(i, total, f"Captioning {os.path.basename(image_path)}...")
+
+            prompt = get_prompt_fn(category)
+            caption, error = SorterEngine.caption_image_vllm(image_path, prompt, settings)
+
+            if caption:
+                # Save to database
+                SorterEngine.save_caption(image_path, caption, settings.get('model_name', 'local-model'))
+                results["captions"][image_path] = caption
+                results["success"] += 1
+            else:
+                # Store error
+                error_caption = f"[ERROR] {error}"
+                SorterEngine.save_caption(image_path, error_caption, settings.get('model_name', 'local-model'))
+                results["errors"][image_path] = error
+                results["failed"] += 1
+
+        if progress_cb:
+            progress_cb(total, total, "Complete!")
+
+        return results
+
+    @staticmethod
+    def write_caption_sidecar(image_path, caption):
+        """
+        Write caption to a .txt sidecar file next to the image.
+
+        Args:
+            image_path: Path to the image file
+            caption: Caption text to write
+
+        Returns:
+            Path to sidecar file, or None on error
+        """
+        try:
+            # Create sidecar path (same name, .txt extension)
+            base_path = os.path.splitext(image_path)[0]
+            sidecar_path = f"{base_path}.txt"
+
+            with open(sidecar_path, 'w', encoding='utf-8') as f:
+                f.write(caption)
+
+            # Fix permissions
+            SorterEngine.fix_permissions(sidecar_path)
+
+            return sidecar_path
+        except Exception as e:
+            print(f"Warning: Could not write sidecar for {image_path}: {e}")
+            return None
+
+    @staticmethod
+    def read_caption_sidecar(image_path):
+        """
+        Read caption from a .txt sidecar file if it exists.
+
+        Args:
+            image_path: Path to the image file
+
+        Returns:
+            Caption text or None if no sidecar exists
+        """
+        try:
+            base_path = os.path.splitext(image_path)[0]
+            sidecar_path = f"{base_path}.txt"
+
+            if os.path.exists(sidecar_path):
+                with open(sidecar_path, 'r', encoding='utf-8') as f:
+                    return f.read().strip()
+        except Exception:
+            pass
+        return None

requirements.txt

@@ -1,2 +1,4 @@
 streamlit
 Pillow
+nicegui
+requests

start.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+# NiceSorter - Start Script
+# Runs the NiceGUI gallery interface on port 8080
+
+set -e
+
+# Navigate to app directory if running in container
+if [ -d "/app" ]; then
+    cd /app
+fi
+
+# Install dependencies if requirements.txt exists
+if [ -f "requirements.txt" ]; then
+    echo "📦 Installing dependencies..."
+    pip install --no-cache-dir -q -r requirements.txt
+fi
+
+# Initialize database
+echo "🗄️ Initializing database..."
+python3 -c "from engine import SorterEngine; SorterEngine.init_db()"
+
+# Start NiceGUI
+echo "🚀 Starting NiceSorter on http://0.0.0.0:8080"
+exec python3 gallery_app.py