History tree nodes stored full data snapshots in memory (5-50MB each),
accumulating with every save. Now:
- New `history_snapshots` DB table stores node data separately
- `save_history_tree` and `sync_to_db` extract snapshots before saving
- In-memory tree nodes only hold metadata (id, parent, note, timestamp)
- Restore and preview load snapshots from DB on demand
- `save_and_snap` uses json roundtrip instead of deepcopy (1 copy not 2)
- `_src_cache` moved to AppState, cleared on file switch
- `strip_snapshots()` method on HistoryTree for explicit cleanup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Combines separate lora name and strength keys into "name:strength" format,
removing 6 strength keys to free output slots for mode. Migration handles
legacy <lora:> wrapper, separate strength keys, and already-merged formats.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Logs with perf_counter timing on: file load/save, DB sync, all
render functions, save & snap (with deepcopy/save/sync breakdown),
graphviz cache hit/miss, node restore, and API endpoints.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adding all 33 DEFAULTS keys to sequences caused the ComfyUI loader
to exceed its 32 output slot limit, triggering tuple index out of
range errors. Only lora strength migration remains on load/sync.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ensures mode and all other DEFAULTS keys are present in DB
sequences without bloating API read responses with unexpected keys.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ensures all sequences have required keys like mode, cfg, etc.
from DEFAULTS via setdefault so they persist on next save.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Force float coercion in both migration paths (load_json and DB read)
and override type detection in get_sequence_keys so lora strength
keys always report as FLOAT regardless of JSON deserialization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Run _migrate_lora_keys on every load_json call so all sequences
get split name/strength keys immediately, not just when the LoRA
expansion is rendered. Next save writes the clean format to disk.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Store lora name and strength as independent JSON keys instead of the
combined <lora:name:strength> format. Legacy format is auto-migrated
on load. Timeline preview updated to show all 6 LoRA slots.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all save_json/load_json/sync_to_db/DB calls off the event loop
with asyncio.to_thread to prevent UI freezes. Cache graphviz SVG by
DOT source hash (bounded LRU of 20). Replace DELETE-all/re-INSERT in
sync_to_db with UPSERT + targeted DELETE. Add DB indexes, COUNT query,
and reduce graph poll interval to 0.5s.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
history_tree.py:
- Cycle protection in generate_graph() parent walk
- KeyError → .get() for malformed node data in commit() and generate_graph()
- UUID collision check with for/else raise in commit() and _migrate_legacy()
- RuntimeError → ValueError for consistent exception handling
tab_timeline_ng.py:
- Re-parent children walks to surviving ancestor for batch deletes
- Branch tip deletion re-points to parent instead of removing branch
- Cycle protection in _walk_branch_nodes and _find_branch_for_node
- Full data.clear() restore instead of merge in _restore_node
- Safe .get('data', {}) in restore and preview
- Reset stale branch selection after node deletion
- json.dumps for safe JS string escaping in graphviz renderer
tab_batch_ng.py:
- NaN/inf rejection in dict_number with math.isfinite()
- _safe_int used in recalc_vace, update_mode_label, frame_to_skip
- Uncaught ValueError from htree.commit() caught with user notification
tab_comfy_ng.py:
- asyncio.get_event_loop() → get_running_loop()
utils.py:
- Atomic writes for save_config and save_snippets
- save_config extra_data can't override explicit last_dir/favorites
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- sync_to_db: use ON CONFLICT for duplicate sequence numbers
- history_tree: html.escape() for Graphviz DOT labels
- tab_timeline_ng: cap history_tree_backup to 10 entries
- tab_batch_ng: add _safe_int() helper for VACE settings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add bounds check on src_batch index in add_from_source and copy_source
- Guard delete callback against stale index after rapid clicks
- Replace __import__('time').time() with time.time() in sync_to_db
- Use deepcopy(DEFAULTS) consistently in utils.py and main.py
- Use JSON.stringify in JS onConfigure fallback path for key storage
- Read state.show_comfy_monitor for checkbox initial value
- Remove unused KEY_BATCH_DATA import from tab_projects_ng
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use asyncio.to_thread for proxy endpoints to avoid blocking event loop
- Add mode to DEFAULTS so it doesn't silently insert 0
- Use JSON serialization for keys in project_dynamic.js (with comma fallback)
- Validate path exists in change_path, friendly error on duplicate rename
- Remove unused exp param from rename closure
- Use deepcopy for DEFAULTS consistently
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
If the original error (e.g., disk full) also prevents ROLLBACK from
executing, catch and suppress the ROLLBACK failure so the original
exception propagates cleanly and the connection isn't left in a
permanently broken state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix delete_proj not persisting cleared current_project to config:
page reload after deleting active project restored deleted name,
silently breaking all DB sync
- Fix sync_to_db crash on non-dict batch_data items: add isinstance
guard matching import_json_file
- Fix output_types ignored in load_dynamic: parse declared types and
use to_int()/to_float() to coerce values, so downstream ComfyUI
nodes receive correct types even when API returns strings
- Fix backward-compat comma-split for types not trimming whitespace:
legacy workflows with "STRING, INT" got types " INT" breaking
ComfyUI connection type-matching
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
HIGH:
- Fix JS TypeError on empty API response: validate keys/types are arrays
before using them; add HTTP status check (resp.ok)
- Fix BEGIN IMMEDIATE conflict: set isolation_level=None (autocommit) on
SQLite connection so explicit transactions work without implicit ones
MEDIUM:
- Fix import_json_file non-atomic: wrap entire operation in BEGIN/COMMIT
with ROLLBACK on error — no more partial imports
- Fix crash on non-dict batch_data items: skip non-dict elements
- Fix comma-in-key corruption: store keys/types as JSON arrays in hidden
widgets instead of comma-delimited strings (backward-compat fallback)
- Fix blocking I/O in API routes: change async def to def so FastAPI
auto-threads the synchronous SQLite calls
LOW:
- Fix missing ?. on app.graph.setDirtyCanvas in refreshDynamicOutputs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix NameError: pass state to _render_vace_settings (tab_batch_ng.py)
- Fix non-atomic sync_to_db: use BEGIN IMMEDIATE transaction with rollback
- Fix create_secondary() missing db/current_project/db_enabled fields
- Fix URL encoding: percent-encode project/file names in API URLs
- Fix import_json_file crash on re-import: upsert instead of insert
- Fix dual DB instances: share single ProjectDB between UI and API routes
- Also fixes top_level metadata never being updated on existing data_files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- db.py: ProjectDB class with SQLite schema (projects, data_files,
sequences, history_trees), WAL mode, CRUD, import, and query helpers
- api_routes.py: REST API endpoints on NiceGUI/FastAPI for ComfyUI
to query project data over the network
- project_loader.py: ComfyUI nodes (ProjectLoaderDynamic, Standard,
VACE, LoRA) that fetch data from NiceGUI REST API via HTTP
- web/project_dynamic.js: Frontend JS for dynamic project loader node
- tab_projects_ng.py: Projects management tab in NiceGUI UI
- state.py: Added db, current_project, db_enabled fields
- main.py: DB init, API route registration, projects tab
- utils.py: sync_to_db() dual-write helper
- tab_batch_ng.py, tab_raw_ng.py, tab_timeline_ng.py: dual-write
sync calls after save_json when project DB is enabled
- __init__.py: Merged project node class mappings
- tests/test_db.py: 30 tests for database layer
- tests/test_project_loader.py: 17 tests for ComfyUI connector nodes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Walks each path component and matches against actual directory entries
when an exact match fails on Linux. Widget resyncs to the corrected
canonical path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Moved end_frame to main settings area (i2v field, not VACE)
- Added vace_length (default 49) with computed output display
showing vace_length + input_a + input_b
- Reverted custom param type coercion (ComfyUI handles conversion)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removed steps, cfg, sampler_name, scheduler, denoise, model_name, and
vae_name that were showing as custom parameters. Added end_frame (0)
and transition (1-2) with VACE Settings widgets. Set default
general_negative prompt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Templates now generate batch_prompt_i2v.json and
batch_prompt_vace_extend.json as batch files. Create New JSON
always creates batch files since single mode is deprecated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- save_json() now writes to a temp file then uses os.replace() for atomic writes
- Replace hardcoded "batch_data", "history_tree", "prompt_history", "sequence_number"
strings with constants (KEY_BATCH_DATA, etc.) across all modules
- Add 29 unit tests for history_tree, utils, and json_loader
- Add type hints to public functions in utils.py, json_loader.py, history_tree.py
- Remove ALLOWED_BASE_DIR restriction that blocked navigating outside app CWD
- Fix path text input not updating on navigation by using session state key
- Add unpin button (❌) for removing pinned folders
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace bare except clauses with specific exceptions (JSONDecodeError, IOError, ValueError, TypeError)
- Add path traversal protection restricting navigation to ALLOWED_BASE_DIR
- Sanitize iframe URLs with scheme validation and html.escape to prevent XSS
- Extract duplicate to_float/to_int to module-level helpers in json_loader.py
- Replace silent modulo wrapping with clamped bounds checking via get_batch_item()
- Remove hardcoded IP 192.168.1.51:5800, default to empty string
- Add try/except around fragile batch history string parsing
- Add JSON schema validation (dict type check) in read_json_data()
- Add Python logging framework, replace print() calls
- Consolidate session state initialization into loop with defaults dict
- Guard streamlit_agraph import with try/except ImportError
- Add backup snapshot before history node deletion
- Add cycle detection in HistoryTree.commit()
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Ethanfel