The JSON file was storing full snapshot data in every history node,
causing files to grow to multi-GB. Now when DB is enabled:
- sync_to_db receives full tree (extracts snapshots to DB)
- save_json receives slim tree (no snapshot data)
- JSON file stays small, DB is authoritative for snapshots
When DB is disabled, JSON still stores full snapshots (only store).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add load_json to tab_timeline_ng imports (NameError on disk fallback)
- Wrap save_history_tree in BEGIN/COMMIT transaction (was autocommitting
each statement, risking partial writes on failure)
- Clean up orphaned history_snapshots in sync_to_db when nodes are
removed from the tree
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- _auto_change_note now loads previous snapshot from DB instead of
reading stripped in-memory node (was producing wrong commit messages)
- _render_mass_update now strips snapshots after save (was leaking RAM)
- Only strip snapshots when DB is enabled (preview/restore still works
without DB via in-memory or disk fallback)
- _render_data_preview adds disk fallback matching _restore_node
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
History tree nodes stored full data snapshots in memory (5-50MB each),
accumulating with every save. Now:
- New `history_snapshots` DB table stores node data separately
- `save_history_tree` and `sync_to_db` extract snapshots before saving
- In-memory tree nodes only hold metadata (id, parent, note, timestamp)
- Restore and preview load snapshots from DB on demand
- `save_and_snap` uses json roundtrip instead of deepcopy (1 copy not 2)
- `_src_cache` moved to AppState, cleared on file switch
- `strip_snapshots()` method on HistoryTree for explicit cleanup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
json.dump in a background thread would crash with "dictionary changed
size during iteration" when the UI mutated data concurrently. Now all
save_json and sync_to_db calls receive a json.loads(json.dumps(data))
snapshot, isolating the serialization from live UI mutations.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ORDER BY sequence_number put all subs (>=1000) at the end. Now groups
by parent (seq/1000), main first, then subs in order.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces slow JSON file parsing with fast DB queries for file loading.
Returns the same dict structure as load_json (top-level + batch_data +
history_tree).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- load_file and on_select try db.load_full_data first (~0.01s),
fall back to load_json only when DB has no data
- Fix unawaited coroutine warning for auto-load (asyncio.ensure_future)
- Fix radio on_change to properly await async load_file
- Reuse current data cache when source file matches current file,
eliminating a redundant 1.3s JSON parse during render
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The COMMIT without BEGIN failed in autocommit mode, crashing ProjectDB
init and leaving _shared_db as None ("Database not initialized").
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Runs at DB startup, only updates rows that have stale separate strength
keys. No-op once all data is migrated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Combines separate lora name and strength keys into "name:strength" format,
removing 6 strength keys to free output slots for mode. Migration handles
legacy <lora:> wrapper, separate strength keys, and already-merged formats.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Logs with perf_counter timing on: file load/save, DB sync, all
render functions, save & snap (with deepcopy/save/sync breakdown),
graphviz cache hit/miss, node restore, and API endpoints.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adding all 33 DEFAULTS keys to sequences caused the ComfyUI loader
to exceed its 32 output slot limit, triggering tuple index out of
range errors. Only lora strength migration remains on load/sync.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ensures mode and all other DEFAULTS keys are present in DB
sequences without bloating API read responses with unexpected keys.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- tab_batch_ng.py: async create_batch with to_thread save/sync
- tab_raw_ng.py: async do_save with to_thread, replace deepcopy
with dict comprehension for display data
- main.py: async create_new with to_thread save
- tab_projects_ng.py: replace per-project count_data_files with
single list_projects_with_file_counts JOIN query
- db.py: add list_projects_with_file_counts method
Zero blocking I/O calls remain in UI callbacks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adding all DEFAULTS keys to API responses shifted output slot
indices, breaking existing ComfyUI workflow links (tuple index
out of range). Only lora migration remains on the DB read path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ensures the project loader API always returns all expected keys
like mode, cfg, etc. even if they were missing from stored data.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ensures all sequences have required keys like mode, cfg, etc.
from DEFAULTS via setdefault so they persist on next save.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Force float coercion in both migration paths (load_json and DB read)
and override type detection in get_sequence_keys so lora strength
keys always report as FLOAT regardless of JSON deserialization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents 'parent slot deleted' RuntimeError when the timeline
tab is destroyed while the timer is still firing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Run _migrate_lora_keys on every load_json call so all sequences
get split name/strength keys immediately, not just when the LoRA
expansion is rendered. Next save writes the clean format to disk.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add _migrate_lora_keys to get_sequence() so the /api endpoints
always return separate lora name and strength keys, even for
data stored in the old <lora:name:strength> format.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Store lora name and strength as independent JSON keys instead of the
combined <lora:name:strength> format. Legacy format is auto-migrated
on load. Timeline preview updated to show all 6 LoRA slots.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all save_json/load_json/sync_to_db/DB calls off the event loop
with asyncio.to_thread to prevent UI freezes. Cache graphviz SVG by
DOT source hash (bounded LRU of 20). Replace DELETE-all/re-INSERT in
sync_to_db with UPSERT + targeted DELETE. Add DB indexes, COUNT query,
and reduce graph poll interval to 0.5s.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When current_project in config references a project not found in the
database, show a warning with buttons to either clear the reference
or recreate the project instead of displaying a misleading label.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: save_json + sync_to_db blocked the event loop while
serializing the growing history tree (all snapshots) as JSON,
causing NiceGUI websocket timeout and client reconnect.
- Skip history tree in snapshot deepcopy (copied only to discard)
- Move save_json/sync_to_db to asyncio.to_thread in all callbacks
- Make save_and_snap, commit, sort_by_number, shift_fts,
_add_sequence, apply_mass_update async
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Set reload=False to prevent watchfiles from triggering full page
reloads when JSON/DB files are written during save
- Timeline graph poll timer (200ms) now deactivates itself when its
parent slot is deleted during UI refresh
- Comfy monitor poll timer catches RuntimeError from deleted parent
slots instead of crashing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
reload=True was watching all file changes, causing full page reloads
when save_json/save_config wrote .json and .json.tmp files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
history_tree.py:
- Cycle protection in generate_graph() parent walk
- KeyError → .get() for malformed node data in commit() and generate_graph()
- UUID collision check with for/else raise in commit() and _migrate_legacy()
- RuntimeError → ValueError for consistent exception handling
tab_timeline_ng.py:
- Re-parent children walks to surviving ancestor for batch deletes
- Branch tip deletion re-points to parent instead of removing branch
- Cycle protection in _walk_branch_nodes and _find_branch_for_node
- Full data.clear() restore instead of merge in _restore_node
- Safe .get('data', {}) in restore and preview
- Reset stale branch selection after node deletion
- json.dumps for safe JS string escaping in graphviz renderer
tab_batch_ng.py:
- NaN/inf rejection in dict_number with math.isfinite()
- _safe_int used in recalc_vace, update_mode_label, frame_to_skip
- Uncaught ValueError from htree.commit() caught with user notification
tab_comfy_ng.py:
- asyncio.get_event_loop() → get_running_loop()
utils.py:
- Atomic writes for save_config and save_snippets
- save_config extra_data can't override explicit last_dir/favorites
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- sync_to_db: use ON CONFLICT for duplicate sequence numbers
- history_tree: html.escape() for Graphviz DOT labels
- tab_timeline_ng: cap history_tree_backup to 10 entries
- tab_batch_ng: add _safe_int() helper for VACE settings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add bounds check on src_batch index in add_from_source and copy_source
- Guard delete callback against stale index after rapid clicks
- Replace __import__('time').time() with time.time() in sync_to_db
- Use deepcopy(DEFAULTS) consistently in utils.py and main.py
- Use JSON.stringify in JS onConfigure fallback path for key storage
- Read state.show_comfy_monitor for checkbox initial value
- Remove unused KEY_BATCH_DATA import from tab_projects_ng
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use asyncio.to_thread for proxy endpoints to avoid blocking event loop
- Add mode to DEFAULTS so it doesn't silently insert 0
- Use JSON serialization for keys in project_dynamic.js (with comma fallback)
- Validate path exists in change_path, friendly error on duplicate rename
- Remove unused exp param from rename closure
- Use deepcopy for DEFAULTS consistently
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Compares current batch data against the last snapshot to generate
descriptive notes like "Added seq 3; Changed: prompt, seed".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Changing the refresh value triggers the node to re-fetch keys from
the API, picking up any new or modified fields in the data.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Sequences: add rename button with name shown in expansion header
- Subsequences: cycle through 6 distinct border colors by sub-index
- Projects: add rename and change path buttons with DB methods
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The subsegment-card CSS class was not being applied to subsequence
expansion items. Add the class conditionally and include the teal
accent CSS rules with a 6px left border.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The actual fix was setting slot.label alongside slot.name. Reverted
onConfigure to read from widget values (which work correctly) and
ensured label is set on both new and reused output slots.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
LiteGraph renders slot.label over slot.name — we were updating name
but the display uses label.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Read output names from info.outputs (serialized node data) instead of
hidden widget values, which ComfyUI may not persist across reloads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The separate ComfyUI-JSON-Dynamic extension provides the same node.
Removes json_loader.py, web/json_dynamic.js, and their tests. Only
ProjectLoaderDynamic remains in this extension.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
JSON.stringify format for hidden widget values didn't survive ComfyUI's
serialization round-trip. Switch to comma-separated strings matching
the proven ComfyUI-JSON-Dynamic implementation. Remove properties-based
approach in favor of the simpler, working pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hidden widget values for output_keys/output_types were not reliably
restored by ComfyUI on workflow reload. Store keys/types in
node.properties (always persisted by LiteGraph) as primary storage,
with hidden widgets as fallback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
JSONLoaderDynamic auto-discovers keys at runtime, making the hardcoded
Standard, Batch, and Custom nodes unnecessary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
JSONLoaderDynamic auto-discovers keys at runtime, making the hardcoded
Standard, Batch, and Custom nodes unnecessary.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ethanfel