Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Ethanfel
|
6c2afb1cbb | ||
|
Ethanfel
|
0d1415fca4 |
@@ -5,7 +5,7 @@
|
||||
<p align="center">
|
||||
<a href="https://registry.comfy.org/publishers/ethanfel/nodes/comfyui-snapshot-manager"><img src="https://img.shields.io/badge/ComfyUI-Registry-blue?logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCI+PHBhdGggZD0iTTEyIDJMMyA3djEwbDkgNSA5LTVWN2wtOS01eiIgZmlsbD0id2hpdGUiLz48L3N2Zz4=" alt="ComfyUI Registry"/></a>
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="MIT License"/></a>
|
||||
<img src="https://img.shields.io/badge/version-3.0.0-blue" alt="Version"/>
|
||||
<img src="https://img.shields.io/badge/version-3.0.1-blue" alt="Version"/>
|
||||
<img src="https://img.shields.io/badge/ComfyUI-Extension-purple" alt="ComfyUI Extension"/>
|
||||
</p>
|
||||
|
||||
@@ -20,8 +20,9 @@
|
||||
## Features
|
||||
|
||||
- **Auto-capture** — Snapshots are saved automatically as you edit, with configurable debounce
|
||||
- **Custom naming** — Name your snapshots when taking them manually ("Before merge", "Working v2", etc.)
|
||||
- **Search & filter** — Quickly find snapshots by name with the filter bar
|
||||
- **Custom naming** — Name your snapshots when taking them manually ("Before merge", "Working v2", etc.); double-click a snapshot's name in the sidebar to rename it later
|
||||
- **Notes** — Attach a freeform note to any snapshot with the note (pencil) button; notes are searchable from the filter bar
|
||||
- **Search & filter** — Quickly find snapshots by name or note with the filter bar
|
||||
- **Restore or Swap** — Open a snapshot as a new workflow, or replace the current one in-place
|
||||
- **Workflow browser** — Browse and recover snapshots from any workflow, including renamed or deleted ones
|
||||
- **Per-workflow storage** — Each workflow has its own independent snapshot history
|
||||
@@ -35,9 +36,12 @@
|
||||
- **Ctrl+S shortcut** — Press Ctrl+S (or Cmd+S on Mac) to take a manual snapshot alongside ComfyUI's own save
|
||||
- **SVG graph previews** — Hover any snapshot for a tooltip preview of the workflow graph; click the eye button for a full-size modal; diff view now shows side-by-side SVG comparison with color-coded highlights (green = added, red = removed, amber = modified)
|
||||
- **Diff view** — Compare any snapshot against the current workflow (one click) or two snapshots against each other (Shift+click to set base); see added/removed/modified nodes, widget value changes, and rewired connections in a single modal
|
||||
- **Snapshot branching** — Swap to an old snapshot and edit to fork into a new branch; navigate between branches with `< 1/3 >` arrows at fork points in the sidebar and timeline, like ChatGPT conversation branching
|
||||
- **Profile manager** — Save and load named sets of workflows as session profiles (like browser tab groups); profiles track which workflows you visited and restore the latest snapshot for each
|
||||
- **Hide auto-saves** — Toggle button next to the search bar hides auto-save snapshots to reduce clutter while keeping manual, locked, and node-triggered snapshots visible
|
||||
- **Pause auto-capture** — Toggle automatic capture on/off for the session without leaving the panel (Auto: On/Off button in the filter row)
|
||||
- **Export / Import** — Download a workflow's full snapshot history as a JSON file and re-import it on another machine or server
|
||||
- **Storage usage** — The sidebar footer shows total snapshot storage used on the server across all workflows
|
||||
- **Retention by age** — Optionally auto-delete snapshots older than a configurable number of days (off by default), alongside the per-workflow count limit
|
||||
- **Lock/pin snapshots** — Protect important snapshots from auto-pruning and "Clear All" with a single click
|
||||
- **Concurrency-safe** — Lock guard prevents double-click issues during restore
|
||||
- **Server-side storage** — Snapshots persist on the ComfyUI server's filesystem, accessible from any browser
|
||||
@@ -178,32 +182,13 @@ Visually inspect any snapshot without restoring or swapping it.
|
||||
|
||||
The SVG renderer draws nodes with their stored position, size, and colors. Links are rendered as bezier curves colored by type (blue for IMAGE, orange for CLIP, purple for MODEL, etc.). Collapsed nodes appear as thin title-only strips. Thumbnails (hover tooltips) auto-simplify by hiding labels and slot dots for clarity at small sizes.
|
||||
|
||||
### 14. Snapshot Branching
|
||||
|
||||
Branching lets you explore multiple variations of a workflow without losing any history — similar to conversation branching in ChatGPT.
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. Work normally — snapshots chain linearly as you edit
|
||||
2. **Swap** to an older snapshot and start editing — the next auto-capture forks into a new branch from that point
|
||||
3. A **`< 1/2 >`** branch navigator appears at every fork point in the sidebar and the timeline
|
||||
4. Click the arrows to switch between branches — the sidebar and timeline update together
|
||||
|
||||
**Details:**
|
||||
|
||||
- Each snapshot stores a `parentId` linking it to its predecessor, forming a tree
|
||||
- Legacy snapshots (from before branching) are automatically chained by timestamp for backwards compatibility
|
||||
- **Pruning is branch-safe** — ancestors of the current branch tip and fork-point snapshots are never auto-pruned
|
||||
- **Deleting a fork point** re-parents its children to the deleted snapshot's parent (with a confirmation dialog)
|
||||
- Switching workflows clears branch navigation state
|
||||
|
||||
### 15. Hide Auto-saves
|
||||
### 14. Hide Auto-saves
|
||||
|
||||
Click the **Hide Auto** button next to the search bar to hide all auto-save snapshots ("Auto" and "Initial"). The button turns blue and switches to **Show Auto** when active.
|
||||
|
||||
The filter works together with text search — both are applied simultaneously. Branch navigators remain visible regardless of the filter. Manual, locked, node-triggered, and "Current" snapshots are always shown.
|
||||
The filter works together with text search — both are applied simultaneously. Manual, locked, node-triggered, and "Current" snapshots are always shown.
|
||||
|
||||
### 16. Session Profiles
|
||||
### 15. Session Profiles
|
||||
|
||||
Save and load named sets of workflows — like browser tab groups for ComfyUI.
|
||||
|
||||
@@ -251,7 +236,7 @@ All settings are available in **ComfyUI Settings > Snapshot Manager**:
|
||||
2. A **debounce timer** prevents excessive writes
|
||||
3. The workflow is serialized and **hash-checked** against the last capture (per-workflow) to avoid duplicates
|
||||
4. The previous graph state is diffed against the current to **detect the change type** (node add/remove, connection, parameter, move, or mixed) — stored as a `changeType` field on the record
|
||||
5. New snapshots are sent to the **server** and stored as individual JSON files under `data/snapshots/`
|
||||
5. New snapshots are sent to the **server** and stored as individual JSON files under `<user_dir>/snapshot_manager/snapshots/`
|
||||
6. The **sidebar panel** and **timeline bar** fetch snapshots from the server and render them with change-type icons
|
||||
7. **Restore/Swap** loads graph data back into ComfyUI with a lock guard to prevent concurrent operations, and updates the graph cache so the next diff is accurate
|
||||
|
||||
@@ -270,26 +255,25 @@ All settings are available in **ComfyUI Settings > Snapshot Manager**:
|
||||
4. The **timeline** updates: the swapped-to snapshot gets a white ring (active), the auto-saved snapshot gets a green dot (current)
|
||||
5. Clicking the green dot swaps back; editing the graph clears both markers (the next auto-capture supersedes them)
|
||||
|
||||
**Branching:**
|
||||
**Branching** (implemented but disabled in the current release — the code remains for a future enable):
|
||||
|
||||
1. Each snapshot stores a `parentId` pointing to its predecessor
|
||||
2. `buildSnapshotTree()` constructs parent/child maps from all records — legacy snapshots (no `parentId`) are chained by timestamp automatically
|
||||
3. `getDisplayPath()` walks the tree from root to tip, following `activeBranchSelections` at each fork point, producing the linear branch view
|
||||
4. The sidebar and timeline render only the current branch; `< 1/3 >` navigators at fork points switch between branches
|
||||
5. **Pruning protection**: before pruning, `getAncestorIds()` collects all ancestors of the branch tip; these IDs plus fork-point IDs are sent as `protectedIds` to the server
|
||||
4. **Pruning protection**: before pruning, `getAncestorIds()` collects all ancestors of the branch tip; these IDs plus fork-point IDs are sent as `protectedIds` to the server
|
||||
|
||||
**Profiles:**
|
||||
|
||||
1. Session tracking records each visited workflow key with timestamps
|
||||
2. **Save** creates a JSON file at `<extension_dir>/data/profiles/<id>.json` with the workflow list and active workflow
|
||||
2. **Save** creates a JSON file at `<user_dir>/snapshot_manager/profiles/<id>.json` with the workflow list and active workflow
|
||||
3. **Load** fetches the latest snapshot for each workflow in the profile and calls `loadGraphData`
|
||||
|
||||
**Storage:** Snapshots are stored as JSON files on the server at `<extension_dir>/data/snapshots/<workflow_key>/<id>.json`. Profiles are stored at `<extension_dir>/data/profiles/<id>.json`. Both persist across browser sessions, ComfyUI restarts, and are accessible from any browser connecting to the same server.
|
||||
**Storage:** Snapshots are stored as JSON files on the server in ComfyUI's user directory at `<user_dir>/snapshot_manager/snapshots/<workflow_key>/<id>.json`. Profiles are stored at `<user_dir>/snapshot_manager/profiles/<id>.json`. Data from older versions (kept under the extension's own `data/` folder) is migrated here automatically on first load. Both persist across browser sessions, ComfyUI restarts, and are accessible from any browser connecting to the same server.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Where are snapshots stored?**
|
||||
On the server's filesystem under `<extension_dir>/data/snapshots/`. Each workflow gets its own directory, and each snapshot is an individual JSON file. They persist across browser sessions and are accessible from any browser connecting to the same ComfyUI server.
|
||||
On the server's filesystem in ComfyUI's user directory under `snapshot_manager/snapshots/`. Each workflow gets its own directory, and each snapshot is an individual JSON file. They persist across browser sessions and are accessible from any browser connecting to the same ComfyUI server.
|
||||
|
||||
**I'm upgrading from v1.x — what happens to my existing snapshots?**
|
||||
On first load after upgrading, the extension automatically migrates all snapshots from your browser's IndexedDB to the server. Once migration succeeds, the old IndexedDB database is deleted. If migration fails (e.g., server unreachable), your old data is preserved and migration will retry on the next load.
|
||||
@@ -303,12 +287,6 @@ Each workflow has its own snapshot history. Switching workflows cancels any pend
|
||||
**I renamed/deleted a workflow — are my snapshots gone?**
|
||||
No. Snapshots are keyed by the workflow name at capture time. Use the workflow picker to find and restore them under the old name.
|
||||
|
||||
**How does branching work?**
|
||||
When you swap to an old snapshot and then edit, the next capture forks into a new branch. A `< 1/2 >` navigator appears at the fork point — click the arrows to switch branches. The tree structure is computed from `parentId` links on each snapshot. Old snapshots without `parentId` (from before v3.0) are automatically chained by timestamp.
|
||||
|
||||
**Can I delete a fork-point snapshot?**
|
||||
Yes. The extension re-parents its children to the deleted snapshot's parent, preserving the branch structure. A confirmation dialog warns you first.
|
||||
|
||||
**What are profiles?**
|
||||
Profiles save a list of workflows you've visited in a session. Loading a profile restores the latest snapshot for each workflow. They're useful for switching between project contexts — like browser tab groups.
|
||||
|
||||
|
||||
@@ -0,0 +1,258 @@
|
||||
# Non-Destructive Timeline — Research Report & Implementation Plan
|
||||
|
||||
**Date:** 2026-06-29
|
||||
**Goal:** Make the Snapshot Manager timeline behave like Autodesk Fusion 360's parametric
|
||||
timeline — jumping back and forth is easy, fast, and **non-destructive** — while fixing the
|
||||
three reported pain points: (1) weak "what changed" info, (2) spammy autosave, (3) workflow
|
||||
switching spams "now save".
|
||||
|
||||
This document has two parts:
|
||||
- **Part 1 — Research report** (web research, 25 claims survived 3-vote adversarial verification).
|
||||
- **Part 2 — Implementation plan** (each recommendation mapped to concrete changes in `js/snapshot_manager.js`).
|
||||
|
||||
---
|
||||
|
||||
# Part 1 — Research Report
|
||||
|
||||
## Executive summary
|
||||
|
||||
Across Fusion 360 (CAD), Houdini (node graph), Unreal (Blueprint diff), Final Cut Pro (NLE),
|
||||
Figma and Google Docs (history panels), the tools that feel non-destructive share four moves:
|
||||
**(a)** jumping back never deletes later work — it *disables and recomputes* it (Fusion) or keeps
|
||||
the live state untouched while you preview (Final Cut Pro skimmer); **(b)** *preview* is decoupled
|
||||
from *commit* — you can hover/scrub other states and still "return to where you were"; **(c)** a
|
||||
restore/jump records the pre-jump state **once** as a checkpoint (Figma) rather than spamming
|
||||
checkpoints; and **(d)** "what changed" is shown **semantically** (named parameters, fixed
|
||||
color-coded change types — Unreal's red/green/cyan/grey), with trivial moves *de-emphasized* and
|
||||
auto-history *coalesced* by time or by filtering out streaming/cosmetic edits (Figma's 30-min
|
||||
cadence; redux-undo's `filter`/`excludeAction`/`groupBy`; Houdini's explicit warning against
|
||||
over-automatic capture). These map cleanly onto a snapshot-per-state model and onto all three pain
|
||||
points.
|
||||
|
||||
## Findings
|
||||
|
||||
### F1 — Non-destructive "jump back" = disable + recompute, never delete (Fusion 360) — **HIGH**
|
||||
Fusion records every step in creation order and lets you "go back in time" to edit earlier
|
||||
decisions without starting over. Mechanically, editing/rolling to a past feature **rolls the
|
||||
history marker to just before it and disables (not deletes) all downstream nodes**, then
|
||||
**recomputes them forward** when you roll back to the end — "all the features that were after that
|
||||
point are preserved and will reappear." Rolling the marker back "has not deleted the features in
|
||||
front of the marker." This is the load-bearing property: navigating history is a *reversible
|
||||
editing surface*, not a destructive log.
|
||||
*Sources: autodesk.com Fusion blog (timeline-edits, beginners-guide-part-4) — primary; help.autodesk.com Fusion-360-API CustomFeatures_UM — primary; productdesignonline.com; Ace Makerspace. Votes 3-0 (×5), 2-1 (×1).*
|
||||
|
||||
### F2 — Navigation gesture == edit gesture, plus a draggable marker with playback controls (Fusion) — **HIGH**
|
||||
A past step is edited in place by **double-clicking** it, or **right-click → Edit Sketch/Edit
|
||||
Feature**. Pure navigation is a **draggable history-marker slider** ("rolling back the design")
|
||||
with **playback controls** (move-to-beginning, previous step, play, next step, move-to-end) and a
|
||||
right-click **"Roll History Marker Here"** to jump the marker directly to any step. So back/forth
|
||||
is a one-gesture, low-friction interaction distinct from editing.
|
||||
*Sources: autodesk.com Fusion blog — primary; productdesignonline.com (blog); Noble Desktop. Votes 3-0.*
|
||||
|
||||
### F3 — Decouple *preview* from *committed position* — the skimmer pattern (Final Cut Pro) — **HIGH**
|
||||
Final Cut Pro uses **two indicators**: a persistent **playhead** (your committed position, fixed
|
||||
unless you move it) and a transient **skimmer** (a preview cursor that follows the pointer). The
|
||||
skimmer "lets you preview clips… **without affecting the playhead position**," so you can "skim to
|
||||
see what's in other clips but still keep your playhead position." This is the canonical
|
||||
"scrub-to-preview, keep your place / return to where I was" pattern — non-destructive preview
|
||||
decoupled from the committed edit position.
|
||||
*Sources: support.apple.com Final Cut Pro (skimmer, intro-to-playback) — primary; Larry Jordan; Ripple Training. Votes 3-0 (×3), 2-1 (×1).*
|
||||
|
||||
### F4 — Semantic, color-coded "what changed" + difference navigation (Unreal Blueprint diff) — **HIGH**
|
||||
Unreal's Blueprint Diff Tool communicates **change type visually** with a fixed legend:
|
||||
**red = removed, green = added, cyan = changed, grey = moved nodes/comments.** It provides
|
||||
**Next/Previous** buttons to cycle differences one at a time and a **clickable navigation tree** to
|
||||
jump to a specific difference. The lesson: show the *kind* of change semantically (not raw
|
||||
indices), and explicitly give **moves their own subdued category** (grey).
|
||||
*Sources: dev.epicgames.com UE Diff Tool docs (UE 5.7) — primary. Votes 2-1 (color legend), 3-0 (navigation).*
|
||||
|
||||
### F5 — Scope change capture explicitly; warn against over-automatic capture (Houdini takes) — **HIGH**
|
||||
Houdini "takes" are **hierarchically overlaid sets of parameter changes**: any parameter you don't
|
||||
explicitly change is **inherited from the parent**, enabling non-destructive parallel variations
|
||||
that preserve the original. Only **explicitly included** parameters are editable in a take (others
|
||||
appear disabled), and the takes pane **shows which parameters changed** for the selected take.
|
||||
Critically, SideFX **explicitly warns against overusing Auto-take mode** because it "makes it easy
|
||||
to unintentionally include parameters… which… can make diagnosing problems difficult" — a
|
||||
first-party anti-pattern for automatic change capture that mirrors this tool's autosave spam.
|
||||
*Sources: sidefx.com/docs/houdini takes + ref/panes/takes — primary. Votes 3-0.*
|
||||
|
||||
### F6 — Restore/jump is non-destructive *because it checkpoints the pre-jump state once* (Figma) — **HIGH**
|
||||
Restoring a previous version in Figma "is a **non-destructive action**, so you can still access the
|
||||
current version." It works by adding **two autosave checkpoints**: one preserving the
|
||||
current/pre-restore state, one marking the restored version. So later work is never silently lost —
|
||||
it's captured as a checkpoint exactly once at the moment of restore.
|
||||
*Sources: help.figma.com version-history — primary. Votes 3-0.*
|
||||
|
||||
### F7 — Coalesce auto-history by time; separate viewing from restoring; named vs auto (Figma, Google Docs) — **HIGH**
|
||||
Figma records an auto **checkpoint every 30 minutes** (time-based coalescing) while keeping the
|
||||
live "current version" continuously up to date — it does **not** snapshot on every edit. Google
|
||||
Docs separates **viewing** an earlier version (read-only; a **"Back"** control returns you to
|
||||
current, no state change) from **restoring** it (explicit "Restore this version"). It also supports
|
||||
**named versions** distinct from auto-saved revisions, with an **"Only show named versions"** filter
|
||||
to cut noise (caps: 40 named/doc, 15/spreadsheet).
|
||||
*Sources: help.figma.com; support.google.com/docs/answer/190843 — primary. Votes 3-0.*
|
||||
|
||||
### F8 — History-noise control patterns: filter, exclude streaming actions, group into chunks (redux-undo) — **HIGH**
|
||||
A `filter` keeps intermediate state changes **out of history** (`state.past`) **without affecting
|
||||
the live state**. For drag-like continuous edits, `excludeAction(['MOVE_CURSOR','UPDATE_OBJECT_POS'])`
|
||||
records **only the final committed state**. And undo/redo in "reasonable chunks" needs deliberate
|
||||
**custom filters + `groupBy`** rather than recording every micro-action. This is the concrete
|
||||
recipe for de-noising autosave: gate out cosmetic/streaming edits, coalesce a burst into one entry.
|
||||
*Sources: redux-undo.js.org/main/faq — primary; GitHub README. Votes 3-0 (×2), 2-1 (×1).*
|
||||
|
||||
## Confidence & caveats
|
||||
- **High confidence** on every finding above (primary vendor docs; mostly unanimous 3-0 votes).
|
||||
- **Vendor-doc framing:** Fusion's "non-destructive" language partly comes from Autodesk marketing
|
||||
blogs, but the *mechanism* (disable + recompute, marker preserves downstream) is independently
|
||||
corroborated by API docs and third parties — solid.
|
||||
- **Edge cases noted but not refuting:** editing a far-upstream Fusion feature can break downstream
|
||||
dependency references (inherent to any dependency graph; the design is still never destroyed);
|
||||
Figma history retention is tier-bounded (30 days on free); redux-undo `filter` is technically a
|
||||
warning-context API, not an endorsement (the *pattern* still holds).
|
||||
- **Coverage gaps:** DAW (Ableton/Logic), DaVinci/Premiere timeline scrubbing, Photoshop history,
|
||||
and Git-GUI detached-HEAD UX were searched but produced no *separately verified* surviving claims
|
||||
beyond what Final Cut Pro / Figma / Google Docs already cover. The FCP skimmer is the strongest
|
||||
scrubbing-ergonomics result.
|
||||
|
||||
## Open questions
|
||||
1. For a **snapshot-per-state** model (vs Fusion's parametric feature graph), should "jump back and
|
||||
edit" create a **branch** automatically, or keep the linear list and rely on non-destructive
|
||||
"Current" checkpoints? (Branching is currently hard-disabled in the build.)
|
||||
2. What's the right **auto-snapshot cadence** — pure event-debounce (today: 3 s), a Figma-style
|
||||
time floor (e.g. ≥1 per N minutes), or both?
|
||||
3. Should **scrub/preview** load the graph at all (expensive), or only show the SVG/thumbnail
|
||||
preview until the user explicitly commits — to keep back/forth instant on large graphs?
|
||||
|
||||
---
|
||||
|
||||
# Part 2 — Implementation Plan
|
||||
|
||||
Each workstream cites concrete code in `js/snapshot_manager.js`. Ordered by impact;
|
||||
**Must-have → Should-have → Nice-to-have**.
|
||||
|
||||
## Implementation status (2026-06-29)
|
||||
**Done** (all four batches landed, 19/19 unit tests on the extracted diff logic pass):
|
||||
- **C1 + C2** — `seedWorkflowBaseline()` + `suppressAutoCapture(SWITCH_GUARD_MS)` on `openWorkflow`; `scheduleCaptureSnapshot` honours the suppression window.
|
||||
- **B1** — `detectChangeType` now classifies move/resize/collapse(+pin) as `"cosmetic"` and never lets a cosmetic flag escalate a real edit; `mode` (mute/bypass) treated as meaningful. `_captureCore` skips `changeType==="cosmetic"` for auto-captures (`skipCosmetic`).
|
||||
- **A1 + A3** — `getLiveWidgetNames()`/`widgetNameFor()` map `widgets_values` indices → names at capture; `computeDetailedDiff`/`computeCaptureMetaDiff` carry names; diff modal shows `seed:`/`text:` (meaningful first) and collapses position/size into one muted "Layout: moved, resized" line; tooltips read `KSampler (seed, cfg)`.
|
||||
- **D1 + D3** — non-destructive jump confirmed (swap re-seeds hash + dedup → repeat steps are storage no-ops); `stepToSnapshot()` + **Alt+◀ / Alt+▶** keyboard step nav with a quiet swap and a `N/total · label` position toast.
|
||||
|
||||
**Deferred (nice-to-have, not yet built):** A2 full Unreal-style color legend per change type; B2 time-based checkpoint floor; B3 Google-Docs "only show manual/named" filter; D2 drag-scrub skimmer mode (hover preview already exists); D4 explicit "return to where I was" affordance.
|
||||
|
||||
## Pain-point ↔ finding map
|
||||
| Pain point | Backed by | Workstreams |
|
||||
|---|---|---|
|
||||
| Weak "what changed" info | F4, F5, F7 | **A** |
|
||||
| Spammy autosave | F5, F7, F8 | **B** |
|
||||
| WF-switch spams "now save" | (codebase bug) F6 | **C** |
|
||||
| Fusion-360 non-destructive feel | F1, F2, F3, F6 | **D** |
|
||||
|
||||
---
|
||||
|
||||
## Workstream A — Semantic "what changed" (MUST) — *F4, F5, F7*
|
||||
|
||||
**Problem (code):** `computeDetailedDiff` (`:565`) compares `widgets_values` positionally and emits
|
||||
`Value[6]: "a cat" → "a dog"` (`:1370`). `getGraphData()` is `app.graph.serialize()` (`:438`), so
|
||||
at capture time the **live** `app.graph._nodes[i].widgets[]` array — each with a `.name` in the
|
||||
same order as `widgets_values` — is available but never used. `detectChangeType` (`:454`) only
|
||||
yields a coarse single bucket.
|
||||
|
||||
**A1. Capture a widget index→name map (MUST).** At capture in `_captureCore` (`:1574`), walk
|
||||
`app.graph._nodes`, build `{nodeId: [widgetName,…]}`, and use it so diffs read `text:`, `seed:`,
|
||||
`cfg:`, `sampler_name:` instead of `Value[i]`. Store a compact, named `captureDiff` (extend
|
||||
`computeCaptureMetaDiff` `:697`). Persisted per snapshot so old snapshots without it degrade
|
||||
gracefully to the current `Value[i]` form.
|
||||
|
||||
**A2. Semantic change classification + Unreal-style legend (SHOULD).** Replace the single
|
||||
`changeType` with a small set the user cares about: `prompt`, `param` (seed/cfg/sampler/steps/…),
|
||||
`model` (checkpoint/LoRA names), `connection`, `node_add`/`node_remove`, and a subordinate
|
||||
`cosmetic` (move/resize/collapse). Reuse `CHANGE_TYPE_ICONS` (`:2755`) with the red=removed /
|
||||
green=added / cyan=changed / grey=cosmetic palette (F4). **Moves/resizes get the grey, de-emphasized
|
||||
treatment** — exactly what you asked for.
|
||||
|
||||
**A3. De-noise the diff modal + tooltip (SHOULD).** In `showDiffModal` (`:1232`) and
|
||||
`formatCaptureDiffLines` (`:720`), put position/size/move rows in a **collapsed "Cosmetic"
|
||||
section** and surface prompt/param/model/connection changes first with their widget names. The
|
||||
hover tooltip headline should read e.g. `~ KSampler (seed, cfg) · + CLIPTextEncode` rather than
|
||||
`~ 2 values`.
|
||||
|
||||
## Workstream B — Tame autosave noise (MUST) — *F5, F7, F8*
|
||||
|
||||
**Problem (code):** every `graphChanged` schedules a capture (`:4236`→`scheduleCaptureSnapshot`
|
||||
`:1671`, 3 s debounce). `_captureCore` only skips `changeType === "move"` (`:1589`), so **resize and
|
||||
collapse/expand fall through as `"unknown"` and ARE saved** as snapshots — pure visual noise. No
|
||||
floor on auto-snapshot frequency.
|
||||
|
||||
**B1. Cosmetic-change gate (MUST).** Generalize `skipMove` → `skipCosmetic`: skip auto-capture when
|
||||
the *only* changes are position/size/collapse/pin (redux-undo `filter`/`excludeAction`; Houdini
|
||||
Auto-take warning F5/F8). Manual snapshots (Ctrl+S `:4297`, Snapshot button `:3901`) and node-trigger
|
||||
captures still save everything. This alone removes most of the spam.
|
||||
|
||||
**B2. Coalesce bursts; optional time floor (SHOULD).** Keep the 3 s event-debounce (already
|
||||
coalesces a typing burst). Add an optional minimum interval between *auto* snapshots and/or a
|
||||
Figma-style time-based fallback checkpoint (F7), configurable via the existing settings
|
||||
(`debounceSeconds` lives at `:4130`).
|
||||
|
||||
**B3. Make auto vs manual legible + filterable (NICE).** Auto-snapshots already carry labels;
|
||||
borrow Google Docs' **"Only show named/manual versions"** filter (F7) in the sidebar so the auto
|
||||
stream can be hidden. The existing search/filter UI (`:2934`+) is the natural home.
|
||||
|
||||
## Workstream C — Fix workflow-switch "now save" spam (MUST) — *codebase bug, F6*
|
||||
|
||||
**Root cause (code):** the `openWorkflow` handler (`:4252`) resets state and seeds
|
||||
`lastCapturedIdMap` for the new tab but **never seeds `lastCapturedHashMap` / `lastGraphData`** for
|
||||
it. ComfyUI's `loadGraphData` for the freshly-opened workflow then fires `graphChanged` →
|
||||
`scheduleCaptureSnapshot` → 3 s later `captureSnapshot("Auto")` runs, finds no seeded hash, **can't
|
||||
dedupe**, and saves a redundant snapshot of a workflow you only just opened.
|
||||
|
||||
**C1. Re-seed hash on switch (MUST).** In the `openWorkflow` `after()` block (`:4255`), after the
|
||||
new graph is live, set `lastCapturedHashMap`/`setLastGraphData` for `newKey` (mirror the setup
|
||||
seeding at `:4309-4322`). The post-load `graphChanged` then dedupes to a no-op.
|
||||
|
||||
**C2. Programmatic-load suppression window (MUST).** Add a short-lived `loadingLock` flag (sibling
|
||||
to `restoreLock` `:35`/`:1170`) set around tab switches and snapshot loads so `scheduleCaptureSnapshot`
|
||||
(`:1671`) ignores the `graphChanged` events that *we* caused. Belt-and-suspenders with C1.
|
||||
|
||||
## Workstream D — Fusion-360 non-destructive navigation (SHOULD) — *F1, F2, F3, F6*
|
||||
|
||||
**Problem (code):** every timeline marker click calls `swapSnapshot(rec)` (`:3955`), which
|
||||
`captureSnapshot("Current")` **before** loading (`:1728`). So *navigating* is what generates "now
|
||||
save" and it feels destructive. There's no prev/next step nav and the current-position indicator is
|
||||
subtle.
|
||||
|
||||
**D1. Non-destructive jump = checkpoint-once-then-load (SHOULD).** Adopt Figma's model (F6): when
|
||||
leaving a dirty live state, capture the pre-jump state **once** (already hash-deduped, so browsing
|
||||
between *saved* states is a no-op), then load the target. Confirm/tighten that repeated back/forth
|
||||
between existing snapshots creates **zero** new "Current" snapshots. Never delete later snapshots
|
||||
(we already don't) — that's the non-destructive guarantee (F1).
|
||||
|
||||
**D2. Preview vs commit — skimmer pattern (SHOULD).** The hover tooltip already previews via
|
||||
SVG/thumbnail (`:3793`). Lean into it: hovering = preview (skimmer), clicking = commit (playhead)
|
||||
(F3). Optionally a "scrub" mode where dragging along the timeline updates only the preview, and you
|
||||
commit on release — keeps back/forth instant on big graphs (open question 3).
|
||||
|
||||
**D3. Prev/Next step + clear current marker (SHOULD).** Add **keyboard step navigation**
|
||||
(`[` / `]` or arrow keys) and Fusion-style playback buttons (begin/prev/next/end) to the timeline
|
||||
(`buildTimeline` `:3869`), cycling snapshots one at a time (Unreal Next/Prev F4; Fusion controls
|
||||
F2). Strengthen the current-position indicator (`marker-current`/`marker-active` `:3940`) so "where
|
||||
am I" is obvious.
|
||||
|
||||
**D4. "Return to where I was" (NICE).** Remember the snapshot you were on before scrubbing and offer
|
||||
a one-click jump back (FCP playhead F3; Houdini go-back-and-forth F5). Lightweight: a single
|
||||
`preScrubSnapshotId` + a "↩ return" affordance.
|
||||
|
||||
---
|
||||
|
||||
## Suggested sequencing
|
||||
1. **C1 + C2** (kill wf-switch spam) — small, isolated, immediate relief.
|
||||
2. **B1** (cosmetic gate) — biggest reduction in autosave noise, low risk.
|
||||
3. **A1 + A3** (named semantic diffs) — the "what changed" payoff you asked for.
|
||||
4. **D1 + D3** (non-destructive jump + step nav) — the Fusion-360 feel.
|
||||
5. **A2, B2/B3, D2, D4** — polish / configurable refinements.
|
||||
|
||||
## Risks
|
||||
- Widget-name mapping (A1) depends on live `_nodes`/`widgets` internals — guard for nodes whose
|
||||
widget count ≠ `widgets_values` length and for headless/serialize-only paths.
|
||||
- The cosmetic gate (B1) must not swallow a *meaningful* change that co-occurs with a move — gate
|
||||
only when changes are **exclusively** cosmetic.
|
||||
- Suppression window (C2) must auto-release even if a load throws (mirror `withRestoreLock` `:1170`).
|
||||
+513
-177
File diff suppressed because it is too large
Load Diff
+94
-27
@@ -4,6 +4,8 @@ HTTP route handlers for snapshot storage.
|
||||
Registers endpoints with PromptServer.instance.routes at import time.
|
||||
"""
|
||||
|
||||
import logging
|
||||
|
||||
from aiohttp import web
|
||||
from server import PromptServer
|
||||
|
||||
@@ -11,10 +13,22 @@ from . import snapshot_storage as storage
|
||||
|
||||
routes = PromptServer.instance.routes
|
||||
|
||||
# Sanity caps to bound disk/memory use from a single request.
|
||||
_MAX_BODY_BYTES = 128 * 1024 * 1024 # 128 MB per request
|
||||
_MAX_MIGRATE_RECORDS = 10000
|
||||
|
||||
|
||||
def _too_large(request):
|
||||
"""True if the request advertises a body larger than the cap."""
|
||||
cl = request.content_length
|
||||
return cl is not None and cl > _MAX_BODY_BYTES
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/save")
|
||||
async def save_snapshot(request):
|
||||
try:
|
||||
if _too_large(request):
|
||||
return web.json_response({"error": "Request too large"}, status=413)
|
||||
data = await request.json()
|
||||
record = data.get("record")
|
||||
if not record or "id" not in record or "workflowKey" not in record:
|
||||
@@ -23,8 +37,9 @@ async def save_snapshot(request):
|
||||
return web.json_response({"ok": True})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/list")
|
||||
@@ -36,8 +51,11 @@ async def list_snapshots(request):
|
||||
return web.json_response({"error": "Missing workflowKey"}, status=400)
|
||||
records = storage.get_all_for_workflow(workflow_key)
|
||||
return web.json_response(records)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/get")
|
||||
@@ -54,8 +72,9 @@ async def get_snapshot(request):
|
||||
return web.json_response(record)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/update-meta")
|
||||
@@ -73,8 +92,9 @@ async def update_snapshot_meta(request):
|
||||
return web.json_response({"ok": True})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/delete")
|
||||
@@ -89,8 +109,9 @@ async def delete_snapshot(request):
|
||||
return web.json_response({"ok": True})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/delete-all")
|
||||
@@ -102,8 +123,11 @@ async def delete_all_snapshots(request):
|
||||
return web.json_response({"error": "Missing workflowKey"}, status=400)
|
||||
result = storage.delete_all_for_workflow(workflow_key)
|
||||
return web.json_response(result)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.get("/snapshot-manager/workflows")
|
||||
@@ -111,8 +135,34 @@ async def list_workflows(request):
|
||||
try:
|
||||
keys = storage.get_all_workflow_keys()
|
||||
return web.json_response(keys)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.get("/snapshot-manager/usage")
|
||||
async def storage_usage(request):
|
||||
try:
|
||||
return web.json_response(storage.get_storage_usage())
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/export")
|
||||
async def export_workflow(request):
|
||||
try:
|
||||
data = await request.json()
|
||||
workflow_key = data.get("workflowKey")
|
||||
if not workflow_key:
|
||||
return web.json_response({"error": "Missing workflowKey"}, status=400)
|
||||
records = storage.get_full_records_for_workflow(workflow_key)
|
||||
return web.json_response({"version": 1, "workflowKey": workflow_key, "records": records})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/prune")
|
||||
@@ -123,21 +173,33 @@ async def prune_snapshots(request):
|
||||
max_snapshots = data.get("maxSnapshots")
|
||||
source = data.get("source")
|
||||
protected_ids = data.get("protectedIds")
|
||||
max_age_days = data.get("maxAgeDays")
|
||||
if not workflow_key or max_snapshots is None:
|
||||
return web.json_response({"error": "Missing workflowKey or maxSnapshots"}, status=400)
|
||||
deleted = storage.prune(workflow_key, int(max_snapshots), source=source, protected_ids=protected_ids)
|
||||
deleted = storage.prune(
|
||||
workflow_key, int(max_snapshots),
|
||||
source=source, protected_ids=protected_ids,
|
||||
max_age_days=int(max_age_days) if max_age_days else None,
|
||||
)
|
||||
return web.json_response({"deleted": deleted})
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/migrate")
|
||||
async def migrate_snapshots(request):
|
||||
try:
|
||||
if _too_large(request):
|
||||
return web.json_response({"error": "Request too large"}, status=413)
|
||||
data = await request.json()
|
||||
records = data.get("records")
|
||||
if not isinstance(records, list):
|
||||
return web.json_response({"error": "Missing records array"}, status=400)
|
||||
if len(records) > _MAX_MIGRATE_RECORDS:
|
||||
return web.json_response({"error": "Too many records"}, status=413)
|
||||
imported = 0
|
||||
for record in records:
|
||||
if "id" in record and "workflowKey" in record:
|
||||
@@ -146,8 +208,9 @@ async def migrate_snapshots(request):
|
||||
return web.json_response({"imported": imported})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
# ─── Profile Endpoints ───────────────────────────────────────────────
|
||||
@@ -163,8 +226,9 @@ async def save_profile(request):
|
||||
return web.json_response({"ok": True})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.get("/snapshot-manager/profile/list")
|
||||
@@ -172,8 +236,9 @@ async def list_profiles(request):
|
||||
try:
|
||||
profiles = storage.profile_get_all()
|
||||
return web.json_response(profiles)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/profile/get")
|
||||
@@ -189,8 +254,9 @@ async def get_profile(request):
|
||||
return web.json_response(profile)
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
|
||||
@routes.post("/snapshot-manager/profile/delete")
|
||||
@@ -204,5 +270,6 @@ async def delete_profile(request):
|
||||
return web.json_response({"ok": True})
|
||||
except ValueError as e:
|
||||
return web.json_response({"error": str(e)}, status=400)
|
||||
except Exception as e:
|
||||
return web.json_response({"error": str(e)}, status=500)
|
||||
except Exception:
|
||||
logging.exception("[Snapshot Manager] request handler error")
|
||||
return web.json_response({"error": "Internal server error"}, status=500)
|
||||
|
||||
+116
-14
@@ -13,6 +13,8 @@ operations. Only get_full_record() reads a file from disk after warm-up.
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.parse
|
||||
|
||||
# ─── Data directory resolution ───────────────────────────────────────
|
||||
@@ -37,8 +39,15 @@ _cache_warmed = set() # workflow keys already loaded from disk
|
||||
|
||||
|
||||
def _extract_meta(record):
|
||||
"""Return a lightweight copy of *record* without graphData."""
|
||||
return {k: v for k, v in record.items() if k != "graphData"}
|
||||
"""Return a lightweight copy of *record* without graphData or thumbnail.
|
||||
|
||||
The (potentially large, base64) thumbnail is replaced by a boolean
|
||||
``hasThumbnail`` flag; clients lazy-load the image via get_full_record.
|
||||
"""
|
||||
meta = {k: v for k, v in record.items() if k not in ("graphData", "thumbnail")}
|
||||
if record.get("thumbnail"):
|
||||
meta["hasThumbnail"] = True
|
||||
return meta
|
||||
|
||||
|
||||
def _ensure_cached(workflow_key):
|
||||
@@ -65,8 +74,17 @@ def _ensure_cached(workflow_key):
|
||||
# ─── Helpers ─────────────────────────────────────────────────────────
|
||||
|
||||
def _workflow_dir(workflow_key):
|
||||
if not workflow_key or not isinstance(workflow_key, str):
|
||||
raise ValueError(f"Invalid workflow key: {workflow_key!r}")
|
||||
encoded = urllib.parse.quote(workflow_key, safe="")
|
||||
return os.path.join(_DATA_DIR, encoded)
|
||||
path = os.path.normpath(os.path.join(_DATA_DIR, encoded))
|
||||
# Defense in depth: urllib.parse.quote() leaves "." and ".." unescaped, so a
|
||||
# key like ".." would escape the snapshots root (the "/" in "../.." *is*
|
||||
# escaped, so escapes are bounded to one level — but block it anyway).
|
||||
# Require the resolved directory to be a direct child of _DATA_DIR.
|
||||
if os.path.dirname(path) != os.path.normpath(_DATA_DIR):
|
||||
raise ValueError(f"Invalid workflow key: {workflow_key!r}")
|
||||
return path
|
||||
|
||||
|
||||
def _validate_id(snapshot_id):
|
||||
@@ -74,6 +92,26 @@ def _validate_id(snapshot_id):
|
||||
raise ValueError(f"Invalid snapshot id: {snapshot_id!r}")
|
||||
|
||||
|
||||
def _atomic_write_json(path, obj):
|
||||
"""Write *obj* as JSON to *path* atomically (temp file + os.replace).
|
||||
|
||||
Prevents a crash or concurrent reader mid-write from observing a
|
||||
truncated/corrupt file (the old in-place open("w") truncated first).
|
||||
"""
|
||||
directory = os.path.dirname(path)
|
||||
fd, tmp = tempfile.mkstemp(dir=directory, suffix=".tmp")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
json.dump(obj, f, separators=(",", ":"))
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.remove(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
# ─── Public API ──────────────────────────────────────────────────────
|
||||
|
||||
def put(record):
|
||||
@@ -84,8 +122,7 @@ def put(record):
|
||||
d = _workflow_dir(workflow_key)
|
||||
os.makedirs(d, exist_ok=True)
|
||||
path = os.path.join(d, f"{snapshot_id}.json")
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(record, f, separators=(",", ":"))
|
||||
_atomic_write_json(path, record)
|
||||
|
||||
# Update cache only if already warmed; otherwise _ensure_cached will
|
||||
# pick up the new file from disk on next read.
|
||||
@@ -132,8 +169,7 @@ def update_meta(workflow_key, snapshot_id, fields):
|
||||
record.pop(k, None)
|
||||
else:
|
||||
record[k] = v
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(record, f, separators=(",", ":"))
|
||||
_atomic_write_json(path, record)
|
||||
# Update cache entry
|
||||
for entry in _cache.get(workflow_key, []):
|
||||
if entry.get("id") == snapshot_id:
|
||||
@@ -207,7 +243,10 @@ def get_all_workflow_keys():
|
||||
if not os.path.isdir(subdir):
|
||||
continue
|
||||
workflow_key = urllib.parse.unquote(encoded_name)
|
||||
try:
|
||||
entries = _ensure_cached(workflow_key)
|
||||
except ValueError:
|
||||
continue # skip stray/legacy dirs whose name is not a valid key
|
||||
if not entries:
|
||||
continue
|
||||
results.append({"workflowKey": workflow_key, "count": len(entries)})
|
||||
@@ -215,7 +254,55 @@ def get_all_workflow_keys():
|
||||
return results
|
||||
|
||||
|
||||
def prune(workflow_key, max_snapshots, source=None, protected_ids=None):
|
||||
def get_storage_usage():
|
||||
"""Return {totalBytes, workflows: [{workflowKey, bytes, count}]} for all snapshots."""
|
||||
workflows = []
|
||||
total = 0
|
||||
if os.path.isdir(_DATA_DIR):
|
||||
for encoded_name in os.listdir(_DATA_DIR):
|
||||
subdir = os.path.join(_DATA_DIR, encoded_name)
|
||||
if not os.path.isdir(subdir):
|
||||
continue
|
||||
size = 0
|
||||
count = 0
|
||||
for fname in os.listdir(subdir):
|
||||
if not fname.endswith(".json"):
|
||||
continue
|
||||
try:
|
||||
size += os.path.getsize(os.path.join(subdir, fname))
|
||||
count += 1
|
||||
except OSError:
|
||||
continue
|
||||
if count == 0:
|
||||
continue
|
||||
total += size
|
||||
workflows.append({
|
||||
"workflowKey": urllib.parse.unquote(encoded_name),
|
||||
"bytes": size,
|
||||
"count": count,
|
||||
})
|
||||
workflows.sort(key=lambda w: w["bytes"], reverse=True)
|
||||
return {"totalBytes": total, "workflows": workflows}
|
||||
|
||||
|
||||
def get_full_records_for_workflow(workflow_key):
|
||||
"""Return all full snapshot records (with graphData) for a workflow, for export."""
|
||||
d = _workflow_dir(workflow_key)
|
||||
records = []
|
||||
if os.path.isdir(d):
|
||||
for fname in os.listdir(d):
|
||||
if not fname.endswith(".json"):
|
||||
continue
|
||||
try:
|
||||
with open(os.path.join(d, fname), "r", encoding="utf-8") as f:
|
||||
records.append(json.load(f))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
continue
|
||||
records.sort(key=lambda r: r.get("timestamp", 0))
|
||||
return records
|
||||
|
||||
|
||||
def prune(workflow_key, max_snapshots, source=None, protected_ids=None, max_age_days=None):
|
||||
"""Delete oldest unlocked snapshots beyond limit. Returns count deleted.
|
||||
|
||||
source filtering:
|
||||
@@ -225,6 +312,9 @@ def prune(workflow_key, max_snapshots, source=None, protected_ids=None):
|
||||
|
||||
protected_ids: set/list of snapshot IDs that must not be pruned
|
||||
(e.g. ancestors of active branch tip, fork-point snapshots).
|
||||
|
||||
max_age_days: when > 0, also delete unlocked/unprotected snapshots older
|
||||
than this many days, regardless of the count limit.
|
||||
"""
|
||||
_protected = set(protected_ids) if protected_ids else set()
|
||||
entries = _ensure_cached(workflow_key)
|
||||
@@ -234,9 +324,23 @@ def prune(workflow_key, max_snapshots, source=None, protected_ids=None):
|
||||
candidates = [r for r in entries if not r.get("locked") and r.get("source") != "node" and r.get("id") not in _protected]
|
||||
else:
|
||||
candidates = [r for r in entries if not r.get("locked") and r.get("id") not in _protected]
|
||||
if len(candidates) <= max_snapshots:
|
||||
delete_ids = set()
|
||||
to_delete = []
|
||||
# Oldest-beyond-count get deleted...
|
||||
if len(candidates) > max_snapshots:
|
||||
for rec in candidates[: len(candidates) - max_snapshots]:
|
||||
to_delete.append(rec)
|
||||
delete_ids.add(rec["id"])
|
||||
# ...as do any candidates older than the age cutoff (locked/protected
|
||||
# snapshots were already excluded from candidates above).
|
||||
if max_age_days and max_age_days > 0:
|
||||
cutoff = time.time() * 1000 - max_age_days * 86400000
|
||||
for rec in candidates:
|
||||
if rec["id"] not in delete_ids and rec.get("timestamp", 0) < cutoff:
|
||||
to_delete.append(rec)
|
||||
delete_ids.add(rec["id"])
|
||||
if not to_delete:
|
||||
return 0
|
||||
to_delete = candidates[: len(candidates) - max_snapshots]
|
||||
d = _workflow_dir(workflow_key)
|
||||
deleted = 0
|
||||
delete_ids = set()
|
||||
@@ -304,8 +408,7 @@ def profile_put(profile):
|
||||
_validate_id(pid)
|
||||
_ensure_profiles_dir()
|
||||
path = os.path.join(_PROFILES_DIR, f"{pid}.json")
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(profile, f, separators=(",", ":"))
|
||||
_atomic_write_json(path, profile)
|
||||
_invalidate_profile_cache()
|
||||
|
||||
|
||||
@@ -349,8 +452,7 @@ def profile_update(profile_id, fields):
|
||||
profile.pop(k, None)
|
||||
else:
|
||||
profile[k] = v
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
json.dump(profile, f, separators=(",", ":"))
|
||||
_atomic_write_json(path, profile)
|
||||
_invalidate_profile_cache()
|
||||
return True
|
||||
|
||||
|
||||
Reference in New Issue
Block a user