ComfyUI-LM-Remote/docs/plans/2026-03-15-civitai-stats-shim-design.md

CivitAI Extended Stats Shim — Design

Problem

The LoRA Manager fetches metadata from CivitAI but discards stats (download count, rating, thumbs up). Users want to sort and filter loras by popularity/quality, and see these stats on model cards — like CivitAI's own UI.

Constraints

• No modifications to the original LoRA Manager codebase (avoid merge conflicts on upstream sync)
• Lives inside ComfyUI-LM-Remote, which already has proxy middleware intercepting API responses
• Must handle CivitAI API rate limits gracefully when bulk-fetching stats
• Stats should persist across restarts (SQLite)

Architecture

Hybrid Shim Approach

• Separate SQLite DB (civitai_stats.db) for extended stats — avoids schema conflicts with the manager's own cache
• Proxy enrichment — the existing middleware intercepts /api/lm/loras/list responses from the remote and merges stats into each item before forwarding to the browser
• New API endpoints on LM-Remote for the bulk fetch button and status checks
• Frontend JS injected via LM-Remote's web/comfyui/ directory, patches sort dropdown and card rendering

Data Flow

List request:
  Browser → GET /api/lm/loras/list
    → proxy fetches response from remote
    → enriches each item with stats from local civitai_stats.db
    → returns merged response to browser

Bulk fetch (initial):
  Browser → POST /api/lm-extra/fetch-stats
    → handler reads models from remote list API
    → extracts civitai_model_id + sha256 for each
    → batches CivitAI API calls (/api/v1/models?ids=id1,id2,...)
    → stores stats in civitai_stats.db
    → WebSocket progress updates
    → frontend auto-refreshes grid when done

Piggyback (ongoing):
  When proxy intercepts metadata sync responses that contain stats,
  opportunistically store them in civitai_stats.db

SQLite Schema

File: civitai_stats.db (stored alongside LM-Remote's config)

CREATE TABLE model_stats (
    sha256 TEXT PRIMARY KEY,
    civitai_model_id INTEGER,
    civitai_version_id INTEGER,
    download_count INTEGER,
    rating REAL,
    rating_count INTEGER,
    thumbs_up_count INTEGER,
    fetched_at REAL
);

Keyed by sha256 — the common identifier between the manager's cache and CivitAI. Joins naturally regardless of file paths or renames.

API Endpoints (LM-Remote)

Method	Path	Purpose
POST	/api/lm-extra/fetch-stats	Bulk fetch stats for all models (button)
GET	/api/lm-extra/stats-status	Check if stats DB is populated, staleness info

Proxy Response Enrichment

The middleware intercepts the /api/lm/loras/list response and:

1. Parses the JSON response body
2. Collects all sha256 values from items
3. Batch-queries civitai_stats.db (single SELECT with IN clause)
4. Merges download_count, rating, rating_count, thumbs_up_count into each item
5. Returns the enriched response

Sorting

New Sort Options

Added to the sort dropdown alongside existing options (name, date, size, usage):

Value	Label
downloads:desc	Most downloaded
downloads:asc	Least downloaded
rating:desc	Highest rated
rating:asc	Lowest rated
thumbsup:desc	Most liked
thumbsup:asc	Least liked

Sort Implementation

When the proxy sees these sort values in the list request, sorting is handled locally:

1. Fetch the full (unpaginated) list from remote
2. Join with stats DB
3. Sort locally
4. Apply pagination
5. Return result

Items without stats sort to the end.

Frontend Changes

All in ComfyUI-LM-Remote/web/comfyui/.

Card Badges

Small badges on each model card (styled like the existing base model badge):

• Download icon + count (e.g., "12.3k")
• Star icon + rating (e.g., "4.8")
• Thumbs-up icon + count (e.g., "89")

Numbers use compact formatting (1000 → "1k", 12345 → "12.3k").

Toolbar Button

A "Fetch Stats" button in the toolbar (next to existing Fetch/Download buttons):

• Chart-bar icon
• Triggers POST to /api/lm-extra/fetch-stats
• Shows progress via WebSocket
• Auto-refreshes grid on completion

Sort Dropdown Patch

JS extension adds the new sort options to the existing #sortSelect dropdown on page load.

Bulk Fetch Strategy

CivitAI's /api/v1/models?ids= endpoint accepts multiple model IDs per request.

1. Collect all unique civitai_model_id values from the lora list
2. Batch into groups (e.g., 50 IDs per request)
3. For each batch, fetch model data and extract stats from the matching version
4. Rate-limit: 1-2 requests/second with backoff on 429 responses
5. Broadcast progress via WebSocket (e.g., "Fetching stats: 150/300 models")

Piggyback Strategy

When the existing metadata sync runs (triggered by the manager's refresh/fetch), the proxy can intercept the CivitAI API response if it flows through, and opportunistically extract stats. This covers new models added after the initial bulk fetch.

Files to Create/Modify

File	Action
stats_db.py	NEW — SQLite wrapper for civitai_stats.db
stats_service.py	NEW — bulk fetch logic, CivitAI API calls, rate limiting
proxy.py	MODIFY — intercept list response, enrich with stats; add sort handling
web/comfyui/lm_stats_ui.js	NEW — card badges, toolbar button, sort dropdown patch