# CivitAI Extended Stats Shim — Design

## Problem

The LoRA Manager fetches metadata from CivitAI but discards stats (download count, rating, thumbs up). Users want to sort and filter loras by popularity/quality, and see these stats on model cards — like CivitAI's own UI.

## Constraints

- No modifications to the original LoRA Manager codebase (avoid merge conflicts on upstream sync)
- Lives inside ComfyUI-LM-Remote, which already has proxy middleware intercepting API responses
- Must handle CivitAI API rate limits gracefully when bulk-fetching stats
- Stats should persist across restarts (SQLite)

## Architecture

### Hybrid Shim Approach

- **Separate SQLite DB** (`civitai_stats.db`) for extended stats — avoids schema conflicts with the manager's own cache
- **Proxy enrichment** — the existing middleware intercepts `/api/lm/loras/list` responses from the remote and merges stats into each item before forwarding to the browser
- **New API endpoints** on LM-Remote for the bulk fetch button and status checks
- **Frontend JS** injected via LM-Remote's `web/comfyui/` directory, patches sort dropdown and card rendering

### Data Flow

```
List request:
  Browser → GET /api/lm/loras/list
    → proxy fetches response from remote
    → enriches each item with stats from local civitai_stats.db
    → returns merged response to browser

Bulk fetch (initial):
  Browser → POST /api/lm-extra/fetch-stats
    → handler reads models from remote list API
    → extracts civitai_model_id + sha256 for each
    → batches CivitAI API calls (/api/v1/models?ids=id1,id2,...)
    → stores stats in civitai_stats.db
    → WebSocket progress updates
    → frontend auto-refreshes grid when done

Piggyback (ongoing):
  When proxy intercepts metadata sync responses that contain stats,
  opportunistically store them in civitai_stats.db
```

## SQLite Schema

File: `civitai_stats.db` (stored alongside LM-Remote's config)

```sql
CREATE TABLE model_stats (
    sha256 TEXT PRIMARY KEY,
    civitai_model_id INTEGER,
    civitai_version_id INTEGER,
    download_count INTEGER,
    rating REAL,
    rating_count INTEGER,
    thumbs_up_count INTEGER,
    fetched_at REAL
);
```

Keyed by `sha256` — the common identifier between the manager's cache and CivitAI. Joins naturally regardless of file paths or renames.

## API Endpoints (LM-Remote)

| Method | Path | Purpose |
|--------|------|---------|
| POST | `/api/lm-extra/fetch-stats` | Bulk fetch stats for all models (button) |
| GET | `/api/lm-extra/stats-status` | Check if stats DB is populated, staleness info |

## Proxy Response Enrichment

The middleware intercepts the `/api/lm/loras/list` response and:

1. Parses the JSON response body
2. Collects all `sha256` values from items
3. Batch-queries `civitai_stats.db` (single SELECT with IN clause)
4. Merges `download_count`, `rating`, `rating_count`, `thumbs_up_count` into each item
5. Returns the enriched response

## Sorting

### New Sort Options

Added to the sort dropdown alongside existing options (name, date, size, usage):

| Value | Label |
|-------|-------|
| `downloads:desc` | Most downloaded |
| `downloads:asc` | Least downloaded |
| `rating:desc` | Highest rated |
| `rating:asc` | Lowest rated |
| `thumbsup:desc` | Most liked |
| `thumbsup:asc` | Least liked |

### Sort Implementation

When the proxy sees these sort values in the list request, sorting is handled locally:

1. Fetch the full (unpaginated) list from remote
2. Join with stats DB
3. Sort locally
4. Apply pagination
5. Return result

Items without stats sort to the end.

## Frontend Changes

All in `ComfyUI-LM-Remote/web/comfyui/`.

### Card Badges

Small badges on each model card (styled like the existing base model badge):

- Download icon + count (e.g., "12.3k")
- Star icon + rating (e.g., "4.8")
- Thumbs-up icon + count (e.g., "89")

Numbers use compact formatting (1000 → "1k", 12345 → "12.3k").

### Toolbar Button

A "Fetch Stats" button in the toolbar (next to existing Fetch/Download buttons):

- Chart-bar icon
- Triggers POST to `/api/lm-extra/fetch-stats`
- Shows progress via WebSocket
- Auto-refreshes grid on completion

### Sort Dropdown Patch

JS extension adds the new sort options to the existing `#sortSelect` dropdown on page load.

## Bulk Fetch Strategy

CivitAI's `/api/v1/models?ids=` endpoint accepts multiple model IDs per request.

1. Collect all unique `civitai_model_id` values from the lora list
2. Batch into groups (e.g., 50 IDs per request)
3. For each batch, fetch model data and extract `stats` from the matching version
4. Rate-limit: 1-2 requests/second with backoff on 429 responses
5. Broadcast progress via WebSocket (e.g., "Fetching stats: 150/300 models")

## Piggyback Strategy

When the existing metadata sync runs (triggered by the manager's refresh/fetch), the proxy can intercept the CivitAI API response if it flows through, and opportunistically extract stats. This covers new models added after the initial bulk fetch.

## Files to Create/Modify

| File | Action |
|------|--------|
| `stats_db.py` | NEW — SQLite wrapper for civitai_stats.db |
| `stats_service.py` | NEW — bulk fetch logic, CivitAI API calls, rate limiting |
| `proxy.py` | MODIFY — intercept list response, enrich with stats; add sort handling |
| `web/comfyui/lm_stats_ui.js` | NEW — card badges, toolbar button, sort dropdown patch |