78 lines
2.3 KiB
Markdown
78 lines
2.3 KiB
Markdown
# SxCP Eval Loop
|
|
|
|
This loop is for tuning the SxCP generator toward stronger Krea2 images.
|
|
ComfyUI sends a generated prompt and image to Codex, Codex analyzes the result,
|
|
then sends back exactly one edited prompt for the next A/B test. Confirmed
|
|
findings become either generator changes or durable prompt rules in
|
|
[`krea2-prompt-guide.md`](krea2-prompt-guide.md).
|
|
|
|
## Channels
|
|
|
|
- `sxcp_eval_in`: ComfyUI to Codex. Contains the prompt text and image path.
|
|
- `sxcp_eval_out`: Codex to ComfyUI. Prompt-only. Do not put analysis here.
|
|
- `sxcp_eval_log`: optional analysis/log channel.
|
|
|
|
## Manual Loop
|
|
|
|
Start the helper after sending a test prompt:
|
|
|
|
```bash
|
|
tools/sxcp_eval_loop.sh 3
|
|
```
|
|
|
|
Every three minutes it prints a structured request asking Codex to:
|
|
|
|
1. Pull `sxcp_eval_in`.
|
|
2. Inspect the image.
|
|
3. Compare it to the prompt and previous edit.
|
|
4. Push one prompt-only edit to `sxcp_eval_out`.
|
|
5. Classify the finding as prompt-only, prompt-guide rule, or generator fix.
|
|
6. Change generator code/data only when the issue is systemic.
|
|
7. Record the finding and update the Krea2 prompt guide when a rule is confirmed.
|
|
|
|
Runtime logs are written under `.sxcp_eval/` and ignored by git.
|
|
|
|
## Optional Command Hook
|
|
|
|
If you have a one-shot Codex command you want to run automatically, set:
|
|
|
|
```bash
|
|
SXCP_EVAL_CODEX_CMD="codex exec" tools/sxcp_eval_loop.sh 3
|
|
```
|
|
|
|
The request is sent on stdin. The command also receives:
|
|
|
|
- `SXCP_EVAL_IN_CHANNEL`
|
|
- `SXCP_EVAL_OUT_CHANNEL`
|
|
- `SXCP_EVAL_LOG_CHANNEL`
|
|
- `SXCP_EVAL_GUIDE_FILE`
|
|
- `SXCP_EVAL_REQUEST_FILE`
|
|
- `SXCP_EVAL_CYCLE_DIR`
|
|
- `SXCP_EVAL_CYCLE`
|
|
|
|
## Evaluation Axes
|
|
|
|
- Identity consistency
|
|
- Outfit continuity
|
|
- Pose/action accuracy
|
|
- Camera compliance
|
|
- Location coherence
|
|
- Crop/framing
|
|
- Prompt noise/repetition
|
|
- Model confusion tokens
|
|
- Overall Krea2 image usefulness
|
|
|
|
## Generator Fix Rule
|
|
|
|
Only edit the generator when the image shows a repeatable, systemic prompt
|
|
failure. Examples:
|
|
|
|
- Selfie wording overrides orbit camera.
|
|
- Clothing continuity loses the selected softcore outfit.
|
|
- POV wording makes the off-camera participant the visual subject.
|
|
- Location camera layout inserts foreground anchors in the wrong place.
|
|
|
|
For one-off model drift, send a cleaner prompt to `sxcp_eval_out` and keep the
|
|
generator unchanged. For repeated prompt behavior, update the generator and add
|
|
the rule to `docs/krea2-prompt-guide.md`.
|