SxCP Eval Loop

This loop is for tuning the SxCP generator toward stronger Krea2 images. ComfyUI sends a generated prompt and image to Codex, Codex analyzes the result, then sends back exactly one edited prompt for the next A/B test. Confirmed findings become either generator changes or durable prompt rules in krea2-prompt-guide.md.

Channels

sxcp_eval_in: ComfyUI to Codex. Contains the prompt text and image path.
sxcp_eval_out: Codex to ComfyUI. Prompt-only. Do not put analysis here.
sxcp_eval_log: optional analysis/log channel.

Manual Loop

Start the helper after sending a test prompt:

tools/sxcp_eval_loop.sh 3

Every three minutes it prints a structured request asking Codex to:

Pull sxcp_eval_in.
Inspect the image.
Compare it to the prompt and previous edit.
Push one prompt-only edit to sxcp_eval_out.
Classify the finding as prompt-only, prompt-guide rule, or generator fix.
Change generator code/data only when the issue is systemic.
Record the finding and update the Krea2 prompt guide when a rule is confirmed.

Runtime logs are written under .sxcp_eval/ and ignored by git.

Optional Command Hook

If you have a one-shot Codex command you want to run automatically, set:

SXCP_EVAL_CODEX_CMD="codex exec" tools/sxcp_eval_loop.sh 3

The request is sent on stdin. The command also receives:

SXCP_EVAL_IN_CHANNEL
SXCP_EVAL_OUT_CHANNEL
SXCP_EVAL_LOG_CHANNEL
SXCP_EVAL_GUIDE_FILE
SXCP_EVAL_REQUEST_FILE
SXCP_EVAL_CYCLE_DIR
SXCP_EVAL_CYCLE

Evaluation Axes

Identity consistency
Outfit continuity
Pose/action accuracy
Camera compliance
Location coherence
Crop/framing
Prompt noise/repetition
Model confusion tokens
Overall Krea2 image usefulness

Generator Fix Rule

Only edit the generator when the image shows a repeatable, systemic prompt failure. Examples:

Selfie wording overrides orbit camera.
Clothing continuity loses the selected softcore outfit.
POV wording makes the off-camera participant the visual subject.
Location camera layout inserts foreground anchors in the wrong place.

For one-off model drift, send a cleaner prompt to sxcp_eval_out and keep the generator unchanged. For repeated prompt behavior, update the generator and add the rule to docs/krea2-prompt-guide.md.

2.3 KiB Raw Blame History