Ethanfel e4dfaac63b Correct 4B 'partial' bias on identical values; harden verdict rule; note model-capability limits ...

The 4B over-uses 'partial' (mislabels identical ref/gen and clear opposites) and
also mis-identifies fine-grained content (e.g. names a position 'doggy'/'cowgirl'
when it is neither). Deterministic fix: force verdict=match when normalized
ref==gen. Prompt hardened to not default to 'partial' (opposites=mismatch). Docs:
the 4B is only reliable for coarse attributes — use the 30B for fine-grained
recognition; prefer grounded geometry axes over named-position labels.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-26 23:43:34 +02:00

..

__init__.py

Initial commit: VLM-as-judge prompt calibration loop

2026-06-26 22:15:56 +02:00

qwen_judge.py

Correct 4B 'partial' bias on identical values; harden verdict rule; note model-capability limits

2026-06-26 23:43:34 +02:00

receptor.py

Initial commit: VLM-as-judge prompt calibration loop

2026-06-26 22:15:56 +02:00