feat: add inject_mode (suffix/prefix) to TI pipeline

Observation: n4_baseline loss barely moved (1.025→0.965 over 3000 steps),
token_norm grew linearly without plateau — generator likely ignores last-K
CLIP positions (EOS/padding zone) where suffix injects.

Fix: add inject_mode parameter throughout the pipeline:
- "suffix": replace last K positions (original behavior, model may ignore)
- "prefix": replace positions 1:1+K right after BOS — highest attention
  weight in CLIP, much stronger gradient signal expected

Changes:
- selva_textual_inversion_trainer.py: _inject_tokens() helper centralises
  the torch.cat construction for both modes; used in training loop and eval;
  inject_mode stored in checkpoint files
- selva_textual_inversion_loader.py: reads inject_mode from checkpoint,
  includes in TEXTUAL_INVERSION bundle
- selva_sampler.py: uses _inject_tokens() via bundle's inject_mode field
- selva_ti_scheduler.py: inject_mode in _PARAM_DEFAULTS, config, and
  _train_inner call
- ti_sweep_1.json: updated with prefix_inject group (n4, n8, n4+warm);
  n4_baseline marked completed; suffix experiments retained for comparison

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-08 23:31:52 +02:00
parent f96265da23
commit e1a2f0ed7d
5 changed files with 105 additions and 59 deletions
+8 -4
View File
@@ -57,10 +57,14 @@ class SelvaTextualInversionLoader:
print(f"[TI Loader] trained {data['step']} / {data.get('steps', '?')} steps "
f"lr={data.get('lr', '?')}", flush=True)
inject_mode = data.get("inject_mode", "suffix")
print(f"[TI Loader] inject_mode='{inject_mode}'", flush=True)
bundle = {
"embeddings": embeddings, # [K, 1024] float32 on CPU
"n_tokens": n_tokens,
"path": str(p),
"init_text": data.get("init_text", ""),
"embeddings": embeddings, # [K, 1024] float32 on CPU
"n_tokens": n_tokens,
"inject_mode": inject_mode,
"path": str(p),
"init_text": data.get("init_text", ""),
}
return (bundle,)