Ethanfel/ComfyUI-Prompt-Calibrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add audio-aware LTX system prompt + dual-image (subject + waveform) wiring

Browse Source 
prompts/ltx_audio_relay_system_prompt.txt: audio-timed LTX relay prompt — IMAGE 1 =
subject appearance, IMAGE 2 = energy waveform, one beat per audio segment (copy frames),
segment NOTES override the energy default. README documents wiring both images into
chat mode (subject -> reference_image, waveform -> generated_image).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Ethanfel
2026-07-04 22:12:32 +02:00
co-authored by Claude Opus 4.8
parent 56f1a62616
commit d7b9907f56
2 changed files with 66 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+10 -4
View File
@@ -73,15 +73,21 @@ turns an audio clip + your free-text motion **notes** into inputs the vision mod
per-segment energy breakdown with **8n+1-snapped frame counts** and stage hints
(establish→build→peak→settle), plus your notes.
Wire it into the judge node in **chat mode**:
Wire it into the judge node in **chat mode**. To give the model both your **reference
subject image** and the **waveform**, use the two image slots (chat mode sees both):
```
LoadAudio ─► SxCP Audio Prompt Guide ─┬─ waveform_image ─► Judge.reference_image
(your notes: "fast on the drop") └─ audio_summary ─► Judge.user_prompt
(LTX system prompt text node) ─────────────────────────► Judge.system_prompt
your subject image ───────────────────────────────────► Judge.reference_image (IMAGE 1)
LoadAudio ─► SxCP Audio Prompt Guide ─┬─ waveform_image ─► Judge.generated_image (IMAGE 2)
(notes: "seg3: fast on the drop") └─ audio_summary ─► Judge.user_prompt
(prompts/ltx_audio_relay_system_prompt.txt) ─────────► Judge.system_prompt
Judge (mode=chat, json_output=true) ─► LTX beats JSON
```
The audio-aware system prompt is in **`prompts/ltx_audio_relay_system_prompt.txt`** — it
tells the model IMAGE 1 = appearance, IMAGE 2 = energy shape, and to build one beat per
segment (copying its frames), with segment NOTES overriding the energy default.
The model then gets the audio's timing/energy + your motion notes and writes beat durations,
camera moves, and deltas that escalate with the music. `librosa` is optional (BPM/beats);
without it you still get the energy envelope + segments.