Multimodal Prompt Enrichment and Safety Dispatch Brief
Define vision-language model tool orchestration to inspect, enrich, and safety-screen generation prompts.
Use this template when setting up an automated pre-generation pipeline where an agent invokes multimodal visual-QA and safety classifiers prior to triggering an image generation backend.
Role: Principal Vision-Language Agent Engineer designing automated pre-generation validation systems.
Context
- Vision-Language evaluation model: {{vlm_engine}}
- Compliance and safety policy: {{safety_taxonomy}}
- Pre-generation enrichment tools: {{prompt_enrichment_tools}}
- Output resolution matrix: {{image_resolution_constraints}}
- Retry and threshold logic: {{retry_policy}}
- Target generation system: {{downstream_generator}}
Task
Author a technical brief defining how an autonomous agent calls {{prompt_enrichment_tools}} and {{vlm_engine}} to inspect, expand, and safety-check user inputs before routing to {{downstream_generator}}.
Method
- Analyze raw user prompts against {{safety_taxonomy}} to identify required validation tool calls.
- Construct the tool execution sequence for semantic expansion using {{prompt_enrichment_tools}}.
- Formulate the function definition for invoking {{vlm_engine}} on reference input images when provided.
- Establish deterministic parameter translation from natural language requests to {{image_resolution_constraints}}.
- Design threshold-based gates that determine whether prompt modification passes or triggers {{retry_policy}}.
- Detail the payload structure dispatched to {{downstream_generator}} upon successful tool execution.
- Document observability hooks to log tool execution latency and token overhead.
Constraints
- MUST enforce strict binary pass/fail criteria for {{safety_taxonomy}} checks.
- MUST NOT send uninspected prompts directly to {{downstream_generator}}.
- Keep the total document length between 500 and 750 words.
- Limit tool definitions strictly to the provided enrichment and inspection toolsets.
- Ensure schema formats match standard JSON tool calling standards.
Output format
1. Tool Calling Sequence
Ordered workflow mapping raw user prompt ingestion to final generation dispatch.
2. Validation & Enrichment Function Signatures
Formal JSON schema definitions for safety screening, visual QA, and style descriptor expansion.
3. Policy Gating & Retry Protocol
Decision matrix specifying conditions under which {{retry_policy}} is activated.
Self-review
- Confirm that all variables ({{vlm_engine}}, {{safety_taxonomy}}, {{prompt_enrichment_tools}}, {{image_resolution_constraints}}, {{retry_policy}}, {{downstream_generator}}) appear correctly in the brief.
- Ensure safety evaluation occurs before parameter formatting.
- Validate that no manual human review steps are introduced into the autonomous loop.
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.