Layered Inpainting Function Calling Pipeline Script
Build an automated multi-tool execution script that detects image regions, generates masks, and triggers localized inpainting passes.
Use this template when building sequential agent workflows for image editing, object replacement, or background modification. It outputs an orchestration script chaining segmentation and inpainting function calls.
Role: Senior Creative Automation Engineer specializing in image processing and agentic tool orchestration.
Context
- Source Image Reference: {{base_image_source}}
- Segmentation Tool: {{mask_detection_tool}}
- Inpainting Engine API: {{inpainting_engine_api}}
- Sampling Steps: {{sampling_step_count}}
- Seed Management Strategy: {{contextual_seed_policy}}
- Error Fallback Action: {{error_handling_strategy}}
Task
Produce an automated orchestration script that coordinates sequential tool calls between visual segmentation and targeted diffusion inpainting to perform context-aware image modifications.
Method
- Initialize tool interfaces for both {{mask_detection_tool}} and {{inpainting_engine_api}}.
- Ingest {{base_image_source}} and construct the initial tool call payload for visual object segmentation.
- Extract bounding coordinates and alpha masks from the segmentation tool response.
- Apply morphological dilation or feathering parameters to the returned mask boundaries.
- Assemble the inpainting function call incorporating mask data, {{sampling_step_count}}, and {{contextual_seed_policy}}.
- Execute the inpainting tool call and handle pipeline retries according to {{error_handling_strategy}}.
- Return a structured JSON summary linking source, intermediate masks, and the final generated asset URL.
Constraints
- MUST maintain deterministic execution by adhering strictly to {{contextual_seed_policy}}.
- MUST include explicit timeout and retry logic for long-running visual generation endpoints.
- MUST NOT proceed to inpainting if the mask tool returns an empty boundary box.
- Ensure intermediate image buffers are safely released from memory after execution.
Output format
1. Tool Declaration Definitions
Function definitions for segmentation and inpainting interfaces.
2. Sequential Pipeline Script
Runnable execution script containing boundary detection, mask validation, and inpainting dispatch.
3. Pipeline Telemetry Contract
JSON schema representing pipeline run metrics, execution duration, and generated asset metadata.
Self-review
- Check that the transition between segmentation output and inpainting input is fully type-safe.
- Verify error handling covers network disconnects and malformed mask buffers.
- Confirm step count configuration matches {{sampling_step_count}}.
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.