Async Media Generation for AI Agent Pipelines: Why Polling Beats Blocking

The Blocking Problem

Direct image generation is tempting — call `generateImage()`, wait for result, move on. But at scale, this blocks your entire agent pipeline.

Generating 100 blog post images at 7 seconds each = 700 seconds (11+ minutes) of idle time. Your agent is stuck.

The solution: **async jobs with polling or webhooks**.

The Job Queue Pattern

MediaEngine exposes the job pattern via REST API:

# 1. Submit a job
curl -X POST https://api.mcpmediaengine.com/api/jobs \
  -H "Authorization: Bearer me_live_..." \
  -d '{
    "type": "image",
    "model": "nano-banana",
    "inputParams": {
      "prompt": "A watercolor forest",
      "aspect_ratio": "16:9"
    }

# Response: { "id": "job_abc123", "status": "pending" }

# 2. Poll for completion curl https://api.mcpmediaengine.com/api/jobs/job_abc123 \ -H "Authorization: Bearer me_live_..."

# Response: { "id": "job_abc123", "status": "completed", "outputs": [{ "url": "..." }] } ```

Polling Strategy

Poll at increasing intervals to avoid hammering the API:

async function waitForJob(jobId: string, maxWaitMs = 60000) {
  const startTime = Date.now();

while (Date.now() - startTime < maxWaitMs) { const response = await fetch( `https://api.mcpmediaengine.com/api/jobs/${jobId}`, { headers: { 'Authorization': `Bearer ${apiKey}` } } ); const job = await response.json();

if (job.status === 'done') { return job.outputs; }

// Exponential backoff: 500ms → 1s → 2s → 5s await new Promise(r => setTimeout(r, delayMs)); delayMs = Math.min(delayMs * 1.5, 5000); }

throw new Error(`Job ${jobId} timed out`); } ```

Webhook Strategy (Preferred for High Volume)

Instead of polling, register a webhook URL. MediaEngine calls you when jobs complete:

// 1. Submit job with webhook
const submitResponse = await fetch(
  'https://api.mcpmediaengine.com/api/jobs',
  {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${apiKey}` },
    body: JSON.stringify({
      type: 'image',
      model: 'nano-banana',
      inputParams: { prompt: 'A cat', aspect_ratio: '16:9' },
      webhookUrl: 'https://yourapp.com/webhooks/mediaengine'
    })
  }

const job = await submitResponse.json();

// 2. Your webhook endpoint receives: app.post('/webhooks/mediaengine', (req, res) => { const { id, status, outputs } = req.body;

if (status === 'done') { // Process the generated image saveImageUrl(id, outputs[0].url); }

res.json({ received: true }); }); ```

Idempotency Keys

If your agent crashes mid-submission, you'll re-submit the same job. Use idempotency keys to avoid duplicates:

const response = await fetch( 'https://api.mcpmediaengine.com/api/jobs', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Idempotency-Key': idempotencyKey }, body: JSON.stringify({ type: 'image', model: 'nano-banana', inputParams: { prompt: 'A blog hero', aspect_ratio: '16:9' } }) } );

// If you re-submit with the same key, you get the original job back ```

Batch Submission Pattern

For 100s of images, submit all jobs first, then wait for all:

// 1. Submit all jobs
const jobIds = await Promise.all(
  posts.map(post =>
    submitImageJob(post.title, post.summary)
  )

// 2. Wait for all in parallel const images = await Promise.all( jobIds.map(id => waitForJob(id)) );

// 3. Attach to posts and publish posts.forEach((post, i) => { post.heroUrl = images[i][0].url; }); await publishAll(posts); ```

This approach is **13x faster** than sequential waiting (100 jobs × 7s each = 700s sequentially vs ~10s parallel wait).

Handling Failures

Always implement retry logic with backoff:

async function submitJobWithRetry(
  params: JobParams,
  maxRetries = 3
) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await submitJob(params);
    } catch (error) {
      if (attempt === maxRetries) throw error;
      const delayMs = Math.pow(2, attempt) * 1000; // 2s, 4s, 8s
      await new Promise(r => setTimeout(r, delayMs));
    }
  }
}

Cost Implication

Polling costs nothing beyond the job itself. Webhooks cost nothing to call but require a public endpoint. Choose based on your architecture.

Next Steps

• Calculate your [pipeline cost](/blog/how-much-does-ai-image-generation-cost-for-an-automated-content-pipeline)
• Try [img2img transforms](/blog/ai-image-styles-compared-watercolor-cartoon-claymation) for style-aware generation
• Review [our case study](/blog/case-study-our-newsletter-generates-its-own-hero-images) of production-scale pipelines