Sign in Subscribe
5 min read cloudflare-workers

Polling Long-Running Agents from a Cloudflare Worker

Share

Polling Long-Running Agents from a Cloudflare Worker

Cloudflare Workers are great at handling short, bursty traffic. They're terrible at waiting. A request handler can run for tens of seconds at most; a cron trigger runs once and exits. Neither is the right tool for "kick off a job that might take five minutes, then come back and read the result."

But that's exactly what you need to do when you dispatch work to an Anthropic Managed Agent — or any other provider that gives you a long-running, asynchronous session.

This post walks through the pattern I landed on after building a small dispatcher that fires a daily reviewer agent and polls a task queue every five minutes: a single Durable Object that owns the job, alarms that drive the polling loop, and a discipline of "never hold a connection." Total surface area: about 670 lines in one TypeScript file. No queues, no external scheduler, no held HTTP requests.

The shape of the problem

Anthropic's Managed Agents API gives you a session-based primitive: you POST /sessions to create one, send it user events, and the agent runs in a sandboxed cloud environment until it returns to idle. Total wall-clock time for a non-trivial coding task is often two to ten minutes.

A naive Worker handler looks like this:

export default {
  async fetch(req, env) {
    const sessionId = await createSession(...);
    await sendEvent(sessionId, prompt);
    while (true) {
      const status = await getSessionStatus(sessionId);
      if (status === 'idle') return ...;
      await sleep(5_000);
    }
  }
}

This will fail. Workers have CPU and wall-clock limits, and even when configured generously, you're paying for a connection that's doing nothing but waiting. The model the platform actually wants you to use is: start the work, exit, come back later.

Durable Objects are how you "come back later" with state intact.

The pattern

A single DO instance owns all in-flight jobs. When a cron fires (or an HTTP request arrives), the Worker proxies into the DO, which:

  1. Creates the agent session
  2. Stores a Job record under job:<id> with the session ID
  3. Adds the job to an active_jobs list
  4. Calls state.storage.setAlarm(Date.now() + 15_000)
  5. Returns immediately

Fifteen seconds later the runtime fires the DO's alarm() handler. It walks the active list, polls each session's status, and either:

  • idle: pulls the events, extracts the agent's text output, marks the job completed, posts the result wherever it's supposed to go (in my case, a notes service), and removes it from the active list.
  • error: marks the job failed and removes it.
  • anything else: leaves it on the active list.

If any jobs remain, the alarm reschedules itself for another 15 seconds. If the list is empty, the DO falls dormant — no compute cost while there's nothing to poll.

export class DispatchManager implements DurableObject {
  async alarm() {
    const activeIds = await this.getActiveJobIds();
    const stillActive: string[] = [];

    for (const jobId of activeIds) {
      const job = await this.state.storage.get<Job>(`job:${jobId}`);
      if (!job || job.status !== 'running') continue;

      const session = await getSessionStatus(this.env.API_KEY, job.sessionId);

      if (session.status === 'idle') {
        const events = await getSessionEvents(this.env.API_KEY, job.sessionId);
        job.result = extractAgentMessages(events);
        job.status = 'completed';
        await this.state.storage.put(`job:${jobId}`, job);
        await this.publishResult(job);
      } else if (session.status === 'error') {
        job.status = 'failed';
        await this.state.storage.put(`job:${jobId}`, job);
      } else {
        stillActive.push(jobId);
      }
    }

    await this.state.storage.put('active_jobs', stillActive);
    if (stillActive.length > 0) {
      await this.state.storage.setAlarm(Date.now() + 15_000);
    }
  }
}

That's the whole loop. The polling cadence (15s) is whatever you want; for agents that finish in minutes, 15-30s is a reasonable default. Tighten it if you care about latency, loosen it if you're rate-limited.

Why a Durable Object and not a queue?

Cloudflare Queues would also work. You'd push a "check this session" message with a delay, the consumer would poll once, and if the session's still running it'd push another delayed message. That's fine, and arguably simpler.

The DO approach wins on three things:

  1. State is already where you need it. The job record, the active list, the dispatch history — all live in the same DO storage that drives the alarm. No separate KV or D1 lookup per poll.
  2. One alarm covers N jobs. The alarm fires once per cycle and walks every active job. With queues you're spending a message per job per poll.
  3. Manual inspection is trivial. A GET /jobs endpoint on the Worker proxies into the DO and lists everything in storage. Same with GET /job/:id for a specific result.

The downside is a single point of contention. If you genuinely need to dispatch hundreds of concurrent jobs, sharding by idFromName(...) across multiple DO instances is straightforward, but for a personal dispatcher one singleton is plenty.

The SSE gotcha

If your dispatched agent posts results back to another Worker via the Model Context Protocol — say, a notes service or a task tracker — you'll likely hit this: most MCP servers respond with text/event-stream, even when the client didn't request streaming. The body looks like:

event: message
data: {"jsonrpc":"2.0","result":{...},"id":1}

A naive await res.json() throws because the body isn't pure JSON. Strip the SSE framing first:

function parseSSEResponse(text: string): string {
  const dataLine = text.split('\n').find(l => l.startsWith('data: '));
  return dataLine ? dataLine.slice(6) : text;
}

const text = await res.text();
const jsonStr = parseSSEResponse(text);
const data = JSON.parse(jsonStr);

This bit me twice during development. Worth a minute of your time if you're chaining Workers through MCP endpoints.

What this gives you

The dispatcher fires on two crons. Once a day it runs an opinionated reviewer agent against a queue of pending changes and posts a verdict. Every five minutes it polls a task table for AI-executable work tagged for it, claims the task with a first-to-update-wins atomic update, and dispatches it.

Both flows share the same DO, the same alarm loop, and the same publish path. Adding a third type of job is a matter of writing a new dispatch/... route on the DO and adding a cron pattern.

The full source for my version is private (it has my agent IDs and a few service-specific hooks), but the shape is what's worth taking. If you need the same pattern: one DO, alarms at a comfortable interval, never hold a connection, strip SSE framing on Worker-to-Worker MCP calls. You can build a remarkable amount on top of that.

When not to do this

If your agent finishes in under 30 seconds, just await it in a fetch handler and ship. The DO+alarm pattern is overhead you take on when the work outlasts the request.

If you need sub-second latency on the result, this isn't your tool — alarms aren't precise schedulers and you'll bake at least one poll cycle of latency into every job. Use a held connection or a webhook.

And if you have many providers dispatching many job types and you're already operating Temporal, Inngest, or similar — use those. This pattern shines specifically when you want a single cheap dispatcher with no infrastructure beyond Cloudflare.

For everything in between — and especially for "I want my Worker to babysit a long-running agent without paying to wait" — a Durable Object with alarms is the smallest thing that works.

Published by: