Build capability-safe failover

Precompute primary and fallback deployments that preserve the hard requirements of an agent workflow.

What you will build

A failover plan generated ahead of time, not during an incident. It contains an ordered deployment list, the requirements each candidate passed, and the price delta of switching away from primary.

1. Define workflow invariants

TypeScript

const requirements = {
  model: "anthropic/claude-haiku-4.5",
  region: ["eu", "global"],
  minimumContextTokens: 128_000,
  capabilities: {
    tool_calling: "supported",
    streaming: "supported",
    structured_output: "conditional",
  },
} as const;

const representativeUsage = {
  inputTokens: 40_000,
  outputTokens: 4_000,
  cachedInputTokens: 24_000,
};

2. Build the eligible candidate set

TypeScript

const page = await allm.deployments.list({
  model: requirements.model,
  lifecycle_status: "available",
  has_pricing: true,
});

const candidates = page.data.filter((deployment) =>
  requirements.region.includes(deployment.region as "eu" | "global") &&
  deployment.context_tokens !== null &&
  deployment.context_tokens >= requirements.minimumContextTokens &&
  deployment.lifecycle.coverage.status !== "stale" &&
  deployment.capabilities.tool_calling?.status === "supported" &&
  deployment.capabilities.streaming?.status === "supported" &&
  ["supported", "conditional"].includes(
    deployment.capabilities.structured_output?.status ?? "unknown"
  )
);

if (candidates.length < 2) throw new Error("FAILOVER_SET_TOO_SMALL");

3. Compare price and enforce provider diversity

TypeScript

const comparison = await allm.deployments.compare({
  deploymentIds: candidates.map(({ id }) => id),
  ...representativeUsage,
  requires: requirements.capabilities,
});

const primary = comparison.data[0];
const fallback = comparison.data.find(
  (item) => item.deployment.provider_id !== primary.deployment.provider_id
);

if (!fallback) throw new Error("NO_PROVIDER_DIVERSE_FALLBACK");

const failoverPlan = {
  generatedAt: new Date().toISOString(),
  primary: primary.deployment.id,
  fallbacks: [fallback.deployment.id],
  usageAssumption: representativeUsage,
};

Avoid correlated fallbacks

Two deployment IDs can still share a cloud, region, upstream lab, or gateway. Provider diversity is a minimum check; add the infrastructure boundaries relevant to your architecture.

4. Execute failover in your gateway

TypeScript

for (const deploymentId of [failoverPlan.primary, ...failoverPlan.fallbacks]) {
  try {
    const response = await gateway.generate({ deploymentId, request });
    await audit.log({ deploymentId, outcome: "success" });
    return response;
  } catch (error) {
    await audit.log({ deploymentId, outcome: "failed", error });
    if (!isRetryableProviderFailure(error)) throw error;
  }
}

throw new Error("ALL_ELIGIBLE_DEPLOYMENTS_FAILED");

Example plan

JSON

{
  "generatedAt": "2026-07-17T08:00:00Z",
  "primary": "dep_anthropic_claude_haiku_4_5_messages_global",
  "fallbacks": ["dep_aws_bedrock_claude_haiku_4_5_converse_eu"],
  "usageAssumption": {
    "inputTokens": 40000,
    "outputTokens": 4000,
    "cachedInputTokens": 24000
  }
}

Production checklist

Rebuild the plan when the catalog release changes.
Run synthetic probes against every fallback before enabling it.
Fail over only on provider failures, not on invalid user requests.
Cap retry count and total latency budget.
Track output-quality and cost changes after failover.