Fail CI when model assumptions drift

Turn catalog and lifecycle changes into a reviewable build check before they affect production.

What you will build

A CI check that retrieves every locked deployment, validates its lifecycle and required capabilities, and reports actionable drift. It fails only on conditions your team has declared blocking.

1. Commit the assumptions, not only the ID

allm.lock.json

{
  "version": 1,
  "catalog_release": "cat_2026_07_14_8fa9b2d7c13e",
  "deployments": [
    {
      "id": "dep_anthropic_claude_haiku_4_5_messages_global",
      "required_capabilities": ["tool_calling", "streaming"],
      "minimum_context_tokens": 200000,
      "allowed_lifecycle": ["available"],
      "maximum_input_micro_usd": 1000000,
      "maximum_output_micro_usd": 5000000
    }
  ]
}

2. Re-evaluate the deployment in CI

scripts/check-allm-drift.ts

import lockfile from "../allm.lock.json" with { type: "json" };
import { ALLM } from "@allm/sdk";

const allm = new ALLM({ apiKey: process.env.ALLM_API_KEY });
const failures: string[] = [];

for (const locked of lockfile.deployments) {
  const current = await allm.deployments.retrieve(locked.id);

  if (!locked.allowed_lifecycle.includes(current.lifecycle.status)) {
    failures.push(`${locked.id}: lifecycle is ${current.lifecycle.status}`);
  }
  if (current.context_tokens === null ||
      current.context_tokens < locked.minimum_context_tokens) {
    failures.push(`${locked.id}: context limit no longer satisfies policy`);
  }
  for (const capability of locked.required_capabilities) {
    if (current.capabilities[capability]?.status !== "supported") {
      failures.push(`${locked.id}: ${capability} is not supported`);
    }
  }
  if (!current.rate_card) {
    failures.push(`${locked.id}: rate card is unavailable`);
  } else {
    if (current.rate_card.input_micro_usd > locked.maximum_input_micro_usd)
      failures.push(`${locked.id}: input rate exceeds ceiling`);
    if (current.rate_card.output_micro_usd > locked.maximum_output_micro_usd)
      failures.push(`${locked.id}: output rate exceeds ceiling`);
  }
}

if (failures.length) {
  console.error(failures.map((item) => `- ${item}`).join("\n"));
  process.exitCode = 1;
}

3. Check upcoming end-of-life separately

TypeScript

const horizon = new Date();
horizon.setUTCDate(horizon.getUTCDate() + 90);

const upcoming = await allm.lifecycle.list({
  resource_type: "deployment",
  end_of_life_before: horizon.toISOString(),
  include_unknown: false,
  sort: "end_of_life",
  order: "asc",
});

for (const record of upcoming.data) {
  console.warn(record.resource_id, record.lifecycle.end_of_life.date);
}

Unknown is different from not announced

A missing EOL date can mean the source is not monitored, is stale, or does not publish a date. Gate critical workflows on lifecycle coverage as well as the milestone value.

4. Add the check to CI

GitHub Actions

name: Model contract
on:
  pull_request:
  schedule:
    - cron: "17 6 * * *"

jobs:
  allm-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - run: pnpm install --frozen-lockfile
      - run: pnpm tsx scripts/check-allm-drift.ts
        env:
          ALLM_API_KEY: ${{ secrets.ALLM_API_KEY }}

Example failure

CI output

Model contract failed
- dep_anthropic_claude_haiku_4_5_messages_global: lifecycle is deprecated
- dep_anthropic_claude_haiku_4_5_messages_global: output rate exceeds ceiling

Review the current lifecycle evidence and regenerate allm.lock.json.

Review policy

Fail on retirement, unsupported hard capabilities, missing required limits, and price ceilings.
Warn on a new catalog release when locked resources themselves did not change.
Require human review for conflicting lifecycle evidence.
Update the deployment choice and catalog release in the same pull request.