Claude API Integration Patterns for Enterprise Systems

The foundational pattern is a thin internal service — the LLM gateway — through which every Claude request in the organisation flows. Application teams call the gateway; only the gateway holds API keys and talks to Anthropic. What it centralises is exactly the list that otherwise gets reimplemented badly ten times:

• Credential custody, per-team virtual keys, and usage attribution for chargeback.
• Rate-limit management: shared queuing, concurrency control, and retry policy in one place.
• Policy enforcement: PII redaction, allowed models per team, max token budgets.
• Uniform logging, tracing, and cost metrics.

Keep it thin. The gateway shapes and forwards Messages API traffic; it must not accumulate business logic, or it becomes the bottleneck team everyone waits on. A proxy with policy, not a platform with opinions — that discipline is what keeps the pattern healthy at scale.

Most integrations start — and many rightly end — as synchronous calls: a service sends a Messages API request and blocks for the answer. It is the correct shape whenever a user or upstream system is actively waiting and the work fits comfortably inside normal HTTP timeout budgets: classification, extraction, single-document summarisation, tool-use loops of a few turns.

The craft is in the envelope around the call:

• Set explicit timeouts shorter than your caller’s, so you fail first and degrade deliberately.
• Bound max_tokens to the task; unbounded generation is unbounded latency.
• Use prompt caching for the stable prefix — system prompt, tool definitions, shared reference text — which cuts both cost and time-to-first-token on every repeat call.
• Make handlers idempotent where retries are possible, keyed by request ID.

When p99 latency stops fitting the interaction — multi-minute agent runs, large-document analysis — that is the signal to switch shapes, not to raise timeouts again.

Work that no human is actively waiting on belongs off the synchronous path. Two async shapes cover the territory. For near-real-time jobs — enrich a record after save, draft a reply for later review — publish to your message broker, let a worker pool call Claude at a controlled concurrency, and write results back with status tracking. You gain buffering against traffic spikes, natural retry semantics, and priority lanes.

For bulk offline work — nightly classification of the day’s documents, corpus re-summarisation, evaluation sweeps — use the batch API directly: submit thousands of requests, collect results asynchronously at reduced cost, and keep all of that volume from competing with interactive traffic for rate limits.

The design questions are the usual distributed-systems ones: idempotency keys per job, a dead-letter queue for poison messages, and result freshness expectations agreed with consumers. A simple triage heuristic holds up well — seconds means synchronous, minutes means queue, hours means batch.

For chat and copilot experiences, perceived latency is time-to-first-token, and streaming transforms it. The Messages API streams responses as server-sent events; the standard enterprise topology is a relay — your backend consumes the Anthropic stream and re-emits SSE to the browser, keeping keys server-side while preserving the incremental feel.

app.get("/chat", async (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  const stream = client.messages.stream({
    model: MODEL, max_tokens: 1024,
    messages: history(req)
  });
  for await (const delta of stream.textDeltas())
    res.write(`data: ${JSON.stringify(delta)}\n\n`);
  res.end();
});

Plan for the unglamorous parts: heartbeats so proxies do not kill idle connections, client reconnection with resume semantics, buffering of tool-use argument deltas until each block completes, and moderation or redaction applied on the streaming path — filtering after display is filtering too late.

Point-to-point tool wiring is the integration debt of the agent era: every assistant hand-rolls its own connectors to the same CRMs, wikis, and databases. The Model Context Protocol (MCP) replaces that N-by-M mess with a hub shape — system owners publish MCP servers exposing curated tools and data sources; any MCP-capable client, from internal agents built on the Agent SDK to developer tooling, connects to them uniformly.

Treat MCP servers as products with contracts:

• Expose capabilities, not raw APIs — a find_customer_orders tool, not a generic query executor.
• Enforce authentication and authorization at the server, making it the audit choke point for tool access.
• Version deliberately; agents are consumers of your contract and break like any other client.
• Start read-only, add gated write tools as trust and controls mature.

Designing this layer well is a course in itself — literally the spine of our Claude API & MCP training.