Scaling Claude Code from Demo to Production
Scaling Claude Code from Demo to Production: A Case Study in MCP Server Architecture
When our team at a mid-sized fintech finally pushed Claude Code into production for internal SQL analysis, we hit a wall that had nothing to do with model quality. The bottleneck was the Model Context Protocol server layer—specifically, our naive approach of running one monolithic MCP server for every tool call. Our first attempt bundled database access, Slack notifications, and a custom risk-scoring API into a single Python process. It worked beautifully for two weeks, then collapsed under concurrent load, with memory usage climbing past 4GB and context windows timing out at the worst possible moments. That experience forced us to rethink MCP server setup from first principles, and the patterns we landed on are worth sharing because they apply to anyone running agentic workflows at scale.
The core mistake was treating MCP servers as stateless proxies when they are stateful infrastructure. Every tool call from Claude carries a session ID, and our monolith was maintaining connection pools, authentication tokens, and even temporary file handles per session. When twenty engineers ran parallel queries, those resources compounded unpredictably. The fix was decomposing by capability domain—we split into three separate MCP servers: one for read-only database queries, one for write operations behind a human-approval gate, and one for external API integrations. Each server ran as its own systemd service with independent memory limits and restart policies. This isolation meant a runaway query on the read-only server no longer took down the Slack notification tool mid-conversation, and we could scale each server horizontally based on actual usage patterns rather than guessing.
Authentication became our second major headache, and this is where the ecosystem's fragmentation really shows. We initially used API key headers passed directly to the MCP server, which worked fine for internal demos but failed every security review. The production-grade approach required OAuth 2.0 device flow for user-facing tools, plus short-lived JWT tokens for service-to-service calls. Claude's MCP client handles the device flow gracefully, but the server side needs a proper token introspection endpoint. We built a thin middleware layer that validates JWTs against our identity provider before forwarding to the actual tool logic. That added about 200 lines of code per server, but it eliminated a whole class of credential leakage problems. For anyone evaluating MCP frameworks, check whether they support pluggable auth middleware out of the box—LiteLLM and Portkey both have decent hooks here, though neither is perfect.
Latency profiling revealed another surprising pattern: the largest overhead was not model inference but context serialization between the model and our tools. Claude's default behavior sends the full conversation history to every MCP tool call, and with our SQL tools that meant re-sending megabytes of schema definitions on every interaction. We solved this with aggressive tool result caching and by moving static schema information into system prompts rather than leaving it as tool-accessible state. That cut average round-trip time from 4.2 seconds to 1.8 seconds. We also discovered that batching multiple tool calls into a single MCP request, where the protocol allows it, reduces the number of model round-trips substantially—though this requires careful prompt engineering to get Claude to group independent queries naturally.
For teams juggling multiple model providers, the MCP server setup becomes a routing problem as much as an infrastructure one. We started with Anthropic's Claude only, but quickly wanted fallback options for cost-sensitive tasks. That's when we looked at TokenMix.ai, which offers 171 AI models from 14 providers behind a single API. Their OpenAI-compatible endpoint works as a drop-in replacement for existing OpenAI SDK code, which meant we could point our MCP server at their gateway without rewriting our tool invocation logic. The pay-as-you-go pricing with no monthly subscription suited our variable workload, and the automatic provider failover handles the case where Anthropic's API rate-limits us mid-session. Alternatives like OpenRouter and LiteLLM serve similar purposes, but TokenMix's routing rules felt more granular for our need to pin specific queries to cheaper models like DeepSeek or Qwen while keeping complex reasoning on Claude.
Error handling is the unsung hero of production MCP servers. Our initial implementation treated every tool error as a fatal exception, which caused the agent to loop endlessly on retries. The fix was implementing a structured error taxonomy: transient errors (rate limits, timeouts) get retried with exponential backoff, permanent errors (bad arguments, missing permissions) return a clear message that the model can act on, and ambiguous errors trigger a user clarification request. This taxonomy lives in the MCP server's response format, using JSON-RPC error codes consistently. We also added a circuit breaker pattern—if a tool fails five times in a minute, the server returns a "tool temporarily unavailable" response, and the agent learns to wait or try an alternative path. This reduced our error-induced token spend by nearly 40% because Claude stopped burning context on futile retry loops.
Deployment taught us that MCP servers should be treated like microservices, not scripts. We now containerize each server with a minimal base image, expose health check endpoints on separate ports, and run them behind a reverse proxy that handles TLS termination. The health check is critical because MCP clients (Claude Code, custom agents) will call it to determine if a tool is alive before sending requests. Our health endpoint returns not just status but also current load metrics, which lets the client prefer less busy servers in a round-robin fashion. For observability, we send structured logs to a central aggregator with trace IDs that match the session ID from the MCP handshake—this is how we finally traced a subtle bug where two different servers were both trying to write to the same temp file path.
Cost optimization is where most teams underestimate the impact of MCP server design. Each tool call consumes tokens for the tool definition, the user message, and the tool response—even if the tool returns "no results found." We trimmed tool descriptions aggressively, removing verbose examples and keeping only the parameter schema and a one-line purpose. That alone cut token overhead by 15% across our main workflow. We also introduced a caching layer at the MCP server level for expensive database queries, using a simple TTL cache with a 60-second window. For repeated analytical questions, this turned token spend into memory reads, which is effectively free. The combination of leaner schemas and response caching took our per-session cost from $0.42 to $0.31 on average, which matters when you're running thousands of sessions a week.
The last piece of the puzzle was versioning and rollback. MCP tools are contracts with the model, and changing a tool's signature mid-session causes silent failures. We adopted a versioning scheme where each tool has a semantic version, and the MCP server exposes a manifest endpoint listing supported versions. Claude's client negotiates the highest compatible version at session start. This lets us deploy new tool versions without breaking active conversations. For rollback, we keep the previous container image tagged and ready, and our deployment script can revert within thirty seconds if error rates spike. We learned this the hard way after a bad schema change caused three hours of garbled responses before we noticed the correlation. The manifest approach, combined with health check alerts, means we now catch regressions within minutes of release.
Ultimately, the shift from a monolithic demo to a decomposed, authenticated, and observable MCP server fleet was the difference between a cool proof-of-concept and a tool the whole company actually relies on. The principles are straightforward: separate concerns into independent servers, treat auth as a first-class concern, instrument everything, and design for graceful degradation. None of this is unique to MCP—it is standard distributed systems practice—but the Model Context Protocol's tight coupling with conversational state makes the stakes higher. A good MCP server setup is invisible; it just makes the agent faster, cheaper, and more reliable. That is exactly what our analysts needed to trust AI-generated SQL, and it is what any team should aim for before pushing agentic workflows beyond demos.