Engineering MCP at Scale: Reducing Token Consumption by 100x

Enterprise scale is being redefined in the AI era. Chatbots and agents present new challenges, require new tools, demand new standards for observability and cost control, and pose new security risks.

Despite the still-relevant benefits of SDKs and CLIs, we recognize the MCP server as one of the most-important tools in the emerging AI ecosystem. Its appeal among both technical- and business-users has driven nearly every major SaaS product to adopt it.

As users adopt MCP servers to automate their otherwise manual and UI-based tasks, the demands placed upon MCP implementations have skyrocketed. Prototypes that were released after hackathons or proof-of-concept spikes are breaking. The industry is learning how to make and maintain MCP servers that are truly enterprise-grade.

In 2026, we built a customer-facing MCP server for a client. What follows are the learnings from that project, expressed in a general form to maintain anonymity in a way that we hope will add value to your implementation.

But first. What exactly is an MCP server?

Put simply, an MCP server is like an interpreter. It translates the vague language-based requests from agents and chatbots into safe and efficient API calls. It provides useful context and guardrails to make the agent's workflow productive.

Payload over-fetching: the hidden "token tax" of MCP scale

Among other things, a good interpreter doesn't define every single possible word option available when a person asks a single question.

When an MCP server does this, we call it "payload over-fetching." If an LLM asks for the count of customers, a poorly-designed MCP server would reply with every possible operation that could be used to aggregate anything related to users.

The big problem is that it often works at first. Until you reach scale, this is often a silent issue. The agent is able to use the MCP, and isn't going to warn you that it's using a lot of tokens to do it. So it fails silently (the worst kind of bug).

Tool proliferation: the naive "token tax" fix

When teams first discover payload over-fetching, they often reach for tools.

To continue our analogy, a tool would be a single entry in an interpreter's phrasebook. They find the best way to explain something once, and then they re-use that phrase every time it comes up.

The problem arises when you optimize the phrasebook back to the point of it being a dictionary. It no longer serves the purpose at that point.

Since every tool requires a full JSON schema definition injected directly into an LLM's system prompt before execution begins, tool creation becomes a guaranteed way to burn even more tokens.

So what to do?

Solution #1: Dynamic toolsets

To continue our interpreter analogy, a "dynamic toolset" would be like an interpreter doing glossary prep before a technical conference. They prepare and memorize the specialized words and phrases needed ahead of an event. But at the event, they are limited to re-using the exact phrases they memorized ahead of time.

This allows us to quickly look up the right tools for the right context. In technical terms, the tools themselves become a searchable catalog rather than a permanent fixture in the prompt.

We can do this in a few ways:

• Category overview: We can initialize the LLM with high-level descriptions of tool domains rather than granular schemas. This helps rule out categories that are unnecessary in the current context.
• Natural language discovery: We can build a search_tools meta-tool that allows us to find relevant functions.
• On-demand schema loading: This works best when we know which tools are needed for which steps in a process, and the tools are only pulled in for the appropriate steps.
• And many others: This is an entire category of optimization worth exploring.

In general, decoupling tool availability from prompt context size is a great way to both decrease token usage and increase output quality.

Dynamic toolset trade-offs

No solution is perfect. Dynamic toolsets are most effective when:

1. latency is less important than context window size and/or
2. complex multi-step workflows

Tool discovery requires a pre-execution search step, so workloads with lots of single-turn interactions may not benefit as much.

It's possible that for certain workloads where latency is desired, the reduction in overall prompt processing time may offset the pre-execution search step. It's worth testing in your workload.

Solution #2: GraphQL MCP

In our interpreter analogy, the GraphQL MCP would be like hiring an interpreter who is also an expert in the subject matter. They have access to the material on demand, and can use it to flexibly address shifting context in a flexible way.

Instead of a bunch of searchable tools, you wind up with two tools: get_graphql_schema and execute_graphql_query. These allow the agent to find "vocabulary", and deploy it in "sentences".

{ "user": { "id": "123", "name": "Jane", "email": "j@x.com",
    "roles": [...20 items...], "metadata": {...100 lines...} } }

{ "user": { "email": "j@x.com" } }

By requesting using parameters like user { email } instead of receiving a full user object, we've seen context payloads shrink by 70% to 80% on production implementations. It also simplifies architecture down to two universal interfaces.

GraphQL MCP trade-offs

GraphQL MCPs are most effective when:

• access patterns are truly unpredictable and/or
• the workload is read-heavy over data that is truly a graph

Since this method is more flexible and nimble, proper constraints need to be placed to avoid the interpreter who responds to inexperienced askers with full depth, just because their question permitted it.

Putting it all together

A great production implementation will leverage both of these strategies. Here's a simple decision matrix to summarize when to reach for each solution.

Architectural decision matrix