Beyond the LEGO: Why MCP Connectors Alone Can’t Build True Knowledge Agents

by Vikram Srinivasan

‍

Preamble – Everyone’s excited about MCP (Model-Context Protocol) because it makes wiring apps together feel like LEGO. But does that mean building a knowledge agent is now trivial? This post builds on my last two posts on precision recall curves and building knowledge agents. Let’s find out.

‍

1 Task agents = CRUD on autopilot

‍

CRUD = Create, Read, Update, Delete — the verbs behind every SaaS button.

Ticket-booking example

1. Read a Slack message: “Book me to NYC, 4–7 Jun.”
2. Create a flight in Sabre.
3. Update a Zendesk ticket with the PNR.
4. Delete the draft if you type “cancel”.

With MCP, each arrow is now a drag-and-drop connector, not a week of REST fiddling. Result: task agents get cheaper and spread fast.

‍

2 Knowledge agents—when plumbing has to read

‍

(From “Why Most Agentic AI Misses the Point”²)

Your CFO asks:

“Across SharePoint, Gmail and Confluence, list every contract over $500 k that expires next quarter and lacks a GDPR clause.”

‍

That question needs business context and unstructured docs (PDFs, PPTs, email). CRUD plumbing alone can’t answer—it’s a knowledge-agent problem.

‍

3 “Why not just hit each system’s search API (now wrapped as an MCP server)?”

‍

The first idea you’ll hear at the whiteboard is always some variation of:

“SharePoint, Gmail, Confluence—and now even our home-grown DMS—each expose a search endpoint.They all support MCP, so the agent can call them the same way.Let’s just fan out a query, merge the results, and call it a day!

‍

It sounds elegant, but several things derail the plan.

‍

4 What good looks like—two pipelines in plain English

‍

4 A Ingestion time

1. Extract & OCR — turns scans, rotated tables, slide images into text.
2. Document understanding & indexing — auto-detects language, dates, currencies, near-dups: “just-enough” enrichment so later queries don’t drown in noise.
3. Hybrid index — a pragmatic blend of the three ingredients above; which parts you keep depends on ops budgets and maintenance tolerance.

‍

4 B Retrieval time

1. Query understanding — parses “next quarter”, normalises “> $500 k”, maps “GDPR” to synonyms.
2. Hybrid search — fans the query across whatever mix the index provides.
3. Re-rank & filter — a small model kicks out footer chatter, boosts the real clause.
4. Answer & citations — “12 contracts totalling $8.3 m risk non-compliance” with click-through sources.

‍

‍

4 C. Search Choices by Each MCP Server

‍

Dropped at indexing time (document understanding)

• No OCR on scans: Any text inside embedded images, tables or slide graphics never makes it into the index—so clauses tucked inside a PDF image vanish.
• Skipped section tagging: Without detecting headers, footers or table-of-contents structure, you can’t target just the “Terms” section—everything’s one big blob.
• No entity extraction: Parties, dates or amounts aren’t recognized, so you can’t later ask “show contracts signed in Q2.”

Dropped at retrieval time (query understanding & reranking)

• Raw keyword match only: Queries like “> $500 k” or “GDPR clause” aren’t normalized or mapped to synonyms—so “500 000 USD” or “privacy addendum” slip through the cracks.
• Bypassed reranker: The small neural model that would demote boilerplate (legal disclaimers, repeated headers) and boost true hits is turned off—so your top results are often generic footers.
• On-the-fly filters: Date or numeric filters run as simple text scans rather than leveraging parsed metadata, adding latency and often missing matches buried mid-document.

For example:

• SharePoint-as-MCP-server may still treat 300-page PDFs as metadata-only to stay fast in the document-library UI.
• Gmail-as-MCP-server may still assume a human can add from: or quotes for precision.
• Confluence-as-MCP-server may still store diagrams as separate attachments; OCR is “nice to have.”
  

These shortcuts shift the precision–recall curve in unpredictable ways. Without consistent enrichment, parsing, and reranking, stitching multiple MCP connectors together hands control of your agent’s quality to whatever corner-cutting connector is weakest.

‍

4 D. Same endpoint shape, wildly different internals—hence the coloured curves never align.

‍

Every MCP connector you plug in brings its own blind spots. First, search is a highly specialized discipline—few teams have the bandwidth or expertise to architect full indexing + retrieval pipelines end-to-end (OCR, section tagging, synonyms, numeric and date parsing, reranking, etc.) at production grade. As a result, every native connector inevitably cuts corners, leaving you with uneven precision and recall.

Second, these out-of-the-box MCP servers know nothing about your business. They don’t speak your industry jargon, apply your internal taxonomies, or encode the regulatory and process nuances that make your enterprise unique. Without that context, even perfectly wired MCP servers will miss the one clause or data point that really matters.

That’s why a truly effective knowledge agent demands:

1. A unified ingestion & retrieval pipeline—standardizing enrichment, indexing, and query parsing so every connector feeds the same high-quality signals.
2. Deep domain & business context integration—injecting your glossary, workflows, and compliance rules so the agent learns what “counts” in your world.

Together, these steps push your precision–recall curve to a new “Unified Frontier” that plain-vanilla MCP integrations can never touch.That’s why our dashed Unified Frontier sits above every individual MCP server curve:

‍

‍

‍

Building a knowledge agent isn’t about how many connectors you can tick off; it’s about whether you can trust the answers when the CFO or regulator is in the room.

• Unstructured side: own the ingestion + hybrid-search stack so you lift the whole precision-recall frontier (dashed line in Figure 1).
  
• One core principle: the farther up-stream you fix search and schema issues, the less downstream prompt-polish you need.
  

Get that retrieval foundation right, and every new MCP connector or agent workflow becomes a value multiplier—not another brittle integration to babysit.

‍

For more insights from Vikram on enterprise AI, market intelligence, and what we’re building at Needl.ai, subscribe to his substack.