How we build cross vendor multi-agent research workforce

AgentX workforce uses multiple AI agents to discover topics and plan out the steps to deliver the solution collaboratively. We share the architecting and the lessons we learned from building the system.

Architecting a Multi-LLM Research Engine at AgentX

At AgentX, we’ve built our research platform on a simple belief: the best answers come not from a single perspective, but from the collective intelligence of many. Our system doesn’t just harness a single large language model (LLM); it coordinates multiple agents, often running entirely different LLMs, each with their own strengths, across web, Google Workspace, and custom data integrations to tackle complex research tasks. (Check out our MCP supports and list of built-in tools)

From Concept to Live System: Key Learnings

Taking this multi-agent research platform from idea to production was a revealing process. We gained hands-on insight into distributed system design, user-centric tooling, and prompt design that truly works in the wild. Rather than relying on a single LLM working through a problem step by step, our approach assembles a “team” of AI agents - potentially powered by different LLMs - working in parallel, each assigned a unique slice of the problem. An orchestrator agent plans the research journey, splitting the task among various specialized subagents, and then synthesizes their findings into a cohesive result.

This approach, while powerful, comes with its own set of trade-offs: balancing coordination and independence, ensuring stable operations, and creating transparent, auditable workflows.

Why a Multi-Agent, Multi-LLM Approach Wins

Research is rarely a straight line - new leads appear, hypotheses change, and the target can shift midstream. Hardcoding every step quickly breaks down. Human researchers collaborate, split up work, and adapt as new findings emerge; we wanted our system to reflect that reality.

A single LLM, no matter how advanced, can get trapped by token limits or miss subtleties. By contrast, a fleet of agents - potentially using different LLMs for different subtasks - can branch out, explore ideas independently, and circle back with a broader, richer perspective. Each agent leverages its own context, tools, and methods before everything is combined and cross-checked.

We’ve seen this pay off in real-world benchmarks. For tasks that demand true breadth, like mapping out all board members of the IT S&P 500 - our distributed system outperformed single-LLM solutions, not because it was “smarter,” but because it could break the problem into pieces and assign them to the right agent (and LLM) for the job.

Efficient Use of LLMs (and Their Limits)

Multi-agent systems shine when they can run in parallel, maximizing the use of token budgets and exploiting the strengths of different models. Our internal evaluations showed that clever task decomposition and strategic model selection matter far more than just throwing more compute at a problem. Swapping in newer LLMs for the right subagent can produce a bigger performance leap than simply upping token limits.

Of course, there’s a price: multi-agent orchestration isn’t cheap. For research-heavy queries, token usage can be 10–15× higher than standard chat, so this architecture is best reserved for open-ended, high-value exploration, not simple or highly interdependent tasks.

Behind the Scenes: Orchestrating Multi-LLM Agents

The AgentX research flow is an orchestrator-worker at its core. The orchestrator agent receives the initial user prompt, decides on the best strategy, and deploys subagents - each possibly using different LLMs or toolkits - to tackle separate threads in parallel.

Typical Workflow:

• Query Breakdown: The orchestrator interprets the user’s need and spins up subagents with explicit roles and goals.

• Parallel Search: Subagents independently gather and process data, using the tools or models best suited to their assigned slice.

• Iterative Synthesis: The orchestrator combines the agents’ outputs, optionally requesting follow-up investigations, and ensures the final synthesis addresses the original need.

• Source Attribution: A dedicated citation agent verifies that every claim is traceable, linking back to reliable sources before delivering a finished research report.

Unlike typical retrieval-augmented generation (RAG) pipelines that simply fetch the most “relevant” snippet, our architecture supports ongoing, adaptive research-pivoting as new information is uncovered and assigning the right resources dynamically.

Designing for Multi-Agent Systems: Prompts, Tools, and Evaluation

Prompting and evaluating a group of agents, each possibly running on a different LLM, isn’t just an exercise in prompt tuning. We quickly learned that:

• Step-by-Step Simulation Pays Off: Running through agent reasoning flows helped us spot where instructions caused duplication, waste, or unclear delegation.

• Clear Subtask Boundaries: The orchestrator must carve out sharp, well-scoped mandates for each subagent. Vague roles result in overlap or missed angles.

• Scaling for Complexity: The system learns when to throw its full weight behind a deep dive or keep things lightweight for routine queries.

• Explicit Tool Guidance: Tool descriptions and usage guidelines need to be precise; even small ambiguities can undermine performance.

• Agents Improving Agents: We empower agents to suggest prompt/tool improvements, running continual test cycles for self-optimization.

• Broad, Then Focused: Subagents start by surveying the big picture, then narrow in, mimicking expert human researchers.

• Transparency in Reasoning: Agents “think aloud,” showing their steps and rationale, crucial for both debugging and continuous improvement.

• Parallelism Unlocks Speed: Moving from sequential flows to parallelized agents and tool calls reduced research time on complex problems by over 90%.

Rather than rigid rules, our process favors adaptable, human-like heuristics, always iterating in tight feedback cycles.

Evaluation, too, is a blend: Quick-turn sample queries expose big wins (or failures) early. LLMs grade output for factuality and efficiency, but humans still catch subtle errors and hallucinations, guiding further tweaks.

Behaviors need tooling: Getting the actions right relies on careful prompting and tool design, observability, and feedback loops. See our open-source MCP servers for adding more capabilities. 

Production Engineering: Stability at Scale

Multi-agent orchestration brings fresh engineering hurdles:

• State Management: Agents need to persist context, survive interruptions, and resume mid-task without losing their thread. We built persistent state and checkpointing into the core system.

• Debugging Parallelism: With so many moving parts, robust tracing and pattern-based monitoring help us spot, reproduce, and resolve issues fast.

• Safe Deploys: New code and prompt changes are released via gradual rollout, minimizing impact on active user sessions.

• Synchronous for Now: Our agents currently operate in sync, waiting for all subagent results before moving on, for clarity and easier troubleshooting. As we move toward async operations, we expect even more efficiency gains.

We use Kubernetes for our production and staging deployments to ensure our AI agents are “always live” even during the system upgrade and update.

Looking Ahead: The Future of Multi-Agent Research

Building production-grade, multi-LLM agent systems isn’t just about scaling up prototype code. Every detail - prompt design, system resilience, evaluation, and cross-team collaboration - matters, as even small errors can quickly propagate.

Still, the payoff is huge. AgentX’s platform helps users unravel complex challenges, discover unexpected connections, and reach answers they’d never find alone. Multi-agent, multi-LLM orchestration is already reshaping what’s possible in research, unlocking new capabilities, insights, and speed.

This is just the beginning. With ongoing improvements in model design, agent coordination, and system infrastructure, we’re excited to see what new frontiers the next generation of multi-agent AI will open.

Why AgentX Doesn’t Chase the “Super Agent” Dream

You might wonder: if multi-agent collaboration is so powerful, why not just build a single, all-knowing “Super Agent” to do everything? At AgentX, our answer is simple - we don’t believe in that myth. There’s no such thing as a superhuman who excels at every discipline, and AI is no different. Specialized agents, each focused on what they do best, whether it’s web search, data synthesis, deep research, or fact-checking, consistently outperform a generalist working alone.

Instead, real breakthroughs come from assembling the right “team” of experts, each with unique skills and perspectives. AgentX’s platform puts this philosophy into practice: we make it easy for builders to combine multiple specialized agents, sometimes powered by different LLMs, to match the problem at hand. The art is in how you compose and coordinate these agents, much like putting together a dream team for a complex project.

We don’t chase a single all-purpose agent. We empower users and developers to design their own combinations - because in research, as in life, the right collaboration is what unlocks truly extraordinary results.