Multi-agent platforms: what to demand before you buy one

You are buying the parts a demo never shows

Every multi-agent demo looks the same. Five agents chat with each other, a plan scrolls past, and something clever appears at the end. None of that tells you whether the system will hold up inside your company. The model already does the clever part, and the model is not what a platform charges you for.

You are buying everything around the model: how work routes between agents, who holds the credentials, and what needs sign-off before it runs. You are also paying for what the team remembers next week. You are paying for whether finished work reaches the tools your company already uses. None of that looks good in a demo. All of it decides whether the platform was the right purchase.

A multi-agent platform runs several specialized AI agents that hand work to each other. Above them sits an orchestration layer, the part that decides who does what. One agent takes the request and breaks it into pieces. It passes each piece to the teammate best suited to it, then puts the results back together. If the idea is new to you, start with what is a multi-agent system and come back here. This post is the buyer's side of the topic.

I run aldena. It is a platform in exactly this category, so I am not a neutral judge. I will flag every place where I use my own product as the example. The tests hold whether or not you ever try it.

A platform carries what a framework hands you

Confusing these two purchases is the fastest way to waste three months.

A multi-agent framework is a code library. LangGraph, CrewAI, AutoGen and the OpenAI Agents SDK (software development kit) all sit at that layer. They give you the building blocks: agent definitions, message passing, and graph or crew abstractions. They are genuinely good at that part.

They also leave everything else to you. You write the orchestration logic, host the runtime, and wire up state storage. You build the permission system, connect every tool your company uses, and build an interface a non-engineer can drive. Then you pay people to maintain all of it. That trade is right when agents are your product and the supporting code is your advantage over competitors. I compare the major options in ai agent frameworks.

A multi-agent orchestration platform ships those layers already running: hosting, orchestration, credential handling, permissions, memory, integrations, and an interface someone outside engineering can drive. Credal's guide to multi-agent platforms makes the enterprise case for this well. Its point is that the next use case should not need a new build or another software subscription. I would state it more strictly. A platform is worth its price only when the second, fifth and twentieth workflow cost configuration instead of engineering.

My version of that test is a headcount estimate. Count the engineers who will still be maintaining agent infrastructure six months after the pilot. On a framework that number is one or two, and it never falls to zero. That is an acceptable answer when you budgeted for those salaries. If your business case promised that the agents pay for themselves, a framework breaks that promise. Those salaries stay on the books.

Orchestration is the first thing I test

Orchestration is the layer that turns one request into coordinated work. It splits the request, assigns the pieces, passes work between agents, and puts the answers back together. It also has to keep the context intact while all of that happens. Every vendor claims it. Few of them let you watch it happen, and watching is what a buyer needs. Four tests.

Can you see the routing? During a live run, one screen should tell you which agent holds the work right now. The same screen should tell you who handed it to them. Reading the logs afterwards does not count. If delegation is invisible, every failure turns into a slow investigation.

Can you change it without code? Team structure should be configuration. Moving a review step or adding a specialist should not need a code change and a deploy. If it does, orchestration stays with your engineers forever and the platform never gets past the pilot.

Are there hard caps? Runaway delegation is the classic multi-agent failure. Agent A asks B, B asks C, C asks A, and the loop bills you until someone notices. Ask the vendor whether hard limits on hand-offs exist at all. Ask whether the platform enforces them, or whether the model is trusted to stop on its own. Ask where those limits are written down. "The model is smart about that" is not an answer, because a limit the model can decide to ignore is not a limit. In aldena the caps are explicit. There is a limit on how many pieces of work an agent hands out in one turn, and a limit on how deep a chain of hand-offs can run. The runtime enforces both, so no prompt raises them. Your platform's limits can differ from mine. What matters is that they exist, that you can find them written down, and that the model cannot change them.

Does context survive the hand-off? When a manager agent passes work down, the agent below needs the real constraints, not a one-line summary of a two-page request. Ask to see what one hand-off actually contains.

I wrote up how I built aldena's orchestration layer in ai agent orchestration. That post covers the failures that forced those caps.

Agents should never hold your keys

Tool calls are where a multi-agent system touches the real world. That makes them the place your security review should spend its time. The pattern to demand has four parts. Credentials live in the platform and never inside an agent's context. Every tool carries explicit limits. Every call is checked against policy before it runs, and every call is logged with its arguments.

The buyer questions that follow from it:

• Where do credentials live? If the answer involves pasting an API key into a prompt, or an agent "knowing" a password, do not buy it. An agent that holds a credential in its context can leak it. An agent that asks the platform to perform the action never sees the credential.
• What is the scope of a connector? A connector is the link between the platform and one outside tool. Connecting your issue tracker for one project should not expose it to every other workspace on the account.
• Is permission set per tool or per integration? "Jira: on" is not a policy. Reading issues and deleting issues deserve different treatment, and that means the platform has to gate each tool on its own.
• What would a prompt injection reach? A prompt injection is a hostile instruction hidden inside content an agent reads. Agents read email, web pages and tickets, so any of those can carry one. A smarter model does not fix this. The fix is scoping. Whatever an agent is tricked into attempting stays inside one project's boundary, and it still has to pass that project's approval policies.

Delivery decides whether the work was real

Plenty of platforms stop at chat. The agents "finish", and the result is a transcript you copy by hand into the systems where work actually counts. That is a chatbot with extra steps, and it limits the value of everything the agents did before it.

Real delivery means the work lands in your systems of record, the systems your company treats as the official version. That means a pull request in GitHub or Bitbucket, tickets in Jira or Linear, a document in Google Drive, or a message in Slack. Delivery into those systems is also what makes governance workable. A pull request can be reviewed, commented on, and undone. A block of code pasted into a chat window cannot.

So for every system your team works in daily, ask two questions. What can the platform put there, and in what form? And what does it refuse to do there until a person says yes? Those refusals tell you how the vendor thinks. A connector that stops at a draft until someone turns sending on is a considered design. A connector that sends mail as you, based on an agent's judgment, is a risk you carry every day. Mine takes the first approach. Aldena's Gmail connector drafts by default. Sending stays off until someone switches it on in that room's connector settings. Even then, every send asks a person first, unless you allow it. Sent mail cannot be recalled, which is why the draft is the safe default.

Guardrails are policy, not prompts

There are two kinds of guardrails and only one of them counts. An instructed guardrail is a line in a system prompt, such as "never delete production data". It fails quietly the first time the model reads something unexpected. Nobody is told. An enforced guardrail is built into the runtime, so no amount of model confusion changes it. Four enforcements to demand.

Approval gates in front of side effects. An approval gate is a stop the run cannot get past without a human answer. Before an agent does anything with real consequences, the run pauses and a person sees the exact action. It continues only after that person approves. The pause has to be durable, so an approval given an hour later resumes the run rather than restarting it.

Role boundaries the runtime enforces. A reviewer that cannot edit code, a manager that cannot merge, a scheduler that cannot execute. If those boundaries are persona text and not real differences in tool access, they will not hold.

Isolation. One project's agents, credentials and memory are sealed off from another's. A mistake stops at that boundary instead of reaching everything the account can see.

Readable state. You can read what an agent remembers, see what it scheduled for itself, and audit what it called. You can then correct any of it. If you cannot inspect the state, you are trusting the vendor rather than governing the system.

Output validation means running a second model as a judge over the first one's work. It appears in most enterprise agent guides, including Credal's, and it does help with quality. But a judge only reads the output after the work is done. The guardrails that keep you out of an incident sit in front of the action.

How the approval gate works in aldena

This is what those four enforcements look like in my own product. Agents in aldena work in rooms. A room is an isolated workspace with its own server, its own connected integrations and credentials, and its own memory. Agents cannot reach anything outside their room, which is the isolation piece.

Inside a room, every tool an agent can reach runs under one of three policies: allow, ask, or deny. The Permissions tab groups tools by where they come from. There are built-ins for files, commands, web access and memory, plus whatever each connected integration adds. You can set a whole group at once and then override single tools. That is how a rule like "reads run free, writes ask first" becomes a setup that takes a couple of minutes.

When a tool is set to ask, the run pauses and shows the exact call the agent wants to make. You answer allow once, always allow, or deny, and the run resumes from the step it paused on. Always allow writes the policy back, so the same decision is never put to you twice. The pause is durable, so answering the next morning is fine. Asking a human a question is itself a tool, and it is the one tool that can never be switched off. An agent that is unsure can always stop and ask instead of guessing.

Two boundaries come set up for you, and I would keep both. The delivery role is written to open the pull request and stop, never to merge its own request. That one is role instruction, so by my own test above it is the weaker kind. The enforced part sits underneath it. Merging is a write tool under the allow, ask, or deny policy, so it stops for you unless you set it to allow. Engineering work ends with a pull request that holds the diff, the test results, and the reasoning. The scrum master presents its whole proposed backlog and waits for your approval before it creates a single ticket in your tracker. Anything the agents scheduled for themselves is listed in room settings with an off switch. Every agent's memory is readable and correctable from the same place. The full write-ups are at human in the loop and isolated rooms.

The pilot checklist I would actually run

Run one real workflow, with a real deliverable someone is waiting for, and score the platform on these:

1. Watch the full run. Could you tell which agent held the work at every moment?
2. Restructure the team in the middle of the pilot. Did routing follow the change without a deploy?
3. Set a destructive tool to require approval, then trigger it. Were you shown the exact call, and did the run resume cleanly after you answered?
4. Deny a tool outright. Confirm the agent cannot reach it at all, rather than being asked nicely not to.
5. Find the output. Is it a pull request, a ticket, or a document in the system of record, or is it only a transcript?
6. Ask where credentials are stored, and what one compromised project could reach.
7. Pull the audit trail for the run. Every tool call should be attributable to one agent.
8. Price the workflow before you run it. You should be able to work out what a run costs from a published price list, not from a "contact sales" quote.
9. Count the engineers this still needs after the pilot ends. That number answers the platform versus framework question with data instead of a slide.

A platform that passes all nine is rare. One that fails the approval-gate test or the delivery test is not a platform yet, whatever the sales deck claims.