There’s a specific number that predicts whether an MSP’s AI deployment will succeed, and it isn’t accuracy.
It’s the percentage of tickets where a technician actually engaged with what the agent produced.
We watch this across every deployment, and the distribution is bimodal in a way that should make every vendor uncomfortable. Some MSPs land at 70, 80, 90% within a few weeks. Others sit at 10% and stay there. The platform is identical. The configuration is comparable. The accuracy scores are within noise of each other.
The difference is entirely in how the deployment was run — and for a long time, when we saw a customer at 10%, our instinct was to look for the product gap. Better recommendations. Better UI. Ship more features and usage will follow.
That instinct was wrong, and it took a couple of near-churns to see it clearly. Low utilization is almost never a capability problem. It’s a management problem wearing a product costume.
The 10% customer
Here’s what it looks like from the inside.
The MSP is live. Integration is clean, the agents are producing good recommendations, the dashboard shows a healthy volume of tickets processed. And roughly one ticket in ten shows any evidence that a human read what the agent suggested.
Ask why, and you don’t get one reason. You get a scattering of small ones. A couple of technicians tried it in week one, hit a recommendation they disagreed with, and stopped. Two others never knew it was there — nobody told them, or somebody told them once in a meeting they half-attended. One senior tech has been vocal that it’s a waste of time, and juniors take their cues from seniors. There’s no lead reviewing whether it’s being used, because nobody was assigned that. And nothing in anyone’s day changes if they ignore it entirely.
None of those are product failures. Every one of them is an unowned deployment.
The uncomfortable part is that this MSP’s renewal conversation will be about product value, and they will be genuinely convinced the tool underdelivered — because from where they sit, it did. The tool was in the building. It just never got introduced to the people who were supposed to use it.
Overload looks like adoption failure and isn’t
The second pattern is quieter, and it hits your engaged users rather than your absent ones.
A supervisor logs in and sees everything: ticket quality scores, dispatch load, satisfaction trends, QA flags, resolution suggestions, anomaly alerts. All of it accurate. All of it arguably useful. And the honest reaction from a smart, invested operator is not enthusiasm — it’s a kind of paralysis. Where am I supposed to look?
Vendors read this as a feature-discovery problem and respond with tours and tooltips. It’s the opposite. The person has discovered the features. There are too many for anyone to act on, so they act on none, and eventually they stop opening the dashboard because opening it feels like a chore with no obvious payoff.
What that user actually wants is to pick one metric — customer satisfaction, say — and look at that and nothing else for a quarter. Saved, persistent, opinionated views: my next three months, showing the single number they’ve decided to move.
A dashboard that shows everything is a dashboard that prioritizes nothing, and prioritization is the entire job. Any platform that expects a service desk manager to assemble their own focus from a buffet of metrics has outsourced the hardest part of the work back to the customer.
Dismissals are the most valuable data you’re throwing away
Here’s the piece almost nobody instruments.
When a technician dismisses a recommendation, that’s usually treated as a null event. The suggestion didn’t apply, it’s gone, move on. But a dismissal carries far more signal than an acceptance does. An accepted suggestion tells you the agent was right. A dismissed one tells you your rules don’t match how this MSP actually works — which is the single most useful thing you can learn in the first month.
The mechanism that works is almost aggressively low-tech. Build a review step into week one: sit down with each user, pull up what they dismissed, and ask why.
The answers cluster fast, and they’re rarely what you’d guess:
- “That’s technically right but we handle that client differently.” Missing client-level context. Fixable in an afternoon.
- “I didn’t understand what it wanted me to do.” Presentation, not logic.
- “We stopped doing it that way six months ago.” Your rules encode a process that no longer exists.
- “That’s not my job, that’s dispatch.” Wrong audience, right recommendation.
Every one of those becomes a rule change. And the second-order effect matters as much as the first: the technician who was interviewed about their dismissals now understands the system is listening, which converts a skeptic into a participant more reliably than any training session.
A related move for the ongoing state — once you’re past week one, a monthly digest of the fifteen things that didn’t work beats real-time alerting. Real-time noise trains people to dismiss reflexively. A monthly list of genuine misses gets read.
Deploy to everyone at once
The received wisdom is to pilot with a small group, prove value, then expand. It’s the safe recommendation and it’s frequently the wrong one.
The strongest deployment I’ve watched recently went the other way: forty technicians, all at once, day one. The reasoning was that everyone works the same ticket categories anyway, so a phased rollout wouldn’t have reduced risk — it would only have reduced the volume of feedback. Forty people surfacing edge cases in week one produces a fine-tuned configuration in three weeks. Five people produce the same result in three months, by which time the enthusiasm that carried the project has evaporated.
Risk was managed a different way, and this is the part people miss: manage risk with human-in-the-loop, not with headcount. Everyone got visibility. Only the two dispatch leads could accept or reject. Everyone else saw the recommendations and could comment on them. Full exposure, controlled blast radius, maximum feedback.
That’s a far better trade than a small pilot with full permissions, which gives you narrow feedback and real risk.
The phased rollout does make sense in one situation: when the team is mid-crisis. An MSP absorbing an acquisition, migrating PSAs, and onboarding new hires simultaneously should not add an AI deployment to the pile. Not because the tool is too much, but because there’s no management attention available to run the deployment properly — and as established, that attention is the whole variable.
Teach them to fish
The last piece is about where the expertise lives after go-live.
The default failure mode is that the customer depends on the vendor for every rule change. Something’s misclassifying, they email support, the vendor adjusts, three days pass. Multiply by every tuning need in the first quarter and both sides are exhausted, the customer feels dependent, and the vendor’s onboarding costs make the account unprofitable.
The alternative is to make agent behavior editable by the customer’s own leads, in plain language, without a support ticket. It’s more work to build and it is the only version that scales — for the MSP and for the vendor. The MSPs that reach high utilization are the ones whose team leads can tune the agent themselves by week three.
That means the real onboarding deliverable isn’t a configured platform. It’s a customer who knows how to reconfigure it.
That’s the bet Mizo made
Mizo’s onboarding assigns a named owner on the MSP side, deploys with human-in-the-loop across the whole team rather than to a pilot subset, and includes a first-week dismissal review — because what your technicians reject in week one is the fastest available map of where the configuration is wrong.
The Service Insights Agent and Performance Agent are built around focus rather than completeness: surface the pattern that matters this quarter, not every metric that could be computed. And agent rules are editable by your own leads in plain language, because a deployment that depends on vendor support for every adjustment has a ceiling built into it.
Where to start on Monday
- Find your utilization number. What percentage of processed tickets show a human interaction with the agent’s output? If you can’t measure it, that’s the first thing to fix.
- Name an owner. One person accountable for adoption, with it written into their objectives. Deployments without a named owner drift to 10% by default.
- Pull the dismissals from the last two weeks and interview three technicians about them. Budget twenty minutes each. Expect to change rules.
- Pick one metric for the quarter and build the view that shows only that. Hide the rest until it’s earned attention.
- Check whether your team leads can change agent behavior without contacting your vendor. If they can’t, that’s your ceiling.
Built by a former MSP operator. The tool your team ignores is more expensive than the tool you never bought.