We Build AI Into Our Platform. And Yes, We’re Worried Too

An honest take from someone inside the contact center technology space

I’ll be upfront with you: I run a contact center as a service platform. AI is baked into what we sell. So when I say that the industry has a serious problem with how it’s deploying AI in customer service, I’m not throwing stones from the outside. I’m standing right in the middle of the glass house.

There’s a lot of noise right now about AI transforming customer service, like reducing costs, improving response times, and scaling support without scaling headcount.

Some of that is real. AI does those things. I’ve seen it work. Our customers have seen it work. But there’s a quieter story unfolding beneath all those green dashboards, and I think it’s time someone in my position said it plainly: a lot of companies are deploying AI in ways that are slowly and invisibly eroding the trust their customers have in them. And the worst part is they don’t even know it’s happening yet.

The Dashboard Doesn’t See What the Customer Feels

Here’s the thing about deploying AI at scale in customer service. The metrics improve almost immediately. For instance, response times drop, interaction volumes that would have overwhelmed a human team get absorbed effortlessly, and CSAT scores don’t tank right away. So, leadership looks at the numbers and declares victory. But that picture is incomplete in a way that’s genuinely dangerous.

Think about a customer who got wrong information from an AI chatbot about a return policy. They didn’t file a complaint. They just went ahead with the return, got denied, lost money, and quietly decided never to shop with that brand again.

That story doesn’t show up in your dashboard.

Neither does the premium customer who spent twenty minutes going in circles with a virtual agent that kept offering her the same three options, never once escalating to a human. She eventually gave up, called a competitor, and moved her account. Again, no complaint filed, no ticket opened. They are just gone.

This is the slow, silent erosion of brand trust that nobody in the boardroom is measuring. The metrics look fine right up until the moment they don’t, and by then, the damage has been accumulating for months.

The Hallucination Problem Is Bigger Than We Want to Admit

Let me talk about something that makes many people in my industry uncomfortable: AI hallucinations in customer-facing contexts are not rare edge cases. They are a structural problem.

What makes this particularly alarming isn’t just that AI gets things wrong; it’s how it gets things wrong. There’s research suggesting that AI models are significantly more likely to use confident, definitive language precisely when they’re delivering incorrect information. Think about that for a second. The AI says “definitely” and “certainly” most often when it’s wrong. So, the customer hears a confident answer, trusts it, acts on it, and then faces the consequences when reality doesn’t match what they were told.

How Does This Play Out in Real Time?

Incorrect pricing, incorrect eligibility information, and misrepresented policies. These aren’t hypothetical harms. A customer who’s been confidently misinformed doesn’t just have a bad experience. They have a bad experience and feel deceived. That’s a completely different emotional response, and it does lasting damage to their perception of the brand.

I’ll be honest. This is something we think hard about in how we’ve designed our own AI features. Confidence without accuracy isn’t a feature. It’s a liability.

The Escalation Problem Is Where Trust Really Dies

If hallucinations are the slow poison, then broken escalation is the moment the patient collapses.

Here’s what I see happen all the time. A customer comes in with a real problem, something that genuinely requires human judgment. The AI can’t solve it.

So, what does it do?

In many implementations I’ve seen, it doesn’t hand off to a human. It loops. It offers the same menu of options. It rephrases the same unhelpful answer. And when the customer finally reaches a human agent, that agent has no context from the previous conversation, so the customer has to start the whole painful story from scratch.

Or worse, the system makes escalation so difficult, so buried, so deliberately friction-filled, that many customers just give up. They conclude that the brand doesn’t actually want to talk to them, that the AI is a wall rather than a door, and they walk away.

Research in this space points to the handoff between AI and human agents as the single most common failure point in AI-enabled customer service.

And frankly, I think some of that failure isn’t accidental. In environments where the goal is to reduce human-agent interactions at any cost, there’s a perverse incentive to make escalation difficult. That’s a short-term cost saving that is quietly costing brands their most valuable customers.

What We’re Actually Trying to Build, and Why It’s Hard

So, what does responsible AI in a contact center stack actually look like? I want to be transparent about this because I think too many vendors in my space hand-wave over it.

First, it means resisting the vanity metrics trap. A 20% improvement in first-response time is meaningless if the response is wrong. A drop in escalations is a red flag, not a green one, if customers are disengaging rather than actually getting resolved. The right metrics are harder to capture, such as resolution quality, repeat-contact rate, and post-interaction retention. But they’re the ones that actually tell you whether your AI is helping or hurting.

Second, it means building real guardrails into AI behavior. This isn’t just about putting a disclaimer at the bottom of a chatbot window. You should architect your AI system with constraints on what it can assert confidently, with verification steps built into sensitive information flows, and a frictionless path to human escalation, especially when a customer
explicitly requests one. This should be a non-negotiable design principle, not an afterthought.

Third, it means ongoing human auditing. Not just reviewing aggregate data, but having people randomly sample actual interactions to catch bias, inaccuracy, and failure patterns before they become brand crises. This is unglamorous, time-intensive work, and many companies skip it in favor of trusting the model. Don’t trust the model without checking the
model.

The Bigger Picture: AI Is a Multiplier, Not a Replacement

Here’s what I’ve come to believe after years of building in this space: AI in customer service is a multiplier. If you have good processes, thoughtful escalation paths, well-trained human agents, and a genuine commitment to customer outcomes, AI can scale all of that beautifully. It handles volume, catches the routine stuff, and frees your humans up for the moments that actually require human judgment.

But if your underlying approach is to minimize human contact, reduce costs at the expense of quality, and treat customer interactions as a number to be optimized rather than a relationship to be built, AI will accelerate exactly that, and your brand will pay for it.

The contact center of the future isn’t all-AI or all-human. It’s an intelligent combination of both, with AI doing what it’s genuinely good at and humans available, genuinely available, without friction, when they’re needed. Getting that balance right is hard. It requires honest conversations about what we’re actually optimizing for, and it requires vendors like us to
build tools that make the responsible choice the easy choice.

We’re not perfect at it. Nobody is yet. But the brands that figure it out first will be the ones customers actually stay loyal to when the dust settles.