UNRESOLVED
The signal AI doesn’t know it’s missing
Robert Cabral’s small team at Runway built AI support that handles first-contact resolution and billing decisions without adding headcount. The handoff context problem got fixed in the first week. What’s still open is harder: how do you know what AI resolved without you ever seeing it?
When Runway’s AI support launched, the handoff broke almost immediately. A customer would explain their situation, get transferred to a Support Specialist, and find themselves starting from scratch. Robert Cabral’s team caught it in the first week. They built a system that aggregates the full conversation, packages any attachments the customer submitted, and delivers a summary as an internal note before a Specialist opens the ticket. The team reads a summary instead of a thread. That fix landed. What’s still open is quieter and harder to close.
AI answered the question. The signal problem is harder to see.
When every ticket came through a human, Runway’s support team had informal signal built into the process. Someone would mention they’d seen three customers complaining about slow generations. Someone else would confirm. That pattern surfaced in minutes and reached the people who could act on it. With AI handling first contact, that loop broke. What AI has is a response to the prompt in front of it.
Robert described a specific failure mode: a customer reaches out because generations are taking longer than usual. AI responds with standard browser-check guidance. It’s technically correct and probably wrong. If there’s a platform-wide slowdown, AI gives the same response to the next fifty customers before anyone on the team notices the pattern. Discord serves as a secondary signal source now. Robert is monitoring both while he builds something more systematic. Timmy Highly manages the Discord community is the reason that signal source works.
He’s building something to close the gap. Every team at Runway has access to LLM tooling and the leeway to build with it. Robert is developing a signal detection layer that can surface patterns across AI conversations before they become a volume problem. It doesn’t exist yet in a finished form. That’s the unresolved.
Robert isn’t skeptical of AI. He’s built his team around it. He’s clear-eyed about its limits.
What Robert’s team actually built
Robert joined Runway when there was no real support structure. He inherited a backlog and started by addressing infrastructure and process before touching AI. Documentation came first. Runway has one person, Haley Mills, whose full-time job is building and maintaining documentation and guides. Robert credits that foundation as the reason AI support actually worked when it went live.
The team is small by design. In a year and a half, headcount hasn’t grown. They recently rebranded from customer support to customer experience to reflect a scope that now includes consumer support, enterprise support, customer education, and a scaled success division for enterprise accounts under a certain threshold that need more than standard support but don’t qualify for full account management.
Hiring into that team is specific. Everyone has some kind of creative background. Robert mentioned a team member with a dance background as an example. Runway’s customers are creatives first, and the empathy that comes from working in a creative discipline translates directly into understanding what those customers value and how they experience frustration with a tool.
AI handles the simple questions: how to reset a password, which model to use for a specific type of content. Billing and refund decisions are automated with hard-coded logic tied to actual account data, usage history, and renewal timing. AI makes the call, but it isn’t doing open-ended reasoning on those decisions. The guardrails are strict, built in partnership with the engineering team. Robert checks reporting weekly for anomalies.
Support Specialists became level two and level three. They handle questions that require industry knowledge, context, or judgment that AI doesn’t carry. When the handoff broke in the first week, the team rebuilt it. The current system packages the full conversation and any customer-submitted attachments as an internal note, so Specialists arrive at a ticket knowing what happened. That change, Robert said, vastly improved how customers perceived the move from AI to human support.
Discord runs as a parallel signal source. Runway’s community, managed by a separate team, surfaces conversations the ticket queue doesn’t capture. Robert monitors both to compare what’s showing up in the community against what’s showing up in AI conversations. It requires someone to look. That’s the gap he’s trying to build past.
Runway moved deliberately on AI. They fixed processes, built documentation, and ran a thorough vendor evaluation before anything went live. When the handoff broke, they caught it in a week and built a fix. The signal problem is harder because it doesn’t announce itself. AI resolves the ticket, the customer leaves, and the pattern only surfaces if someone’s watching for it. That’s the work that doesn’t come with the platform.
Robert’s unresolved isn’t a handoff problem anymore. It’s a visibility problem. When AI handles the volume, the support team stops being the first set of eyes on customer frustration. The question he’s working through is how to get those signals back without rebuilding the manual process that AI was supposed to replace.
If you’ve built something that captures AI signal patterns at scale, or you’ve found a way to close the loop between what AI resolved and what the team actually learned from it, Robert would like to hear from you. That’s what this show is for.
Links mentioned
Follow the show
Got a problem you’re living with?
Unresolved is built around honest conversations with Manager and
Director-level CX leaders. If you’ve got a problem you haven’t
quite cracked yet, that’s exactly the kind of episode we’re after.
Be a guest