UNRESOLVED
Seeing which knowledge source is failing, and why
Beth-Ann Sher’s team at Fin preps knowledge for smaller releases in minutes or hours, work that used to take weeks, and a daily Claude task backstops what slips through. She walks through that pipeline and the visibility gap she hasn’t closed: no readiness score for each source Fin’s AI Agent draws on.
Engineers at Fin file a product intake form while they’re still building a feature. When it’s roughly working, they run a Claude skill that writes a “what we built” document, then add its limitations and the plans it’s available on. That document goes into a tracker anyone at the company can see, and it creates a task for the manager who assigns the knowledge prep. Beth-Ann Sher’s team starts from there.
No single view of how knowledge is performing
Beth-Ann can see Fin’s automation rate, and how often a piece of content is used and resolves a conversation. Below that level, the picture gets thin.
Fin’s AI Agent draws on two help centers, developer docs on another platform, customer stories from the company blog, snippets, and internal articles. She’d like a content readiness score next to each source, then a way to drill into the weakest one and see which piece is underperforming and why. The content might be irrelevant to the customer’s question, or it might have gone nine months untouched while Fin keeps using it.
The human side has the same gap. She can’t see what a reader does after finishing a help center article, so prioritizing what to fix is tricky.
I’d like better visibility into how content is performing, for humans and for AI, on a more granular level.
What Beth-Ann’s team built to keep pace with releases
Beth-Ann works with one other Knowledge Manager and a rotation of 8 frontline Support Specialists who get time out of the Inbox. Her stack is Intercom with Operator and Fin inside it, plus Claude and Superhuman Docs.
She was hired as the first primary owner of a help center that customer education, which sat in Marketing. When it moved to support, the job became troubleshooting, and sharing product limitations and workarounds became acceptable.
Release volume forced a dedicated role for new product introduction, or NPI. That manager coordinates with Engineers and Product Managers, then passes information to the Knowledge Managers and the support team. Here’s how a release moves:
- Engineers submit the intake form in Superhuman Docs during the build.
- A Claude skill drafts the “what we built” document, and the Engineer edits in limitations and plan availability.
- The document lands in a company-wide tracker and creates a task for the NPI manager, who assigns help center prep and Support enablement.
- The Knowledge team feeds it to Operator, which finds everything the change affects and drafts the updates.
Prep for smaller releases that used to take weeks now takes minutes or hours. Two checks still stay with people, because Operator has no context for audience rules or plan requirements unless someone supplies it. For a net-new article, she follows the steps in the product herself. The team also saves a style guide and a release readiness checklist to Operator’s instructions, so every teammate’s chat applies them.
The fallback for what slips through is newer. Fin has a Slack channel called Product Updates where a post goes up every time something ships. Beth-Ann set up a recurring Claude task that reads that channel, and connects to Operator to draft changes the team might have missed.
She’s also working with Fin’s Customer Success team on a proactive use for Fin, where it starts the conversation. If a customer has access to a feature and hasn’t started using it, or has content nobody has reviewed in over six months, Fin can reach out.
Beth-Ann’s pipeline starts before a release exists. Engineers describe what they built while they’re building it, and every later draft works from that description. She said that when knowledge stays in sync with the product, Fin can resolve most queries about a new feature from day one. What she can’t yet see is which source feeding those answers is weakest, or why.
Beth-Ann is still working toward a readiness score for each source, a way to find the specific piece of content pulling one down, and a read on why. If you’ve built a Content Score, her LinkedIn is the place to start.
That’s what Unresolved is for.
Links mentioned
Follow the show
Have a story to tell?
So many CX leaders are testing with AI and rebuilding with AI as their foundation. Every one of us have to reinvent everything that we’ve known about customer experience systems and processes.
You’re not alone, so let’s do this journey together.