Go-Live Playbook

Rushing straight to full automation on day one is the most common mistake teams make. This playbook describes the three-phase rollout that gives your team confidence in TR-AI's accuracy before it operates autonomously.

Phase 1 — Shadow mode (Week 1–2)

In shadow mode, TR-AI processes every incoming ticket and logs what it would have done — but does not post any replies or move any tickets. Your agents handle everything as normal.

What to do in shadow mode

1. 1
   Connect the webhook but disable auto-reply

   In your Dashboard, set the confidence threshold to 1.0 (effectively disabling auto-reply). TR-AI will still process tickets and log outcomes.
2. 2
   Review the resolution log daily

   Check the Dashboard → Recent resolutions feed. For each resolved outcome, open the ticket in your helpdesk and compare the AI-generated reply to what your agent actually sent.
3. 3
   Identify KB gaps

   Every triage outcome represents a ticket TR-AI could not resolve. Review these tickets and add or improve the corresponding KB articles.
4. 4
   Target: 70%+ simulated resolution rate

   After one week, if at least 70% of tickets show a resolved outcome in the log, you are ready for Phase 2. If lower, continue improving the KB.

Phase 2 — Assisted mode (Week 3–4)

In assisted mode, TR-AI posts replies but does not close tickets. Your agents review each AI reply before the ticket is marked resolved. This lets you catch any incorrect replies before they reach customers.

What to do in assisted mode

• Keep the confidence threshold at 0.82 (the default) — in assisted mode every reply still waits for approval.
• Have agents review every AI reply before sending or closing.
• Track the percentage of AI replies that agents approve without edits. Target: 85%+.
• For every edited reply, identify whether the issue was a KB gap or a reply formatting problem and fix accordingly.

Phase 3 — Full automation

Once your team is confident in reply quality (85%+ approval rate in Phase 2), enable full automation: TR-AI posts replies and closes tickets without agent review.

Ongoing monitoring

• Review the triage queue weekly — these are tickets TR-AI could not resolve. Each one is a KB improvement opportunity.
• Monitor CSAT scores for AI-resolved tickets vs agent-resolved tickets.
• Re-index the KB whenever you publish new documentation or make significant product changes.

Phase summary