Confidence Thresholds
Tune when Ticket0's AI is allowed to send replies without operator review.
Ticket0 scores every AI draft from 0 to 100 using a rule-based system. The score decides whether the draft is auto-sent to the customer, shown to an operator for one-click review, or suppressed entirely — and you control where the auto-send line sits.
The two thresholds
There are effectively two thresholds in the pipeline:
- Auto-reply threshold — workspace-configurable, default 85. Drafts at-or-above this score are auto-sent (if
autoReplyis on). - Human-required floor — hardcoded at 60. Drafts below this score are considered too weak to show; the ticket goes to the operator with an empty composer.
That gives three decision bands:
| Score | Decision | What the operator sees |
|---|---|---|
≥ auto-reply threshold | auto_send | Nothing to do — AI already replied (and the ticket moves to Pending). Applies only when autoReply is on; below that, it falls into human_review. |
60 – (threshold − 1) | human_review | Draft is pre-populated in the composer, ready to review and send. |
< 60 | human_required | Empty composer, no draft. The operator writes from scratch. |
Raising the auto-reply threshold shrinks the auto-send band and grows the human-review band (more drafts land in the composer for approval). Lowering it does the opposite — more drafts are sent without review, fewer land in the operator's inbox.
What goes into the score
The score combines six signals (stored on each ticket as draftReplyConfidenceBreakdown for transparency):
- KB match quality — cosine similarity of the top retrieved knowledge-base article. Higher match → bigger bonus.
- Classification confidence — how certain the classifier is about the ticket's topic and intent.
- Customer sentiment — positive nudges the score up;
negativeandfrustratedpull it down. - Topic risk — billing and account topics apply a penalty, because mistakes in those areas hurt more.
- Pattern matches — the more resolved similar tickets we find, the more confident we are the draft reflects your team's actual handling.
- Draft quality — very short drafts (< 15 words) are penalised; longer, substantive drafts get a small bonus.
Every component's contribution is visible in the ticket's draft breakdown — there are no black-box signals.
Configuring the threshold
Go to AI Agent → Features → Auto-reply confidence threshold and enter a value between 60 and 100. (Setting it below 60 has no effect — the human-required floor takes precedence.)
The threshold only fires when autoReply is on. With autoReply off, the score still determines whether a draft is shown (≥ 60) or suppressed (< 60), but nothing is ever auto-sent — every draft waits for an operator.
Tuning guidance
Starting points by risk profile:
- General support (default) — 85. Errs on the side of operator review.
- Low-risk, high-volume FAQ / order status — 75–80. More auto-sends when the AI is reasonably confident.
- Sensitive areas (refunds, account changes, compliance) — 90+. Essentially only send when the AI is near-certain.
Review AI Agent → Overview weekly during rollout: look at the auto-send count, the draft-accept vs draft-edit ratio, and any customer complaints tied to AI-sent replies. If edits or complaints are trending up, the fix is usually better KB content or tighter tone/persona instructions — not the threshold. The threshold is the last lever to pull, after content and prompts.
Lowering the auto-reply threshold sends more replies without operator eyes on them. Raise it if auto-sent replies are being edited after the fact, flagged by customers, or misrepresenting policy.
Related
- AI feature toggles — turn
autoReplyon/off - Setting up your OpenRouter key — required before AI features work
- How the AI scores drafts — the agent-side view of the same scoring