How can we help?

Articles in this section

Troubleshooting Common AI Chat Issues in ServiceAI

Ricky Cecchini
  • Updated

This article was previously titled "Troubleshooting Common Orion Assistant Issues." It has been refocused to cover AI Chat — the main chat surface and home page — which is where most reported issues live.

AI Chat is the main chat surface in ServiceAI — the home page when you sign in, the chat inside Ticket Details, and the chat that powers the PSA pods. Like any AI tool, its quality depends on clear questions, well-configured rules, and a current view of your data.

Most issues fall into a few common buckets: responses that miss the mark, Agent Mode Rules that don't seem to apply, missing or stale context from your PSA, and persona mismatches. The sections below walk through the troubleshooting steps for each.

If these steps don't resolve your issue, please submit a ticket via support@cloudradial.com.

AI Chat Not Responding Appropriately

If AI Chat's response is off-topic, too generic, or doesn't reflect your data:

  • Confirm you are in the correct persona. Agent mode pulls from internal tools and your PSA; User mode reflects what an end user would see. Use the segmented toggle at the top of the chat to switch
  • Rephrase the question with more specifics. "Why is this ticket stuck?" is harder than "Summarize ticket 12345 and tell me what's blocking it"
  • Check the Related Sources sidebar. If no sources were consulted, AI Chat answered from the model alone — that usually means your question didn't trigger a tool call, so try mentioning a ticket number, company name, or KB topic explicitly
  • If the response cites a KB article that is out of date, update the article and try again. AI Chat is only as accurate as its sources

Agent Mode Rules Not Taking Effect

If a rule you added under Settings > Rules doesn't appear to be shaping responses:

  • Confirm the rule is saved. After clicking + Add, the rule must appear in the list before it takes effect
  • Start a new conversation or refresh the page. Existing open conversations may continue running with the previous rule set until the next turn
  • Confirm you are in Agent mode. Agent Mode Rules don't apply to User-mode conversations
  • Tighten the wording. Vague rules ("Be helpful") rarely change behavior; specific rules ("Use numbered steps for procedures with three or more actions") do
  • Look for conflicts. Two rules that pull in opposite directions will be followed inconsistently. Remove or merge contradictions
  • Remember that Agent Mode Rules also apply to the Fast Feedback agent summary triage posts to your PSA. If you only see the effect in one place, that's expected — both surfaces use the same rules but run at different times

Missing Context, Tickets, or Sources

If AI Chat says it can't find a ticket, company, or KB article you know exists:

  • Confirm your PSA sync is current. Navigate to Settings > Integrations and check the last sync timestamp for your PSA. Trigger a manual sync if it is stale
  • Confirm your KB sync is current. KB articles created in your PSA or ServiceAI within the last few minutes may not yet be indexed
  • Use exact identifiers. Ticket numbers and company names match more reliably than partial phrases
  • Refresh the page. AI Chat caches some context within a session; a refresh forces it to re-fetch on the next turn
  • If the source exists but the score pill in Related Sources is very low, the embedding match was weak. Reword the question to use vocabulary closer to the article or ticket title

Persona Mismatch

If responses look like an end-user reply when you wanted an internal one (or vice versa):

  • Check the segmented persona toggle at the top of the chat. Agent = internal, User = end-user preview
  • Inside an Autotask or ConnectWise PSA pod, persona is locked to Agent and the toggle is hidden. That is intentional — the pod is always a tech-facing surface. To preview the User experience for a pod ticket, open the same ticket from inside ServiceAI's Ticket Details page and use the AI Chat dropdown's User mode option
  • Conversations in the sidebar are grouped by persona, so an Agent-mode conversation will not show up in User-mode history and vice versa

Conversation History Looks Incomplete

If a conversation you remember having is not in the sidebar:

  • Confirm you are looking at the right persona. The sidebar shows the current persona's history; switch the toggle to find Agent or User conversations
  • The sidebar caps at five items per group with a "See more" expander. Click See more to view the full list
  • If you remember a conversation URL, paste it into the address bar — every conversation has its own URL and will load even if it has rolled out of the sidebar

PSA Pod Chat Issues

The chat inside the ConnectWise, Autotask, HaloPSA, and Zendesk pods uses the same AI Chat workflow as the home page. If the pod chat misbehaves:

  • Confirm the ticket loaded into the pod. The pod URL includes the ticket id; if that is missing or invalid, AI Chat has no context to work with
  • If the pod requires a login redirect, complete the login and reopen the ticket in the PSA so the iframe reloads
  • Check Pod Quick Actions under Settings > AI Behavior > Pod Quick Actions. A misconfigured quick action will send a bad initial prompt every time it is clicked
  • Pod conversations are Agent persona by design and will follow your Agent Mode Rules

Related articles

Comments

0 comments

Article is closed for comments.