ServiceAI continuously displays three essential metrics throughout the platform that assess your readiness for AI deployment and overall service desk performance. These scores are referred to as RPS (Relative Performance Score). They serve as key indicators for improvement and provide insights across the platform.
The RPS values range from 0 to 10 and analyze various data sets to deliver actionable insights regarding service desk performance and AI confidence levels.
This guide outlines the meaning of each RPS value, the method of calculation, and how to interpret these scores to make informed AI deployment decisions.
What is an RPS Value?
RPS values are metrics generated by AI that analyze your historical ticket data to assess readiness for AI deployment across four key areas:
- Ticket RPS - This is a quality score that assesses a ticket's usefulness for AI training by evaluating the completeness of its comments and the absence of unnecessary complexity. The higher the score, the more confidence the AI has (and you can have) that this ticket - or type of ticket - is AI-solvable.
- Agent RPS - This evaluates human technician performance based on response quality, tone, speed, and interaction patterns. The higher this score, the more likely it is that your agents are working on tickets with the right combination of process, communication, and understanding.
- User RPS - This estimates the user's experience based on the ticket conversations. User RPS can change over time as more comments are added. The higher this score, the more likely tickets are good AI training sources since more comments mean more data for the AI to train on.
Note that RPS values can shift based on the selected date range.
Score Ranges and Color Coding
ServiceAI employs a standardized scoring system ranging from 0 to 10, accompanied by color-coded indicators:
- Red (Below 6): Indicates a need for attention, suggesting low AI readiness or confidence.
- Yellow (6-8): Reflects moderate performance, with opportunities for improvement.
- Green (8+): Denotes strong performance and/or good readiness for AI deployment.
How RPS Values Are Calculated
Ticket RPS Calculation
The Ticket RPS is based on AI confidence in answering inquiries using the existing ticket history related to the end user's submission or subject, along with the details of troubleshooting and further explanation within the ticket body.
Calculation factors include:
- Historical ticket resolution patterns for similar issues.
- The quality and completeness of the ticket conversation regarding resolution steps.
- Lack of complex and repeated resolution instructions that do not yield issue resolutions.
Agent RPS Calculation
The Agent RPS is calculated using a combination of ticket response analysis, which includes assessing tone, evaluating response quality, and measuring the speed and cadence of responses to users.
Calculation factors include:
- Communication tone and professionalism.
- The technical accuracy of responses.
- Response time and interaction flow.
- Effectiveness in problem resolution.
User RPS Calculation
User RPS is determined by analyzing ticket comment sentiment and user reactions to agent interactions. The more detailed and lengthy the conversation, the better the AI can be trained on managing future tickets of this type.
Calculation factors include:
- Sentiment analysis of user communications.
- Response patterns related to agent assistance.
- Length of conversation and the addition of new comments.
- Overall tone and cooperation in interactions.
Interpreting Your RPS Values
Interpreting your RPS values can help you drive strategic decisions as a service desk manager. RPS values are the quickest way to gain an understanding through a numerical scale of the state of your performance across the four key metrics.
Ticket RPS Interpretation
- High Scores (7+): These tickets contain complete, well-structured information that's ideal for training AI systems. The conversations are thorough without being overly complex.
- Lower Scores (< 6.9): Tickets may be incomplete, lack proper documentation, or contain excessive complexity that makes them poor candidates for AI training.
Key Takeaway: A low Ticket RPS isn't inherently a problem; it helps identify which companies, users, or ticket types may not be suitable for AI replacement and still require human expertise. Conversely, high Ticket RPS can highlight potential opportunities for AI to manage these tickets completely.
Agent RPS Interpretation
- High Scores (7+): Agents are performing well with effective communication, proper ticket management, and good user rapport. Scores of 8+ indicate excellent performance.
- Low Scores (< 6.9): Agent performance needs improvement in areas like communication clarity, ticket resolution processes, or user interaction skills.
Key Takeaway: Agent RPS should consistently remain high. Declining scores may indicate the need for additional training, coaching, or support resources to sustain service quality.
User RPS Interpretation
- High Scores (7+): Your users are having positive experiences with your support. Their ticket interactions suggest they're satisfied with resolutions and the support process. This indicates good AI training potential from these user conversations.
- Low Scores (< 6.9): User experience may need attention. Lower scores suggest users might be frustrated, receiving incomplete resolutions, or having poor interaction quality that wouldn't serve as good AI training data.
Key Takeaway: User RPS can reveal relationship dynamics; high scores suggest strong client relationships, while low scores may indicate the need for specialized agent assignments or increased attention to prevent churn.
Using RPS Scores for AI Deployment Decisions
RPS values also offer contextual intelligence that shifts based on where they are viewed within ServiceAI.
Dashboard Level (Overall Performance)
- Ticket RPS: Represents ticket quality across all tickets in the service desk in the set time range.
- Agent RPS: Indicates average human agent performance within your service desk in the set time range.
- User RPS: Reflects average end-user satisfaction and conversation across all tickets in the service desk in the set time range.
Individual Ticket Level (Specific Analysis)
- Ticket RPS: Represents AI confidence in handling this specific ticket, expressed as "Automation Ability".
- Agent RPS: Reflects the agent's performance in this particular interaction, expressed as "Performance Rating".
- Requester (User) RPS: Indicates user satisfaction with the progress towards resolution of this specific ticket, expressed as "Outcome Satisfaction".
Agent Level (Technician Analysis)
- Ticket RPS: Reflects AI confidence in addressing the agent's tickets based on available historical data, expressed as "Automation Ability".
- Agent RPS: Indicates this agent's individual performance metrics based on available historical ticket data, expressed as "Performance Rating".
- Requester (User) RPS: Represents user satisfaction while working with this specific agent based on available historical ticket data, expressed as "Outcome Satisfaction".
User Level (End-User Analysis)
- Ticket RPS: Measures AI confidence in managing this user's requests based on available historical ticket data, expressed as "Automation Ability".
- Agent RPS: Evaluates human agent performances when assisting this user based on available historical ticket data, expressed as "Performance Rating".
- Requester (User) RPS: Represents this user's overall satisfaction and interaction patterns based on available historical ticket data, expressed as "Outcome Satisfaction".
Company Level (Client Analysis)
- Ticket RPS: Indicates AI confidence across all tickets submitted by this client organization based on available historical ticket data, expressed as "Automation Ability".
- Agent RPS: Measures human agent performance while interacting with this company's users based on available historical ticket data, expressed as "Performance Rating".
- Requester (User) RPS: Reflects overall satisfaction patterns in this client relationship based on available historical ticket data, expressed as "Outcome Satisfaction".
Strategic Decision Making with RPS Values
Utilize your analysis of RPS to make well-informed decisions regarding:
AI Deployment Readiness: High Ticket RPS scores across critical areas signify readiness for AI implementation.
Training Priorities: Low Agent RPS scores highlight opportunities for coaching and skill enhancement.
Client Relationship Management: User RPS patterns assist in optimizing agent assignments and identifying relationship risks.
Documentation Gaps: Low Ticket RPS scores pinpoint areas that require improved knowledge base articles.
Resource Allocation: RPS trends inform decisions about where to focus time and training resources.
By systematically monitoring and acting upon RPS insights, you can enhance service desk performance while progressing towards successful AI deployment that complements rather than replaces human expertise where it is most vital.
Comments
0 comments
Article is closed for comments.