How to Troubleshoot When AI Employee Responses Do Not Meet Expectations?

I. Function Introduction

When AI employees give incorrect answers, incomplete answers, irrelevant answers, or answers that do not conform to the expected knowledge base content, the " Troubleshooting Assistant " can be used to analyze the reasons.

The troubleshooting assistant will help you determine whether an AI's response meets expectations, understand why the AI answered in that way, why it did not use the expected knowledge base content, and provide corresponding optimization suggestions for each AI response.

You can use the troubleshooting assistant in the following ways:

On the SaleSmartly chat page, the AI's response will take you directly to HelpKnow for troubleshooting.
On the HelpKnow "Analysis" page, find the answers you need to investigate through "Conversations" or "Messages".
In HelpKnow's "Question and Answer Test," you can directly review the answers generated by the test.

The troubleshooting assistant only provides analysis results and optimization suggestions; it does not automatically modify AI employee configurations. This article will introduce the above-mentioned troubleshooting entry points and how to view and use the troubleshooting results.

Link to this article: "How to Troubleshoot When AI Employee Responses Do Not Meet Expectations?"

II. Instructions for Use

1. Check AI responses from SaleSmartly

If you use SaleSmartly to receive customers, you can directly access the details page of the corresponding AI response from the SaleSmartly chat page.

In SaleSmartly, open the customer session you need to investigate, find the AI message whose answer is not as expected, move your mouse over the message, and click "View Source" below the message.

After clicking, the system will redirect you to HelpKnow and open the details page for the AI answer. You can use the troubleshooting assistant on the right side of the page to analyze the answer.

This method can directly locate the AI answer that caused the problem, eliminating the need to manually search for the corresponding conversation or message in HelpKnow.

2. Check the HelpKnow analysis logs.

You can also access the corresponding AI employee, click "Analysis" in the left-hand menu, and find the AI's response that you need to investigate through "Conversation" or "Messages".

In the "Conversations" section, you can view a complete dialogue and then select a specific AI response for further investigation. Conversations generated from question-and-answer tests will also be displayed here, accessible through the "Preview & Debug" section; genuine inquiries from users via websites or social media can also be viewed here.

After finding the target conversation, click "Details", then click "Poor answer? Investigate the reason" below the AI answer you need to analyze.

If you have identified a specific message that needs to be investigated, you can switch to "Messages", find the message and click "Details", and use the investigation assistant on the right side of the details window.

3. Directly check in the HelpKnow Q&A test.

Before or after deploying an AI employee, you can proactively send questions through the "Q&A Test" to check the AI employee's response.

Find the AI answer you need to analyze, and click "Not good answer? Investigate the reason" below the answer.

4. Select the problem to investigate.

After opening the troubleshooting assistant, you can directly select the following issues:

"Analyzing the effectiveness of the responses"
"No images/videos/attachments were referenced?"
Why did you answer that way?
Why didn't you answer according to the knowledge base?

You can also describe the specific problem or fill in the expected answer in the input box.

5. View the analysis results

The troubleshooting assistant will analyze whether the current answer meets expectations, identify potential problems, and provide corresponding optimization suggestions.

When the investigation results involve adjustments to AI employee configurations, the page will display corresponding guide buttons. Clicking the button will directly take you to the relevant configuration for modification, without needing to exit the current page and manually search for the corresponding function.

You can adjust the relevant configurations based on the investigation results and optimization suggestions, and retest the AI employee's response performance.

III. Applicable Scenarios

Investigate abnormal responses in real consultations: When AI employees provide incorrect, incomplete, or irrelevant answers during actual consultations, analyze the specific reasons.
Before release, check the response effectiveness: Simulate user questions in the Q&A test to check whether the AI employee's response meets expectations.
Optimize AI employee configuration: Based on the investigation results, check the knowledge base, persona, or other relevant configurations to continuously optimize the response performance of AI employees.

IV. Precautions

The analysis assistant analyzes only one AI response at a time, and does not analyze all responses in the entire conversation.
The results of the investigation are used to help locate the problem, but they still need to be verified in conjunction with the actual knowledge base content and the configuration of AI staff.
The troubleshooting assistant will not automatically modify the AI employee configuration. The guide buttons in the troubleshooting results are only for quick access to the corresponding configuration; you still need to make the specific modifications yourself.
After adjusting the configuration, it is recommended to retest the Q&A. Once the answers are confirmed to meet expectations, the AI employee needs to be released for the changes to take effect in real-world interactions.