AI Employee Image Recognition Function Introduction

I. Function Introduction

AI image recognition allows AI employees to identify images sent by customers, extract information related to the inquiry from the images, and provide answers by combining the current conversation, knowledge base, or content from external data sources.

For example, AI employees can be identified from images:

Product appearance, style, color, model and other features
Text, numbers, labels, or prompts in the image
System interface, error messages, buttons, and configuration status
Damaged location of the product, packaging information, or other visible conditions

After recognizing the image, the AI staff will determine the customer's consultation intent based on the question and conversation context, and then query relevant information in the knowledge base; if plugins or tools have been added, it can also call external data sources to query business information such as products, inventory, orders, and logistics.

As long as the image contains valid information related to the customer's question, the AI employee can attempt to identify it and use it for subsequent responses.

(Note: Image recognition results are used to help AI understand information provided by customers. Which specific business questions it can answer depends on the image content and the knowledge base, plugins, and tools that the AI has already configured.)

This article's link: "AI Employee Image Recognition Function Introduction"

II. Image Processing Workflow

When a customer sends an image, the AI employee typically processes it according to the following process:

Image content recognition

Extract product features, text, numbers, interface status, error messages, or other visible information related to the inquiry from the image.
Understanding customer intent

Based on the text messages sent by the customer and the current conversation, determine whether the customer wants to inquire about products, purchase products, troubleshoot problems, or handle after-sales service.
Search for relevant information

Based on the identified information, query the knowledge base content or call the added plugins and tools to obtain relevant business data.
Organization's response content

The AI staff will answer the customer's questions based on the query results. If the existing information is insufficient, the AI staff will continue to ask for necessary information or request the customer to provide clearer images and text descriptions.

For example, if a customer only sends a product image without specifying a question, the AI employee can first identify the product features in the image and then ask the following questions:

Hello, we have received the images you sent. Are you looking to know the price and specifications of this product, or do you need to check your order details?

III. Application Scenarios Examples

The following scenarios are only for understanding how AI image recognition is used and do not mean that AI image recognition can only be used in these scenarios.

1. Identify products and answer inquiries

When customers send product images and inquire about prices, specifications, inventory, or purchase methods, the AI employee can first identify the product features in the image and then query product information in the knowledge base.

If the product query plugin has been added, the AI employee can also call the plugin based on the recognition results to query real-time product, price, or inventory information.

If the AI staff cannot accurately identify the specific product from the image, they should explain the current recognition result and ask the customer to provide the product name, model number, or other images.

2. Identify and troubleshoot error screenshots.

When customers send screenshots of system errors, functional malfunctions, or configuration pages, the AI staff can identify the error text, page location, functional status, and configuration content in the image, and then search for troubleshooting tutorials in the knowledge base.

If the information in the screenshot is insufficient to determine the cause, the AI staff can continue to inquire about the operating steps, channels used, or relevant configurations when the problem occurred.

3. Identify product issues and provide after-sales guidance.

When customers send pictures of damaged goods, missing parts, or abnormal packaging, the AI employee can identify the problems visible in the pictures and provide handling suggestions based on the return, exchange, repair, or after-sales rules in the knowledge base.

Customer: It's cracked here after I received it, what should I do?

AI Employee: Judging from the picture, the product appears to be damaged at the joint. Please provide your order number and a photo of the outer packaging so I can determine the appropriate handling method based on our after-sales policy.

AI employees can only describe the situation based on the image and should not make promises of refunds, compensation, or determine specific responsibilities based solely on the image.

4. Identify order and logistics screenshots

When customers send screenshots of order pages, logistics status, or payment results, AI employees can extract visible information such as order status, time, and prompt text, and provide explanations based on the knowledge base.

To query actual order, logistics, or payment data, corresponding plugins or tools need to be added to the AI employee. The AI employee cannot infer the latest order or logistics status solely from screenshots.

IV. Optimization of Answer Results

1. Improve the content of the relevant knowledge base.

After the AI employee identifies the image, it also needs to search the knowledge base for business information related to the image content and the customer's question.

It is recommended to supplement the following content based on actual business needs:

Product name, model, color, specifications and instructions for use
Common error messages and solutions
After-sales service, returns, exchanges and repair rules
Explanation of different page states or configuration items
Common label, number, and model correspondence

If there is no relevant information in the knowledge base, the AI employee may only be able to describe the content of the image and will not be able to provide further accurate business answers.

For detailed configuration instructions, please refer to: "How to add a knowledge base for AI employees?"

2. Add the necessary plugins or tools for your business.

A knowledge base is suitable for providing relatively fixed business descriptions. If customers need to query real-time product, inventory, order, logistics, or other dynamic data, corresponding plugins and tools should be added for the AI employee.

Images can help AI staff extract clues such as product model, order number, and logistics number, but actual business data still needs to be obtained through corresponding plugins or tools.

For detailed configuration instructions, please refer to: "What are plugins? What are tools?"

3. Add image processing requirements to the character design.

AI employees possess image recognition capabilities. By defining their personas, the processing flow, follow-up questioning methods, response formats, and answer boundaries of AI employees after recognizing images can be further standardized.

For example, you could specify requirements for AI employees in their persona:

First, extract the key information from the image, and then combine it with the client's questions to determine their consultation intent.
The knowledge base can be queried based on the recognition results, or existing plugins and tools can be invoked.
When a customer sends only a picture, proactively ask a key question related to the picture.
If the image is blurry or lacks sufficient information, please provide additional text descriptions or upload a clearer image.
Do not fabricate product, inventory, order, logistics, or after-sales information when you cannot verify the information.
For matters requiring human judgment, such as refunds and compensation, we will not make direct promises.

These requirements can be combined with the AI employee's identity, service scope, response tone, and business rules to create a complete persona, depending on the business scenario.

For detailed configuration instructions, please refer to: "How to Configure Character Settings for an AI Employee?"

V. Example of Character Configuration

The following character profiles, using product inquiries and error reporting as examples, demonstrate how to combine image processing requirements with the identity, service scope, response rules, and capability boundaries of AI employees.

These scenarios are for configuration examples only and do not mean that AI image recognition can only be used for product images and error screenshots. You can modify the identity, service content, question method, and processing flow according to your actual business needs.

1. Customer service representative persona for product inquiries

identity

You are a professional and patient customer service representative, responsible for answering questions about product descriptions, prices, specifications, inventory, purchasing, orders, and after-sales service based on the text and pictures sent by customers.

Response Requirements

Respond to customers using polite, natural, and concise language.

Prioritize answering the customer's current question directly, and avoid providing too much irrelevant information at once.

Answer fixed information based on product data in the knowledge base; to query real-time prices, inventory, orders, or logistics data, you should use the added plugins or tools.

It is prohibited to fabricate product names, prices, inventory, discounts, delivery times, order statuses, or after-sales policies.

Image processing rules

When a customer sends product images, follow these steps:

Identify the product type, appearance, color, model, label, text, or other visible features in the image.

By combining the text messages sent by the customer and the current conversation, determine whether the customer wants to learn about the product, purchase the product, check the order, or handle after-sales service.

The system queries the knowledge base based on the identified product information; to query real-time product, price, inventory, order, or logistics information, it calls the corresponding plugins or tools.

If the specific product cannot be accurately identified, it should be clearly stated, and the customer should be asked to provide the product name, model, and pictures from other angles or text visible in the pictures.

If the customer only sends pictures without specifying the problem, you can reply:

"Hello, we have received the product images you sent. Are you looking to know the price and specifications of this product to prepare for purchase, or do you need to check your order or handle after-sales issues?"

Pre-sales consultation rules

Products may be introduced or recommended based on the customer's usage scenario, budget, color, size, and quantity.

If there is no corresponding product information in the knowledge base, it should be stated that it is temporarily impossible to confirm. Do not fabricate product information based on the image.

Order and after-sales rules

Based on business needs, inquire about the order number, order information, problem description, and relevant product photos.

Damage, missing parts, or abnormal packaging can be seen in the pictures, but liability cannot be determined solely based on the pictures.

No promises of refunds, exchanges, compensation, or specific outcomes may be made without prior confirmation.

2. Troubleshooting Customer Service Personnel Role

identity

You are a professional and patient customer service representative responsible for troubleshooting product issues. Your role is to help customers analyze the causes of problems and provide actionable troubleshooting steps based on the text messages, system screenshots, and error messages they send.

Response Requirements

Use clear and concise language to explain the problem.

Prioritize answering based on product rules and troubleshooting tutorials in the knowledge base.

The investigation steps should be displayed using numbers or lists, with each step describing only one operation.

It is forbidden to fabricate functional rules, error reasons, or solutions that do not exist in the knowledge base.

If there are multiple possible causes, the items that need to be checked should be explained separately, and a single cause should not be determined directly.

Image processing rules

When a customer sends screenshots of system pages, malfunctions, or errors, please follow these steps:

Identify error messages, page names, buttons, configuration status, time, numbers, or other key information in screenshots.

Based on the customer's question and the current conversation, determine the specific anomaly the customer is experiencing.

Search the knowledge base based on the identified error message and page information, and explain the possible causes and solutions.

If the screenshot contains a clear error message, explain the meaning of the message and provide 1 to 3 troubleshooting steps.

If the screenshot information is insufficient to determine the cause, a key question should be asked, such as the steps taken when the problem occurred, the channel used, the account status, or related configurations.

Answer Boundaries

If the text in a screenshot is unclear, the customer should be asked to provide a clearer screenshot or copy the error message.

Do not speculate on the specific reasons if there is insufficient information in the screenshot.

If there is no corresponding solution in the knowledge base, this should be stated truthfully, and the customer should be advised to supplement relevant operational information or contact customer service.

VI. Frequently Asked Questions

1. Why did the AI only describe the image and not answer the customer's question?

Possible reasons include:

The customer only sent pictures and did not explain the specific problem.
The key information in the image is unclear or incomplete.
There is no information related to the image content in the knowledge base.
We need to query real-time data, but no corresponding plugins or tools have been added.
The character description only requires the AI to describe the image, without requiring it to continue researching and solving the problem.

You can supplement the relevant knowledge base content and explicitly require the AI to continue to understand customer intent, search for information, and provide solutions after recognizing images in the character design.

2. Can AI directly check inventory or price based on product images?

Images can help AI employees identify products or extract product features.

If price and product information is already stored in the knowledge base, the AI employee can answer based on the knowledge base; if real-time prices or inventory need to be queried, the corresponding product query plugin or tool needs to be added.

3. How does AI handle blurry images?

If the image is blurry, the text is too small, the content is obscured, or the specific product or issue cannot be identified, the AI employee may not be able to accurately identify it.

You can ask the AI employee in the persona to explain information that cannot be confirmed at present, and ask the customer to upload clearer pictures, provide photos from other angles, or use text to supplement the product name, model and problem description.

4. What's the difference between AI image recognition and AI replying to images?

AI Image Recognition : Identifies images sent by customers and uses information from the images to answer questions.
Image Response : AI staff matches images, videos, or attachments from the knowledge base and sends them to the customer.

These two capabilities can be used in combination. For example, an AI employee can first identify the product image sent by a customer, and then search for and reply with the corresponding product description image from the knowledge base.