AI Applications - Improving Customer Experience

 Improving Customer Experience with AI (Work-in-progress)

Landscape:

Generative AI benefits all knowledge-based workflows. CX is a knowledge-based organization; the more your team knows about the product, the related products, customers, use cases, and sales cycle, the better they can support the customer. This refers to learning from multiple parameters, and with the increasing efficiency of LLMs and TPUs, Gen AI can do this better than humans.

Think of your support engineers as superheroes, constantly connecting the dots - creating, collecting, and making sense of information to solve customer problems. Gen AI can handle the heavy lifting of processing vast amounts of data, freeing your team to focus on the creative problem-solving and human connection that truly sets them apart.

Support Engineering Resources can be used to train these domain-specific Support LLMs (note a startup idea) and provide reinforced learning with human feedback(RLHF). New startups like ema unlimited, Amolina, Covea, Einstein by Salesforce, Workato, Gainsight, and AiSera provide point GenAI solutions for CX. But there is a much more enormous scope beyond these startups' narrow Support LLMs.  

Based on my extensive work with the CX team and talking with my peers in AI and Support Industry, I am open-sourcing my thought process on improving the CX organization with AI without being superficial.

I hope you find this helpful and you can implement some of it in your organization. Customers for the Win!!

Security Caution: Be aware of cross-training LLMs with multiple customer-specific PII data. This is a data breach. Also, numerous CVEs have various vector DB attacks where a malicious prompt could expose PII data. So follow best practices and use "data-cleaning" products like Amazon Glue, Granica, and BedRock to prepare your data for LLMs. Also, multi-level LLMs should be used to keep sensitive data private.

Interesting News Article:

Note that the savings mentioned here are for non-technical work at a finance company. There is value in Gen AI for "highly technical IT" support, but the benefits are in improving the lives of Customers and Support Engineers while reducing the time to resolve.

Benefits of introducing GenAI in CX 

What is our True North in customer-facing organization? 

• Our CX Team Members ( improve their quality of work/life and happiness in work)
• Customers ( gain value out of our product)
• Company/CX Vision and Mission (plus Margins)
• Be a good company ( reduce the carbon footprint, Do Good, Beneficial to the society)

Why do we need Gen AI in customer-facing org?

• 60% of the cases are similar and worked by different engineers, providing different answers and causing delays. GenAI can identify and provide solutions similar to your best engineer
• First Responses based on the customer query need more quality. How would my best engineer respond to this query - this can be easily replicated and provide the best quality responses
• CSM/TAM organizations spend a lot of time generating reports and understanding the reports (for example - analysis of a storage usage trend or lack of new feature uptake can be highlighted by Gen AI easily)
• these knowledge-based tasks can be automated

With this, let me detail some benefits that can be converted to a PRD and provided to your AI vendors like AISera/Support Logic/Movate/Coveo/SalesForce Einstein/ServiceNow Co-pilot.

Customer facing Benefits

• Delight in Self Service

No customer wakes up in the morning and opens a case, even if your support team is the best in the world. Next to being in the product  ( more on this in my next blog on Service-ai-ability), the Support Portal should be designed to provide meaningful summarized responses through chat or search. The prompt should be designed/engineered to follow the train of thought.

Real-world Example:

Customer: I got an alert that my backup jobs are failing 

AI: Can you provide me the alert so I can answer appropriately?

Customer: Error Code:+Alert

AI: Based on the error code, your backup on Server A failed due to a SOAP error. The following solutions in that order resolved this error. ( Solution 1: 60% cases resolved, Solution 2:25%, Solution 3: 20%, etc.)

Customer: Solution 2 resolved the issue.

AI can continue and gracefully end the conversation and update the solution rate. The customer is delighted that they can resolve this issue themselves. The customer will have a positive promoter attitude towards the vendor.

• Customer-aware Chat Responses (ChatBot) 

If LLMs can retain the context (incremental learning) of previous issues, chats, and customer sentiment and provide a chain-of-thought personalized response, customer satisfaction and retention will be the highest in the industry. With the advent of fast storage (VAST Data), faster CPUs (Nvidia/GPUs), and Vector DB to correlate and RAG/fine-tuning (to get the latest info about the customer), the multi-parameter LLMs can be trained to do this quicker.

Some customers access the support case form directly, so ensure it is redirected to GenAI.

Real-World Example:

Customer: I got an alert that my backup jobs are failing 

AI: Can you provide me the alert so I can answer appropriately? Also, note that you saw similar issues last week, but the previous week's solution (Solution 2) did not fix it permanently. So, based on the new training data, I see other customers saw this failure repeatedly. With that, I have a new solution for you. Do you want to try it?

Caution: Do not make the customer engage in endless chat conversations (like telephone bots asking to press a key and forever asking to press 1, press 2, press 3 and then press 1). Chat conversations taking more than 2 iterations should have the option to bypass and open the case with all the relevant conversation information. Keep it easier for the customers, and don't ask them to enter the same information again in the case description.

Always have a customer feedback button to review and fine-tune GenAI chatbots.

• Correlating the issue with existing solutions (identifying blast radius)

We covered this subject in the customer-aware chat responses. GenAI can create more correlations in the vector DB as it is training through JIRA/Confluence/KB/Emails/Slack and Cases. A good support engineer is always looking for correlations and connecting the dots, and GenAI is expected to do the same with an effective fingerprinting engine. 

With this training, GenAI can summarize the results and provide detailed steps to resolve the issue.

Based on analyzing the data from multiple customers, GenAI can predict if an issue is bubbling up to the top and proactively help the vendors send notifications to the impacted customers.

• Reducing the time to resolve and increasing the FRR (First Response Resolution) Rate

Suppose GenAI can identify and resolve the issue on the first contact. In that case, the customer can focus on other projects instead of sending back-and-forth emails or conference calls with the support engineers.

• Self Serviceable Product

Though it is not part of the product ( refer to the next blog on supportability), the customer can ask a few questions and resolve the issue themselves without spending time on calls.

CX Organization Benefits ( CX LLM)

Support Organization Benefits: (Support LLM)

Here, I will focus on building a Support LLM on all nuances for the Support team while collecting the data securely from ZenDesk/SFDC/JIRA/ServiceNow. Then, make sense of these data to analyze customer sentiment, provide the first response, translate to an appropriate language, fix grammatical errors, create the calendar events, send notes after the phone call, sort by relevance/priority/urgency, give the following steps, escalate to the management/SMEs, generate KBs, scripts, etc. AI could be a case watcher, curating the backlog while the engineer focuses on solving the complex technical issues.

That said, AI not only acts as a personal assistant, it can also play an active role in providing technical solutions. 

• Co-pilot with a meaningful first response

The Support Engineer should co-pilot with a meaningful response that can be sent to the customer based on the prior chat, case description, customer sentiment, and how your best support engineer would respond to the customer. The support engineer should review this response to ensure it is proper before sending it to the customer. Customers should never feel that they are receiving info from a new engineer.

If the support engineer's review is positive 90% of the time, this first response engine is fine-tuned enough to be sent to the customer without the review. Random Checks and balances are essential so that GenAI is not totally unsupervised.

It would 

• Customer Sentiment Analysis

GenAI can constantly monitor customer sentiment. If the sentiment is poor, then escalate those cases to the support management so they can connect with the customer to reduce the low CSAT score.

Some indicators are - frequent escalations, certain words - frustration, case resolution taking longer than SLAs, too many handovers, and lack of customer response.

Einstein from Salesforce and Support Logic Sentiment Analyzer are some excellent examples.

• Granular Intelligent Case Categorization

As LLM is trained on the case data, it can intelligently categorize a case into multiple categories. It will quickly generate reports for the relevant engineering and sales team and take appropriate action to reduce the number of incoming cases. Manual categorization is tedious and error-prone. 

Categories can include Features (SQL Backup, VMware backup), Internal Components (processes), and Verticals ( Federal/Finance/Education/Government).

• Ref: Reducing the time to resolve and increasing the FRR (First Response Resolution) Rate

Gen AI-assisted co-pilot can help resolve cases by

• providing summarized responses of how this issue seen with other customers was fixed by other Engineers
• recently seen issues will be automatically added to the summary by incremental learning done by RAG
• we can provide confidence in the summary so the support engineers can review and verify it before sending that to the customer
• Subsequent Responses are created automatically based on the meeting minutes from phone calls (using Fathom), updating the notes in the CRM tool/case tool, creating the responses and waiting in the backlog queue for that response to be approved.
• this summary could be added to a KB, docs, or field notice as confidence increases.
• Case Deflection 

GenAI summarizes various case notes, KBs, tech docs, white papers (via tasks-specific LLMs/SLM- Support language model), recent issues (via RAG) and external articles(generic LLMs) and provides the right solution to the customer. This helps the customer to resolve the problems themselves. GenAI results are better than Search Engine results.

60% of incoming cases are repeat cases that GenAI can better service and support than most support engineers.

• Level Up Support Engineers to train LLM (Semi-Supervised/RLHF)

To increase confidence in the GenAI answers, we need highly skilled Support Engineers who know the support domain and the product to run through various scenarios and act as QA engineers/RLHF. These Senior Engineers have a lot of tribal knowledge. 

• KB Generation based on Workarounds and Serviceability Tool Generation via Code LL (Code LLaMa)/Code Co-pilot (self-training)

We can provide a way to efficiently train the LLMs with draft resolution summaries/notes/brain dumps. In that case, GenAI can produce high-quality documents and identify/correlate the incoming support cases with this resolution.

In addition to writing documents, if there is a workflow, it can be automated through GenAI - Code generator. 

Example: applying config change by editing the .conf file and restarting a service. This can be scripted and given to the customer instead of the customer following multiple steps that can be easily missed.

• Customer Aware/Domain Aware - Translation Service for Replies and Conference Calls

Currently, Support Engineers use Google Translate or expensive translation services, neither of which is context/customer sentiment or product-aware. LLMs can be trained in both the product and structured language quickly(pre-trained models) and based on the customer sentiment, they can add polite words in the translation (for example, -san to address Japanese customers)

• Auto Case Updates based on intelligent note-taking during/after Zoom Calls and a Case Crawler for the subsequent case updates

If the case is support-pending, the Case crawler in AI logic can review the current case's past and similar issues to craft a draft for the subsequent response. The Support Engineer can review and send that response to the customer.

Again, creating the meeting notes after the phone call/zoom calls will reduce the support engineer's effort.

• Build a code-less escalation and handover processes.

Building code on top of Salesforce to create internal escalations for specific customers with stricter SLAs and email/Slack internal escalations based on SLAs requires a lot of coding in SFDC/Service Now and slows down these essential tools. With Gen AI, as LLMs can manage multiple parameters, all these conditions can be checked and alerted to management accordingly during case crawling.

Use GenAI to manage auto handover processes when certain conditions are met. (p1, customer replied, Support Engineer is not online)

• Build basic Kepner-Trego troubleshooting steps.

A Kepner-Tregoe matrix is used to find the causes of a problem. It isolates the who, what, when, where, and how aspects of an event, focusing on the elements that impact the event and eliminating the elements that do not. 

This logic can be included in GenAI, though logical troubleshooting is challenging to code. As GenAI approaches Human Intelligence and AGI, it is a good stretch goal for the newer AI systems.

• Build  and Update Dashboards - Improved Visibility

As GenAI - domain-specific LLMs with Case crawls, we can prompt questions like - build me a support dashboard with TTR, Aged cases more than 14 days and so on. LLMs are trained in support jargon and can read cases from support CRM tools like Salesforce, Zendesk, DevRev, and ServiceNow.

• Vendor Specific - Log Analysis and Fingerprinting Engine

A security-minded company would deploy generic and domain-specific LLMs to extract more specific data from the logs and case data.

  a) Internal Secure LLMs  -or-

  b) hashed Vector DB using RAG to send data to generic LLMs (delete the embeddings after summarization from LLMs)

Currently, every vendor deploys ML to analyze logs, determine the blast radius of an issue, or identify a unique pattern. LLMs can be deployed to ask for common patterns in a set of logs or identify a pattern and generate an action based on it. 

For example, if the error pattern in logs is A, and the software version is B, send out a Field Notice to upgrade them to version C.

• Field Notices to give heads up to the customer

We can ask GenAI to prettify the technical doc and notify all customers who match a certain version to upgrade to B.

• Calendar - Availability (bonus points)

GenAI can parse the case for the next action and then parse the calendar for the support engineer's availability. Then, send a calendar invite to the customer based on the availability of the support engineer. 

CSM/TAM Organization Benefits

• CSM/TAM can understand customers' sentiments, summarize their case reviews, and plan the next steps. However, browsing cases and CSAT scores take a long time to understand customer sentiment.
• We can request GenAI to prepare slides and RCA docs as CX domain-specific LLMs mature.
• Product/Vendor Specific LLMs that browse through the customer call home data, CSMs/TAMs can review and find hotspots like the old version, oversubscribed licenses, etc much more quickly
• The productivity and effectiveness of CSM/TAM delivery will be improved by GenAI doing most repetitive tasks, such as sentiment analysis, usage review, case reports, feature usage reports, review of how the peers/similar vertical customers have implemented, etc.
• CSMs/TAMs can focus on building relationships and upselling under having human interaction with their customers about their future plans for security, AI, and data analytics and what their peer teams are planning for similar workflows.
• GenAI can do most of the reactive work, while CSMs and TAMs can do the proactive work.

Professional Services Organization Benefits:

• GenAI can review the SoW and identify gotchas to be reviewed and corrected by PS practitioners/project managers.
• GenAI can help review the plan and find any hotspots
• Code Llama/Code review GenAI tools (Tab9, Github Co-pilot) can help with reviewing the migration scripts
• PS teams generally develop simple scripts and utilities that can be sped by code co-pilots
• PS develops a lot of training for the 3rd party vendors. GenAI can be used to deliver these trainings. Another use case is to update the training materials/video when there is a change due to a new software release
• To be seen: Plugin to Mavenlink/FinanceForce tools that help with PS planning.

 Security Considerations:

• Secure Responsible AI will ensure one customer's PII is not shared with another customer.
• Protection via good authorization and authentication at each layer of GenAI architecture—prompt, RAG, LLM, Data Cleaning, Data layer, and Vector DB—is important. All of these are prone to MITM attacks. The bouncer at each stage verifying the identity may be important.
• RAG reduces data storing in the general purpose LLM in the cloud.
• Review the GenAI infra's architecture to prevent data from being cross-pollinated to another customer.
• Confidential Computing, which provides the next protection level, has recently gained popularity. It is a cloud computing technology that protects data during processing. Exclusive control of encryption keys delivers stronger end-to-end data security in the cloud. Confidential computing technology isolates sensitive data in a protected CPU enclave during processing.

Unveiling the Magic of Generative AI: A Journey of Learning and Discovery

Unveiling the Magic of Generative A.I.: A Journey of Learning and Discovery

The world of Artificial Intelligence is captivating, and as it continues to evolve, I'm on a personal mission to share my learning experiences with you. Through this blog, I hope to solidify my understanding of Generative A.I. while empowering others to grasp this revolutionary technology.

The Power of Creation:

Generative A.I., like a skilled artist, can create entirely new content – images, text, and answers – based on the knowledge it has absorbed. This ability is fueled by vast training data, allowing it to understand patterns and replicate them in novel outputs.

Beyond Imitation: The Spark of Human-like Logic:

While Generative A.I. excels at mimicking existing data, Artificial Generative Intelligence (AGI) aspires to push the boundaries further. This next-generation technology aims to incorporate a more human-like approach, incorporating logical reasoning to enhance its capabilities.

The Building Blocks of Generative A.I.:

Let's delve into the key components that power this remarkable technology:

• Data Sources: Imagine the training data as the paint and brushes of an artist. This information comes from various sources like S3, SQL databases,  Data Lakes, and Lake Houses; most can store structured and unstructured data.
• Data Cleaning: Before feeding the data to the A.I., it's vital to ensure it's accurate and unbiased. Tools like Trifecta and Glue help refine the data, making it easier for the A.I. to understand and utilize effectively and de-sensitize the data. Make Data Great Again for LLM.
• Vector Databases: These specialized databases store information in a multidimensional format, allowing efficient retrieval based on similarities. Think of it as organizing your paintbrushes by colour for faster access while creating a masterpiece. MetaData stored in Embeddings. These vectors can be hashed for authorized customers so no other can access the data. These embeddings are sent to LLMs to produce the content (text, image,etc)
• Large Language Models (LLMs): These are the workhorses of Generative A.I., capable of learning and adapting rapidly. They come in various forms, including general-purpose, task-specific, and domain-specific models, each suited for different purposes. Their capabilities are measured by metrics like MMLU and the number of parameters they can process. 1-bit LLMs are a fad now.
• Retrieval-Augmented Generation (RAG): This technique helps ground the A.I.'s outputs in reality. Imagine telling your artist friend to paint a car but specifying it as the latest model navigating a new road system. RAG incorporates such additional context to enhance the accuracy and relevance of the output. RAG helps send only the data as needed to LLM and then clean up the data to prevent LLM Data leaks.
• Fine-Tuning: Similar to an artist refining their technique, Generative A.I. models can be further specialized by exposing them to targeted training data relevant to specific tasks.
• Hallucinations: Just like artists can make mistakes, A.I. models can sometimes produce inaccurate or nonsensical outputs. This can happen due to inadequate training, noisy data, unclear prompts, and a lack of constraints.
• Natural Language Processing (NLP): This field bridges the gap between human language and A.I., allowing us to interact with the models using natural language prompts and questions. Hugging Face Transformers pipleline is a popular platform for accessing pre-trained NLP models.
• Prompt Engineering: Just like giving clear instructions to an artist, crafting effective prompts is crucial for guiding the A.I. towards desired outcomes. This involves providing a transparent thought process and clear instructions to ensure the A.I. generates the intended images, text, or answers.
• TPUs/GPUs - To accelerate LLMs' massive data processing, Tensor processing Units can speed up the training process by optimizing for neural network computations/matrix multiplications. Measured by tokens per second

The Applications of Generative A.I.:

The potential of this technology is vast, with applications spanning:

• Image and video generation: Imagine creating visual content tailored to specific needs.
• Question-answering systems: Get reliable answers to your queries, powered by A.I.'s knowledge.
• Chatbots/Agents: Interact with intelligent bots for information.
• Chain of thought reasoning: Gain insight into the A.I.'s thought process behind its outputs, fostering trust and understanding.
• Summarization and Paraphrasing: Entire customer calls, meetings or Terms and Conditions docs could be efficiently summarized so that others can more easily digest the meeting minutes. LLMs can take large amounts of text and boil it down to the most important bytes.
• Translation between Languages - Context Based (disk in medicine and technology could mean different things), Sentiment Based ( politeness when responding to an irate customer) and relation based ( sending a note to wife/customer/collegue )

This is just a glimpse into the fascinating world of Generative A.I. Stay tuned for future blog posts where we'll delve deeper into each component and explore the exciting possibilities they present.

Thank you for joining me on this exploration of A.I.!

P.S.: Rewording Powered by Gemini(google model)