Mindful Software: Building Agentic Automations using GenAI

Currently, software development and automation are painful. The software or automation team has to complete almost 95% of the process, taking care of all corner cases or the tribal knowledge accumulated over the years. If the developer misses anything, it comes back as a bug, and only the software engineer or automation developer can fix it by including the corner case. In addition to fixing the code, this process has to go through the entire lengthy software development life cycle of change management, QA and deployment in the sandbox and production.

With Kognitos, our customers develop the basic logic for their processes using English syntaxes and improve their accuracy over time by adding learnings. The learnings could be a simple one-liner, new logic to address the corner case, or a new document type. Thus, we create a way to capture tribal knowledge methodically and keep the records forever.

Neuroplasticity: Kognitos's method is not new, but this is how our human brains are designed. Babies are born with fewer neural connections. Humans learn a lot from their surroundings, and other developed humans in the first few years. 

With Kognitos, when the system encounters a new condition/situation, the system prompts an exception and waits for the process owner to review the English exception. The business operations team provides guidance on addressing the new situation. Until the exception is handled, the process will not use any compute resources.

Kognitos supports multiple learnings for similar exceptions, and Gen AI guides the system to the best context based Learning for the current situation or document. Ref: https://caff-ai-nate.blogspot.com/2024/03/vector-databases.html 

Unlearning an obsolete condition:

It is as easy as deleting the Learning from the UI instead of having to rewrite the entire automation.

Example:

The Main Process was activated through an email titled "CarDealer---Customer-Service-PUBLISHED-to-review-an-email-7jxxxxx@sb.kognitos.com." As you can read, we request chat GPT to classify this email concisely and send an email accordingly.

get the email body as the email text

ask koncierge

  the openai model is "gpt-4o"

  the task is "Review {the email text} and {the email subject}and classify the email based on the following rules: For any email inquiring about when a new vehicle will be delivered, the output should be 'Vehicle Delivery Updates'. For any email about fuel for their car, the output should be 'EV Card Issues'. For any email with mileage questions, the output should be 'Mina'. For any email where the sender is stating that they have been in an accident or their vehicle has been damaged, the output should be 'Please refer abcd.sharepoint.com/dealing-in-an-accident'. For any email inquiring about service or repairs, the output should be 'Please refer abcd.sharepoint.com/how-to-service'. Be concise."

get the above as the output

split the email sender with

  the delimiter is "@"

get the above as the email values

get the first email value

get that as the username

send an email to the email sender where

  the subject is "RE: {the email subject}"

  the message is "Dear {the username},<br><br>Thank you for reaching out.<br><br>Based on the content of your message, I have determined that you should...<br><br><br>{the output}  <br><br><br>Thank you again for your inquiry. Please feel free to reach out with any further inquiries.<br><br>Cheers,<br>Kognitos<br><br><br>From: {the email sender}<br>Date: {the email date}<br>Subject: {the email subject}<br><br>{the email body}"

As we see, we forgot to add a condition to see what happens if the information is missing from the email,

Our excellent car salesman can answer the exception used in similar situations. (Learning)

What is AgenticAI?

Agentic AI is a type of AI-driven automation that allows AI agents to perform complex tasks independently and to adapt to changing situations. It can analyse data, recognise patterns, and make decisions without human intervention.

This forays into an Agentic AI solution for your automation. As Kognitos Automation develops, all these exceptions can be used to learn as much as possible without the pitfalls of current GenAI (hallucinations and lack of predictable outcomes). These human interventions, i.e., exceptions, can be learnt. Thus, the Kognitos process created for automation can become more Agentic as the written process and LLMs evolve.

Our product has all the elements of Agentic AI except that we require minimal human intervention when encountering a new situation that needs to be considered while implementing the Kognitos process. As the Kognitos system learns these exceptions, it will eventually be trained to become Agentic. Our Kognitos system can generate new processes with minimal human input as the LLM models evolve.

Nonetheless, we are enhancing the process development lifecycle through the SDLC feature. Stay tuned for more updates.

Watch this demo to understand how our platform interacts with SAP - https://www.kognitos.com/resources/videos/extracting-information-from-sap-sales-order-with-kognitos/ 

Kognitos: Your AI-Powered Automation System

Introduction:

Kognitos is a revolutionary business automation platform that harnesses the power of Generative AI (GenAI) to streamline your workflows. Our intuitive, English-based interface empowers you to create and manage automation without complex coding.

Analogy:

It is like a human learning a new skill, like how most of us learn to drive.

1. Read the Drivers Manual ( Connect to Kognitos Books)

2. Learn to Drive the car with the learner's license ( Playground testing)

3. Pass the driver’s test ( move it to process)

4. Learn while driving on the new roads and conditions - new signal on the road (exception handling)

5. Drive with less effort and zero accidents

Key Features:

• Natural Language Automation: Describe your desired automation in plain English using our innovative FlexGrammar syntax.
• Serverless Infrastructure: Leverage the efficiency and scalability of serverless architecture to reduce costs and complexity.
• Third-Party Integrations: Seamlessly connect to your favorite tools and applications through our extensive library of integrations.
• Patented Exception Handling: Kognitos learns from exceptions, adapting to new scenarios and avoiding costly downtime. 
• Continuous Learning: Our platform improves its understanding of your unique processes, ensuring optimal performance over time with manual exception handling

How It Works:

Architecture 

1. Create Automation: Use our user-friendly interface to define your automation in plain English.
2. Test and Refine: Experiment with your automation in the playground to ensure it meets your specific needs.
3. Deploy and Scale: Promote your playground to a process. These processes/automation can be triggered via email or scheduled. 
4. Continuous Improvement: Kognitos learns from exceptions and adapts to new scenarios based on input, ensuring your automation remains effective forever.

Benefits:

• Increased Efficiency: Automate repetitive tasks, allowing your team to focus on higher-value work.
• Reduced Errors: Minimize human error and ensure accuracy in your processes.
• Faster Time-to-Value: Quickly implement automation without extensive technical coding expertise.
• Scalability: Easily adapt your automation to changing business needs.
• Cost Savings: Leverage the efficiency of serverless SaaS Platform

Example Automation:

• Sample Code(s): ( with our UI preview)
• Extract doc related to a vendor and translate it into different languages

• Sample 2:(connect to external app) Find the Name in the email body Get the above as the lead name Find the Title in the email body Get the above as the lead title Connect to Salesforce create a lead in Salesforce with    the lead status is "New"    the last name is the lead name    the title is the lead title    the lifecycle stage is "marketingqualifiedlead" Sample 3 ( with GPT prompts): process each file as follows get the file as a scanned document get the document's lines      ask koncierge          the task is "{the lines} \n-----\n You will be provided with a questionnaire. Find the following information in the document: telephone number, e-mail, nationality. Print the telephone number, e-mail, nationality. No explanation necessary."          the openai model is "gpt-4o"          the rules are "do not include any explanation", "do not include any description", "in the case that a value is not found, just print 'value not in the document' for it. Do not ask for further guidance", "make sure the output is ONLY a json list of rows"         the response format is "table" create a table from the above answer Conclusion: Thanks for reading this blog. Kognitos is moving the automation code to English, where the domain experts in accounting/finance and HR can write and manage their day-to-day automation tasks with minimal help from IT/programming experts. If you need more information about how Kognitos can help with your workflow/business automation, please visit http://www.kognitos.com. Please read the Kognitos Blogs from our CEO and other top-notch industry leaders: http://www.kognitos.com/blog.

  
  

Embedding AND Vector Databases - creating a long term memory

What Are Vector Databases? - Intelligent memory of the GenAI

While traditional databases store data in rows and columns, a vector database stores data as math vectors. Each piece of data is represented as a point in high-dimensional space, with hundreds or thousands of dimensions. This allows very sophisticated relationships between data points to be captured.

Searching and analyzing vector databases relies on vector mathematics and similarity calculations. By comparing vector positions, highly relevant results can be returned, even if there are no exact keyword matches.

Vector databases index and store the vector embeddings/tokens for faster retrieval at interactive speeds and similarity search with capabilities like CRUD (create, read, update, and delete) operations, horizontal scaling, and serverless.

Why Are Vector Databases Important for AI?

Vector databases are ideal for managing and extracting insights from the enormous datasets required to train modern AI models. Key advantages include:

In the midst of the Gen AI revolution, efficient data processing is crucial not only for GenAI but also for efficient semantic search. GenAI and semantic search rely on vector embeddings/tokens. This vector data representation carries semantic information critical for the AI to gain understanding and maintain a long-term memory they can draw upon when executing complex tasks.

Embeddings/Tokens

LLMs generate embeddings with many attributes or features linked to each other to represent different dimensions essential to understanding patterns, relationships, etc., making their representation challenging to manage.

That is why we need a specialized database to handle this data type. Vector databases like Pinecone meet this by offering optimized storage and querying capabilities for embeddings. Vector databases have the capabilities of a traditional database that are absent in standalone vector indexes and the specialization of dealing with vector embeddings, which traditional scalar-based databases lack.

Embeddings (arrays of numbers) represent data(words and images transformed into numerical vectors that capture their essences). For example, the phrase puppy and dog will have similar embeddings with vectors close to each other. These embeddings are stored on the vector DB.

Puppy = 0.3, 0.5, 0.9, 0.8, 0.4...]

Dog =[0.1,0.51, 0.6, 0.2, 0.8,,,]

Numbers depend on the ML algorithm and model.

If you can convert a text, sentence or image into many vectors, you can compare, detect, and find the closest cosine similarity, semantic similarity, etc.

OpenAI’s text embeddings measure the relatedness of text strings. Embeddings are commonly used for:

• Search (where results are ranked by relevance to a query string)
• Clustering (where text strings are grouped by similarity)
• Recommendations (where items with related text strings are recommended)
• Anomaly detection (where outliers with little relatedness are identified)
• Diversity measurement (where similarity distributions are analyzed)
• Classification (where text strings are classified by their most similar label)

Embedding Apps - GloVe, OpenAI, Word2Vec

Vector DBs are pinecone, Milvus, PgVector, Weaviate

here is how to create an embedding for the text "food" via an Open AI model 

curl https://api.openai.com/v1/embeddings \

  -H "Authorization: Bearer sk-“VPVgpYYi5znT3BlbkFJj0otiGN" \

  -H "Content-Type: application/json" \

  -d '{

    "input": "food"",                             

    "model": "text-embedding-ada-002",

    "encoding_format": "float"

  }'

More details: (Credits)

1. https://platform.openai.com/docs/api-reference/embeddings

2. good video course - explains the theory as well as the setting up of vector db

3. 

 3. https://www.youtube.com/watch?v=ySus5ZS0b94

Right size the vector DB:

Setting up Vector stores introduces new challenges. For example, correctly partitioning large data that cannot fit entirely in RAM in vector stores like Milvus is not easy. 

- Doing it poorly/under partitioning can result in some queries taking up too much RAM and bringing the service down.

- RAG responsiveness significantly depends on reducing the probes required to find relevant documents. So avoid over-partitioning as well

The Road Ahead

As GenAI moves into mainstream applications, vector databases' role will only grow. Their ability to organize and structure knowledge in a format tailored for AI aligns with the needs of next-gen generative models. 

Combining vector databases and transformers allows GenAI to understand language meaning rather than just keywords. This next-generation AI capability, powered by vector math, delivers such natural, intelligent conversations.

Data for AI - Storage, ETL, Prepare, Clean and update the data

Taking your good data to AI

The most commonly used phrases

• Garbage in, Garbage Out 
• Bad input produces bad output 
• Output can be only as good as input. 

Soon: Ethically Sourced, Organically Raised, Grass Fed Data at a Higher Price.

If we properly source and manage the data, LLMs will be trained on the correct data, causing fewer hallucinations. Unremembering or Unlearning specific segments of LLM will be one of the significant facets of GenAI in future.

Teaching the kids wrong things is worse than not teaching them at all.

https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist

Why do we need to be careful about source data?

1. Incorrect Information: This could lead to AI providing answers that could be disruptive. Need to be careful when prescribing steps for a problem that could lead to severe complications

2. PII and Secure Data: Inadvertently sharing the secure private data of one client to another client. Data Classification and Desensitization using GenAI to preprocess the data to be utilized by AI is becoming a significant business proposition. There are quite a few startups in this space.

3. Feeding Data driven by an agenda: IMHO, we all know about the Gemini fiasco that was providing results that were not truthful because the truth hurts or the truth is not politically correct

4. Properiterary/Copyright Data: How do we monetize and attribute these proprietary research data to the correct author and content creator to prevent plagiarism and reward the inventor? This would be another area of new startups.  

5. Using Publically Available Data has its downfall as well.

https://venturebeat.com/ai/openais-sora-the-devil-is-in-the-details-of-the-data/

"Generative AI copyright battles have been brewing for over a year, and many stakeholders, from authors, photographers and artists to lawyers, politicians, regulators and enterprise companies, want to know what data trained Sora and other models — and examine whether they really were publicly available, properly licensed, etc."

The legal side is a big part of this, but let us review the technical side.

Here are some thoughts on data - types of data, storing, accessing, cleaning, preparing and updating the data

1) Structured Data: Structured data fits neatly into data tables and includes discrete data types such as numbers, short text, and dates.

2) Unstructured Data: Unstructured data, such as audio and video files and large text documents, doesn't fit neatly into a data table because of its size or nature.

3) How to store the data - Fast Storage like VAST and Pure stocks are rising as demand for low latency storage requirements increase

4) Sourcing the data without latency - primary data accessed by the business applications can't be used for observability using AI insights/analytics because it will impact the performance of the production business applications. Again, backup data can't used for analytics as it will generally be a few days older, and the answers will be aged. Databricks/Snowflake are pioneers in the Warehouse/DataLake and Lakehouse technologies with ETL pipelines using Apache Spark to manage both structured and unstructured data with the ability to run CPU-intensive queries on these data. This helps to replicate the data almost immediately for training LLM/analytics purposes.

5) Preparing the data for AI - 

     a) Improve the data quality, 

     b) integrate multiple data sources - Data integration can help you access and analyze more data, enrich your data with additional attributes, and reduce data silos and inconsistencies. ETL with data sync can help. Databricks is helpful for this.

     c) Data labelling: To label your data, you can use tools and techniques such as data annotation, classification, segmentation, and verification.

     d) Data augmentation can help with data scarcity, reduce bias, and improve data generalization and robustness.

     e) Data Governance: Data governance involves defining and implementing policies, processes, roles, and metrics to manage your data throughout its lifecycle. It can help you ensure that your data quality, integration, labelling, augmentation, and privacy are aligned with your AI objectives, standards, and best practices. You can use frameworks and platforms such as data strategy, stewardship, catalogue, and lineage to establish your data governance. 

6) Desensitizing the data for AI: To protect your data privacy, you can use tools and techniques such as data encryption, anonymization, consent, and audit.

7) Data management with proper Authentication/Authorization(IAM): Store and Isolate the data based on the users. Multitenancy and reducing cross-pollination of data without less cost. Having one LLM for each client will be an expensive proposition.

Secure-minded design to protect the data:

 Tier structure for LLMs—general, Domain-Specific, and private LLMs to protect the data or RAG/Grounding with hashed metadata embeddings in VectorDB.

Product improvements using GenAI - Serviceability and Usability

 Serviceability is the last in the mind of most Product managers and Developers.  90% of the product users have less time to play around with various configurations to make the product work, as this is one of many products they manage.  Nobody has time to read the docs.

The product managers say that their product is like Apple, but they need to remember to provide guardrails and alerts in a way that makes the product self-serviceable or self-healable.

In this blog, we will review how GenAI can improve products' usability and how product managers make the product suitable for LLMs to learn fast.

- Make the products and documents GenAI ready

- Products utilize GenAI principles to self-heal and be better usable/serviceable.  LLMs for Proactive monitoring and self-healing:

GenAI Ready Product:

  1.  Logs:

Logs generated by the product should have a clear structure, making it easy for LLMs to train on these logs.  Easily identifiable PII data.

Error: TimeStamp: Message in Clear English 

Info: Timestamp: Message in Clear English

All the processes in your product should follow a similar pattern.

  2.  GuardRails LLM

Guide the customer to an optimal solution rather than allowing them to shoot themselves in the foot.  These guardrails can be trained by LLMs (product-specific LLMs running within the product - smaller LLM  footprint that acts as a well-trained product user), 

Do not allow the customers to install the new software if the storage or memory crunch is already in the system.

  3.  Customer pattern learner LLM

This LLM can sit in the product or run in the SaaS to understand the customer usage/use case and provide solutions to the customer.  This LLM can alert the customer if any anomaly is spotted. 

Customers using an older code version with a bug with a specific use case can be alerted to upgrade.  (Version recommender)

  4. Utilizing LLM for insights/analytics and file walker algorithms ( backup vendors and others browse files to identify patterns and can use GenAI tech for LLMs/Vector DBs).

Convert all the ML-based analytics to LLM-based analytics.

  5.  Prompt Engineering: Simplify the UI experience for the customer.  Current UX/UI can be used for advanced users.

Example: The prompt can be - "Identify current bottlenecks and suggest a solution"  

Because there were too many zombie processes, a CPU Bottleneck was identified.  Also, identify these processes and kill them.

Prompt Example for a backup software: Show me the current job that protects VMWare-SQL-Server-5  or Protect SQL-Server6 (provides the step or configures it automatically)

6.  If you are shipping Hardware with your product, LPU/GPU-ready hardware may be the future, or you can ship the call home data to GPU clusters in Amazon to run insights and analytics.

7. Better Product APIs to interact with LLMs: LLMs should be able to connect to the data source, log in to the product, and automatically change the product's configuration as per the prompt. This will help with AI-powered automation. There is a new development in this area called AI-APIs. AI APIs take things a step further by using machine learning and natural language processing to understand requests, generate relevant responses, and complete tasks.

Documents suitable for GenAI:

If the product vendor generates the product documents, they should be structured and parsed through AI.  Reinforced/supervised training of AI is good for verifying that it can produce clear, concise, and correct answers for a specific vendor software version without hallucinations and contradictions for the questions asked and can translate correctly in multiple languages.

LLMs are good at reading documents and summarizing them.  A quick test run of various prompts with LLMs trained on the new docs for every version/white paper will improve confidence.

LLMs for Proactive monitoring and self-healing:

Model 1: The product sends the call-home data to the SaaS-based vendor monitoring system.  Based on the above requirements of the logs/alerts being AI-ready, this data is AI-ready and will spend less time in the data cleaning/prep phase. 

Model 2: An on-prem Master that collects data from multiple points (IoTs, Clusters, nodes, Servers) and looks for anomalies. 

• Pros: Secure, Quick identification/Trained for local data—useful for Cameras to monitor a break-in; 
• Cons: Local Admin required and upgrades required, Limited processing power

LLMs can analyze these data for

• Identify the anomaly and the corresponding fingerprint, provide solutions or apply the solution if it exists. 
• Walk through the logs/alerts to identify new issues and alert the respective teams (Engineering/Field Notice/CSMs).  Create a draft doc on the field notice.
• LLMs can identify the blast radius of this fingerprint.
• LLMs can be live monitors of your product, and LLMs can be enabled to fix the issue or create docs or scripts to resolve the issue.
• LLMs can scan these logs much faster than the current log parsers/file walkers.

Examples:

  LLMs monitor the logs for FATAL failures and review whether this is a known issue or an unknown issue, triggering an appropriate action.

  LLMs monitoring the video input, start tracking the person who broke the glass based on glass shattering sound

Coming attractions in this blog space

Here is what you can expect in this space: I plan to write at least one blog monthly, if not more frequently. I appreciate your support in providing feedback and sharing this blog.

Let us learn together and make the world better with GenAI.

Index:

• GenAI fundamentals
• Data Preparation  - Desensitizing, Better Data to reduce Hallucinations
• Vector DB options
• LLM Models
• RAG
• Prompt Engineering

• Gen AI Apps
• Make GenAI work for CX
• Make GenAI work for the Serviceability of your product

• Review of Startups in this space

AI Applications - Improving Customer Experience

 Improving Customer Experience with AI (Work-in-progress)

Landscape:

Generative AI benefits all knowledge-based workflows. CX is a knowledge-based organization; the more your team knows about the product, the related products, customers, use cases, and sales cycle, the better they can support the customer. This refers to learning from multiple parameters, and with the increasing efficiency of LLMs and TPUs, Gen AI can do this better than humans.

Think of your support engineers as superheroes, constantly connecting the dots - creating, collecting, and making sense of information to solve customer problems. Gen AI can handle the heavy lifting of processing vast amounts of data, freeing your team to focus on the creative problem-solving and human connection that truly sets them apart.

Support Engineering Resources can be used to train these domain-specific Support LLMs (note a startup idea) and provide reinforced learning with human feedback(RLHF). New startups like ema unlimited, Amolina, Covea, Einstein by Salesforce, Workato, Gainsight, and AiSera provide point GenAI solutions for CX. But there is a much more enormous scope beyond these startups' narrow Support LLMs.  

Based on my extensive work with the CX team and talking with my peers in AI and Support Industry, I am open-sourcing my thought process on improving the CX organization with AI without being superficial.

I hope you find this helpful and you can implement some of it in your organization. Customers for the Win!!

Security Caution: Be aware of cross-training LLMs with multiple customer-specific PII data. This is a data breach. Also, numerous CVEs have various vector DB attacks where a malicious prompt could expose PII data. So follow best practices and use "data-cleaning" products like Amazon Glue, Granica, and BedRock to prepare your data for LLMs. Also, multi-level LLMs should be used to keep sensitive data private.

Interesting News Article:

Note that the savings mentioned here are for non-technical work at a finance company. There is value in Gen AI for "highly technical IT" support, but the benefits are in improving the lives of Customers and Support Engineers while reducing the time to resolve.

Benefits of introducing GenAI in CX 

What is our True North in customer-facing organization? 

• Our CX Team Members ( improve their quality of work/life and happiness in work)
• Customers ( gain value out of our product)
• Company/CX Vision and Mission (plus Margins)
• Be a good company ( reduce the carbon footprint, Do Good, Beneficial to the society)

Why do we need Gen AI in customer-facing org?

• 60% of the cases are similar and worked by different engineers, providing different answers and causing delays. GenAI can identify and provide solutions similar to your best engineer
• First Responses based on the customer query need more quality. How would my best engineer respond to this query - this can be easily replicated and provide the best quality responses
• CSM/TAM organizations spend a lot of time generating reports and understanding the reports (for example - analysis of a storage usage trend or lack of new feature uptake can be highlighted by Gen AI easily)
• these knowledge-based tasks can be automated

With this, let me detail some benefits that can be converted to a PRD and provided to your AI vendors like AISera/Support Logic/Movate/Coveo/SalesForce Einstein/ServiceNow Co-pilot.

Customer facing Benefits

• Delight in Self Service

No customer wakes up in the morning and opens a case, even if your support team is the best in the world. Next to being in the product  ( more on this in my next blog on Service-ai-ability), the Support Portal should be designed to provide meaningful summarized responses through chat or search. The prompt should be designed/engineered to follow the train of thought.

Real-world Example:

Customer: I got an alert that my backup jobs are failing 

AI: Can you provide me the alert so I can answer appropriately?

Customer: Error Code:+Alert

AI: Based on the error code, your backup on Server A failed due to a SOAP error. The following solutions in that order resolved this error. ( Solution 1: 60% cases resolved, Solution 2:25%, Solution 3: 20%, etc.)

Customer: Solution 2 resolved the issue.

AI can continue and gracefully end the conversation and update the solution rate. The customer is delighted that they can resolve this issue themselves. The customer will have a positive promoter attitude towards the vendor.

• Customer-aware Chat Responses (ChatBot) 

If LLMs can retain the context (incremental learning) of previous issues, chats, and customer sentiment and provide a chain-of-thought personalized response, customer satisfaction and retention will be the highest in the industry. With the advent of fast storage (VAST Data), faster CPUs (Nvidia/GPUs), and Vector DB to correlate and RAG/fine-tuning (to get the latest info about the customer), the multi-parameter LLMs can be trained to do this quicker.

Some customers access the support case form directly, so ensure it is redirected to GenAI.

Real-World Example:

Customer: I got an alert that my backup jobs are failing 

AI: Can you provide me the alert so I can answer appropriately? Also, note that you saw similar issues last week, but the previous week's solution (Solution 2) did not fix it permanently. So, based on the new training data, I see other customers saw this failure repeatedly. With that, I have a new solution for you. Do you want to try it?

Caution: Do not make the customer engage in endless chat conversations (like telephone bots asking to press a key and forever asking to press 1, press 2, press 3 and then press 1). Chat conversations taking more than 2 iterations should have the option to bypass and open the case with all the relevant conversation information. Keep it easier for the customers, and don't ask them to enter the same information again in the case description.

Always have a customer feedback button to review and fine-tune GenAI chatbots.

• Correlating the issue with existing solutions (identifying blast radius)

We covered this subject in the customer-aware chat responses. GenAI can create more correlations in the vector DB as it is training through JIRA/Confluence/KB/Emails/Slack and Cases. A good support engineer is always looking for correlations and connecting the dots, and GenAI is expected to do the same with an effective fingerprinting engine. 

With this training, GenAI can summarize the results and provide detailed steps to resolve the issue.

Based on analyzing the data from multiple customers, GenAI can predict if an issue is bubbling up to the top and proactively help the vendors send notifications to the impacted customers.

• Reducing the time to resolve and increasing the FRR (First Response Resolution) Rate

Suppose GenAI can identify and resolve the issue on the first contact. In that case, the customer can focus on other projects instead of sending back-and-forth emails or conference calls with the support engineers.

• Self Serviceable Product

Though it is not part of the product ( refer to the next blog on supportability), the customer can ask a few questions and resolve the issue themselves without spending time on calls.

CX Organization Benefits ( CX LLM)

Support Organization Benefits: (Support LLM)

Here, I will focus on building a Support LLM on all nuances for the Support team while collecting the data securely from ZenDesk/SFDC/JIRA/ServiceNow. Then, make sense of these data to analyze customer sentiment, provide the first response, translate to an appropriate language, fix grammatical errors, create the calendar events, send notes after the phone call, sort by relevance/priority/urgency, give the following steps, escalate to the management/SMEs, generate KBs, scripts, etc. AI could be a case watcher, curating the backlog while the engineer focuses on solving the complex technical issues.

That said, AI not only acts as a personal assistant, it can also play an active role in providing technical solutions. 

• Co-pilot with a meaningful first response

The Support Engineer should co-pilot with a meaningful response that can be sent to the customer based on the prior chat, case description, customer sentiment, and how your best support engineer would respond to the customer. The support engineer should review this response to ensure it is proper before sending it to the customer. Customers should never feel that they are receiving info from a new engineer.

If the support engineer's review is positive 90% of the time, this first response engine is fine-tuned enough to be sent to the customer without the review. Random Checks and balances are essential so that GenAI is not totally unsupervised.

It would 

• Customer Sentiment Analysis

GenAI can constantly monitor customer sentiment. If the sentiment is poor, then escalate those cases to the support management so they can connect with the customer to reduce the low CSAT score.

Some indicators are - frequent escalations, certain words - frustration, case resolution taking longer than SLAs, too many handovers, and lack of customer response.

Einstein from Salesforce and Support Logic Sentiment Analyzer are some excellent examples.

• Granular Intelligent Case Categorization

As LLM is trained on the case data, it can intelligently categorize a case into multiple categories. It will quickly generate reports for the relevant engineering and sales team and take appropriate action to reduce the number of incoming cases. Manual categorization is tedious and error-prone. 

Categories can include Features (SQL Backup, VMware backup), Internal Components (processes), and Verticals ( Federal/Finance/Education/Government).

• Ref: Reducing the time to resolve and increasing the FRR (First Response Resolution) Rate

Gen AI-assisted co-pilot can help resolve cases by

• providing summarized responses of how this issue seen with other customers was fixed by other Engineers
• recently seen issues will be automatically added to the summary by incremental learning done by RAG
• we can provide confidence in the summary so the support engineers can review and verify it before sending that to the customer
• Subsequent Responses are created automatically based on the meeting minutes from phone calls (using Fathom), updating the notes in the CRM tool/case tool, creating the responses and waiting in the backlog queue for that response to be approved.
• this summary could be added to a KB, docs, or field notice as confidence increases.
• Case Deflection 

GenAI summarizes various case notes, KBs, tech docs, white papers (via tasks-specific LLMs/SLM- Support language model), recent issues (via RAG) and external articles(generic LLMs) and provides the right solution to the customer. This helps the customer to resolve the problems themselves. GenAI results are better than Search Engine results.

60% of incoming cases are repeat cases that GenAI can better service and support than most support engineers.

• Level Up Support Engineers to train LLM (Semi-Supervised/RLHF)

To increase confidence in the GenAI answers, we need highly skilled Support Engineers who know the support domain and the product to run through various scenarios and act as QA engineers/RLHF. These Senior Engineers have a lot of tribal knowledge. 

• KB Generation based on Workarounds and Serviceability Tool Generation via Code LL (Code LLaMa)/Code Co-pilot (self-training)

We can provide a way to efficiently train the LLMs with draft resolution summaries/notes/brain dumps. In that case, GenAI can produce high-quality documents and identify/correlate the incoming support cases with this resolution.

In addition to writing documents, if there is a workflow, it can be automated through GenAI - Code generator. 

Example: applying config change by editing the .conf file and restarting a service. This can be scripted and given to the customer instead of the customer following multiple steps that can be easily missed.

• Customer Aware/Domain Aware - Translation Service for Replies and Conference Calls

Currently, Support Engineers use Google Translate or expensive translation services, neither of which is context/customer sentiment or product-aware. LLMs can be trained in both the product and structured language quickly(pre-trained models) and based on the customer sentiment, they can add polite words in the translation (for example, -san to address Japanese customers)

• Auto Case Updates based on intelligent note-taking during/after Zoom Calls and a Case Crawler for the subsequent case updates

If the case is support-pending, the Case crawler in AI logic can review the current case's past and similar issues to craft a draft for the subsequent response. The Support Engineer can review and send that response to the customer.

Again, creating the meeting notes after the phone call/zoom calls will reduce the support engineer's effort.

• Build a code-less escalation and handover processes.

Building code on top of Salesforce to create internal escalations for specific customers with stricter SLAs and email/Slack internal escalations based on SLAs requires a lot of coding in SFDC/Service Now and slows down these essential tools. With Gen AI, as LLMs can manage multiple parameters, all these conditions can be checked and alerted to management accordingly during case crawling.

Use GenAI to manage auto handover processes when certain conditions are met. (p1, customer replied, Support Engineer is not online)

• Build basic Kepner-Trego troubleshooting steps.

A Kepner-Tregoe matrix is used to find the causes of a problem. It isolates the who, what, when, where, and how aspects of an event, focusing on the elements that impact the event and eliminating the elements that do not. 

This logic can be included in GenAI, though logical troubleshooting is challenging to code. As GenAI approaches Human Intelligence and AGI, it is a good stretch goal for the newer AI systems.

• Build  and Update Dashboards - Improved Visibility

As GenAI - domain-specific LLMs with Case crawls, we can prompt questions like - build me a support dashboard with TTR, Aged cases more than 14 days and so on. LLMs are trained in support jargon and can read cases from support CRM tools like Salesforce, Zendesk, DevRev, and ServiceNow.

• Vendor Specific - Log Analysis and Fingerprinting Engine

A security-minded company would deploy generic and domain-specific LLMs to extract more specific data from the logs and case data.

  a) Internal Secure LLMs  -or-

  b) hashed Vector DB using RAG to send data to generic LLMs (delete the embeddings after summarization from LLMs)

Currently, every vendor deploys ML to analyze logs, determine the blast radius of an issue, or identify a unique pattern. LLMs can be deployed to ask for common patterns in a set of logs or identify a pattern and generate an action based on it. 

For example, if the error pattern in logs is A, and the software version is B, send out a Field Notice to upgrade them to version C.

• Field Notices to give heads up to the customer

We can ask GenAI to prettify the technical doc and notify all customers who match a certain version to upgrade to B.

• Calendar - Availability (bonus points)

GenAI can parse the case for the next action and then parse the calendar for the support engineer's availability. Then, send a calendar invite to the customer based on the availability of the support engineer. 

CSM/TAM Organization Benefits

• CSM/TAM can understand customers' sentiments, summarize their case reviews, and plan the next steps. However, browsing cases and CSAT scores take a long time to understand customer sentiment.
• We can request GenAI to prepare slides and RCA docs as CX domain-specific LLMs mature.
• Product/Vendor Specific LLMs that browse through the customer call home data, CSMs/TAMs can review and find hotspots like the old version, oversubscribed licenses, etc much more quickly
• The productivity and effectiveness of CSM/TAM delivery will be improved by GenAI doing most repetitive tasks, such as sentiment analysis, usage review, case reports, feature usage reports, review of how the peers/similar vertical customers have implemented, etc.
• CSMs/TAMs can focus on building relationships and upselling under having human interaction with their customers about their future plans for security, AI, and data analytics and what their peer teams are planning for similar workflows.
• GenAI can do most of the reactive work, while CSMs and TAMs can do the proactive work.

Professional Services Organization Benefits:

• GenAI can review the SoW and identify gotchas to be reviewed and corrected by PS practitioners/project managers.
• GenAI can help review the plan and find any hotspots
• Code Llama/Code review GenAI tools (Tab9, Github Co-pilot) can help with reviewing the migration scripts
• PS teams generally develop simple scripts and utilities that can be sped by code co-pilots
• PS develops a lot of training for the 3rd party vendors. GenAI can be used to deliver these trainings. Another use case is to update the training materials/video when there is a change due to a new software release
• To be seen: Plugin to Mavenlink/FinanceForce tools that help with PS planning.

 Security Considerations:

• Secure Responsible AI will ensure one customer's PII is not shared with another customer.
• Protection via good authorization and authentication at each layer of GenAI architecture—prompt, RAG, LLM, Data Cleaning, Data layer, and Vector DB—is important. All of these are prone to MITM attacks. The bouncer at each stage verifying the identity may be important.
• RAG reduces data storing in the general purpose LLM in the cloud.
• Review the GenAI infra's architecture to prevent data from being cross-pollinated to another customer.
• Confidential Computing, which provides the next protection level, has recently gained popularity. It is a cloud computing technology that protects data during processing. Exclusive control of encryption keys delivers stronger end-to-end data security in the cloud. Confidential computing technology isolates sensitive data in a protected CPU enclave during processing.