Many businesses wisely invest significant time and resources into creating content for their customers. From advertising to blogs, social media posts to support documentation, content plays a crucial role in engaging and nurturing prospects and customers throughout their journey.
Understandably, these companies now want to use all of that content in a wide variety of Gen-AI use-cases, particularly in specialized on-brand chat experiences for their prospects and customers. Unfortunately we believe that the most widely discussed approach to using existing content for this purpose is doomed to fail.
I’m talking about how people are trying to use content today in Recall Augmented Generation, or Content-RAG for short, in which old content is chopped up into pieces which are then retrieved as needed to answer incoming questions.
"Now is the time for businesses to shift their focus from content creation to knowledge management."
The Limitations of Content driven Recall Augmented Generations (Content-RAG)
Here is one big reason that Content-RAG fails: say you have a great library of blog posts but parts of many of those posts are no longer valid, maybe because of some changes in the law or market conditions. Unfortunately, all of that incorrect information is inseparable from the good information when it is used directly to support chat experiences or other Gen-AI applications.
You could rewrite your old content, which is hard work, or you could remove any posts or other content with invalid information, but that’s like throwing the baby out with the bathwater.
"We have found a better way for our clients: manage the knowledge, generate the content!"
First, invest in capturing and managing knowledge rather than creating content. Then, use Gen-AI to create or co-create content, including chat experiences, from that knowledge. That’s what I mean when I say “content is dead, long live knowledge”.
You still need all that content for the customer journey but such content should be generated or updated automatically as needed. Knowledge is timeless and is the true core intellectual property of your company. So what do we mean by knowledge? In many cases, it can be as simple a collection of questions and answers!
A Better Way
Here is how we are making this work for our clients. Rather than storing and retrieving chunks of old content in response to questions, we are helping our clients create and manage a table of “Known Good Answers” (KGAs). Each KGA includes a question, an answer, and metadata such as source URL, status, date of last update, etc. We then store those KGAs as documents for recall in response to user questions. We call this Knowledge-RAG.
This is still a RAG-like solution but we are using easy-to-manage KGAs as the source of truth instead of bits and pieces of old content that are impossible to curate. That’s one big reason why Knowledge-RAG is better than Content-RAG.
"In the Gen-AI era, investing in knowledge has a far greater return than investing in content"
Creating, Managing and Using KGAs
Remember all that old content? It still has value. Use it to generate candidate KGAs then forget about it! Going forward simply invest in maintaining a great KGA database and use Gen-AI for new content.
For one financial services provider, we extracted 1500 candidate KGAs from their historical blog posts. They narrowed that down to about 400 KGAs that they thought would answer 99% of all inbound prospects questions in a chat experience. This is something they can easily curate and manage going forward.
We are helping another client use AI to generate KGA candidates from public sources of information. When there is news in their field they want to use, we leverage Knowledge-RAG to find the existing KGAs that are impacted by that news, and to suggest rewrites of the answers!
When our client wants to generate a customer report or some other content, our applications find relevant KGAs and submit them to a Gen-AI tool with the appropriate prompt. How easy would it be to use Gen-AI to rewrite a great old blog post but corrected with updated knowledge!
Viola…on-demand content from fresh, curated, proprietary knowledge. Knowledge comes first, not content.
Take Action
Don’t wait. Start building your knowledge base and using it to generate content. If you aren’t sure how to go about it, reach out and get in touch!
###
Content-RAG
A Content-RAG chat experience works like this: First, the content is chopped into “chunks” (yes that is the technical term) and ingested into a specialized “vector” database that stores the chunks based on information they contain (yes, it’s complicated and amazing).
Then, when a customer poses a question, the vector database tries to find the chunks with information that is similar to the question using the idea of “semantic closeness”.
The few closest chunks are then given to an LLM, like ChatGPT or Bard, along with the original user’s question and a prompt that says something like “try to answer this question using those chunks”. There are several problems with this approach
- Chunks are relatively large so only a few of the close matches can be given to the LLM to generate a final answer. What if the best answer was contained in the one chunk that got left out?
- The source material may become progressively more and more out of date, which means that the chat experience will continue to rely on increasingly incorrect information or you have to throw out the entire original content
- The idea that semantic closeness in a vector database is a good way to match questions with chunks of text that may contain answers, seems suspect. It seems to work in many cases but it is not perfectly reliable. Others have also reported on this as a potential mis-use of the vector database technology.
When you hear that it is easy to build a great demo with Content-RAG based chat, but hard to deliver a reliable customer ready solution, those are some of the big reasons why.
###
Knowledge-RAG
Instead of searching for “chunks” like in Content-RAG, when a user question comes in, we search for KGAs where the question part is a close match to the user question, we then pass the KGAs (questions and answers) that we find into the LLM along with the original question to produce a final response to the user.
This is still Recall Augmented Generation but it is using KGAs instead of chunks of old content. Here are a few reasons why we expect Knowledge-RAG to be better than Content-RAG:
- KGAs are short, simple and easy to curate. When something changes in the world, just invalidate or update those entries, probably using Gen-AI to search for impacted KGAs.
- KGAs are much smaller than “chunks” so you can feed a large number of closely matching KGAs to an LLM which can then apply the full context of the dialog to select the best answers and produce an accurate and context-appropriate answer for the user
- Questions are much more likely to be semantically close to rephrasings of the same questions than they are to chunks that have part of an answer embedded somewhere in the text. Searching for similar questions to a user question is better aligned with the true nature of the vector search technology.
###
About Larry Arnstein
As the CTO of Simply Augmented, Larry brings broad technical and business experience to the company. Larry joined Simply Augmented after a long entrepreneurial career that included helping lead Impinj to IPO in 2016 as a member of the executive leadership team, followed by the acquisition of Xnor.ai by Apple in 2019 where he was head of market and product strategy. Larry then left Apple to become the CTO and co-founder of AirTerra, a retail supply chain solution provider that was acquired by American Eagle Outfitters in 2021.
About Simply Augmented
Simply Augmented is a leading provider of AI-driven workflow solutions aimed at enabling businesses to optimize the key operational aspects that bring value to their clients. With Simply Augmented, you get more than just an AI service provider; you get a partner committed to your business’s success. We strive to make AI accessible, understandable, and beneficial for all businesses, regardless of size or industry.
Our AI development team is ready to help your business at every step. We’ll guide you from planning to launch, and from training to support. Whether it’s integrating process automation or conversational AI, you can think of us as a dedicated AI partner.
