Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Publish AI, ML & data-science insights to a global community of data professionals.
How to create a chatbot to answer questions about file’s content
AI is everywhere.
It is hard not to interact at least once a day with a Large Language Model (LLM). The chatbots are here to stay. They’re in your apps, they help you write better, they compose emails, they read emails…well, they do a lot.
And I don’t think that that is bad. In fact, my opinion is the other way – at least so far. I defend and advocate for the use of AI in our daily lives because, let’s agree, it makes everything much easier.
I don’t have to spend time double-reading a document to find punctuation problems or type. AI does that for me. I don’t waste time writing that follow-up email every single Monday. AI does that for me. I don’t need to read a huge and boring contract when I have an AI to summarize the main takeaways and action points to me!
Learn this step by step with the interactive AI Engineer roadmap.
These are only some of AI's great uses. If you’d like to know more use cases of LLMs to make our lives easier, I wrote a whole book about them.
Now, thinking as a data scientist and looking at the technical side, not everything is that bright and shiny.
LLMs are great for several general use cases that apply to anyone or any company. For example, coding, summarizing, or answering questions about general content created until the training cutoff date. However, when it comes to specific business applications, for a single purpose, or something new that didn’t make the cutoff date, that is when the models won’t be that useful if used out-of-the-box – meaning, they will not know the answer. Thus, it will need adjustments.
Training an LLM model can take months and millions of dollars. What is even worse is that if we don’t adjust and tune the model to our purpose, there will be unsatisfactory results or hallucinations (when the model's response doesn't make sense given our query).
So what is the solution, then? Spending a lot of money retraining the model to include our data?
Not really. That’s when the Retrieval-Augmented Generation (RAG) becomes useful.
RAG is a framework that combines getting information from an external knowledge base with large language models (LLMs). It helps AI models produce more accurate and relevant responses.
Let’s learn more about RAG next.
Let me tell you a story to illustrate the concept.
I love movies. For some time in the past, I knew which movies were competing for the best movie category at the Oscars or the best actors and actresses. And I would certainly know which ones got the statue for that year. But now I am all rusty on that subject. If you asked me who was competing, I would not know. And even if I tried to answer you, I would give you a weak response.
So, to provide you with a quality response, I will do what everybody else does: search for the information online, obtain it, and then give it to you. What I just did is the same idea as the RAG: I obtained data from an external database to give you an answer.
When we enhance the LLM with a content store where it can go and retrieve data to augment (increase) its knowledge base, that is the RAG framework in action.
RAG is like creating a content store where the model can enhance its knowledge and respond more accurately.
Summarizing:
Uses search algorithms to query external data sources, such as databases, knowledge bases, and web pages.
Pre-processes the retrieved information.
Incorporates the pre-processed information into the LLM.
Now that we know what the RAG framework is let's understand why we should be using it.
Here are some of the benefits:
Enhances factual accuracy by referencing real data.
RAG can help LLMs process and consolidate knowledge to create more relevant answers
RAG can help LLMs access additional knowledge bases, such as internal organizational data
RAG can help LLMs create more accurate domain-specific content
RAG can help reduce knowledge gaps and AI hallucination
As previously explained, I like to say that with the RAG framework, we are giving an internal search engine for the content we want it to add to the knowledge base.
Well. All of that is very interesting. But let’s see an application of RAG. We will learn how to create an AI-powered PDF Reader Assistant.
This is an application that allows users to upload a PDF document and ask questions about its content using AI-powered natural language processing (NLP) tools.
The app uses Streamlit as the front end.Langchain, OpenAI's GPT-4 model, and FAISS (Facebook AI Similarity Search) for document retrieval and question answering in the backend.
Let’s break down the steps for better understanding:
Loading a PDF file and splitting it into chunks of text.
This makes the data optimized for retrieval
Present the chunks to an embedding tool.
Embeddings are numerical vector representations of data used to capture relationships, similarities, and meanings in a way that machines can understand. They are widely used in Natural Language Processing (NLP), recommender systems, and search engines.
Next, we put those chunks of text and embeddings in the same DB for retrieval.
Finally, we make it available to the LLM.
Preparing a content store for the LLM will take some steps, as we just saw. So, let’s start by creating a function that can load a file and split it into text chunks for efficient retrieval.
Next, we will start building our Streamlit app, and we’ll use that function in the next script.
We will begin importing the necessary modules in Python. Most of those will come from the langchain packages.FAISS is used for document retrieval; OpenAIEmbeddings transforms the text chunks into numerical scores for better similarity calculation by the LLM; ChatOpenAI is what enables us to interact with the OpenAI API; create_retrieval_chain is what actually the RAG does, retrieving and augmenting the LLM with that data; create_stuff_documents_chain glues the model and the ChatPromptTemplate.
Note: You will need to generate an OpenAI Key to be able to run this script. If it’s the first time you’re creating your account, you get some free credits. But if you have it for some time, it is possible that you will have to add 5 dollars in credits to be able to access OpenAI’s API. An option is using Hugging Face’s Embedding.
This first code snippet will create the App title, create a box for file upload, and prepare the file to be added to the load_document() function.
Machines understand numbers better than text, so in the end, we will have to provide the model with a database of numbers that it can compare and check for similarity when performing a query. That’s where the embeddings will be useful to create the vector_db, in this next piece of code.
Next, we create a retriever object to navigate in the vector_db.
Then, we will create the system_prompt, which is a set of instructions to the LLM on how to answer, and we will create a prompt template, preparing it to be added to the model once we get the input from the user.
Moving on, we create the core of the RAG framework, pasting together the retriever object and the prompt. This object adds relevant documents from a data source (e.g., a vector database) and makes it ready to be processed using an LLM to generate a response.
Finally, we create the variable question for the user input. If this question box is filled with a query, we pass it to the chain, which calls the LLM to process and return the response, which will be printed on the app’s screen.
Here is a screenshot of the result.
And this is a GIF for you to see the File Reader AI Assistant in action!
In this project, we learned what the RAG framework is and how it helps the LLM to perform better and also perform well with specific knowledge.
AI can be powered with knowledge from an instruction manual, databases from a company, some finance files, or contracts, and then become fine-tuned to respond accurately to domain-specific content queries. The knowledge base is augmented with a content store.
To recap, this is how the framework works:
1️⃣ User Query → Input text is received.
2️⃣ Retrieve Relevant Documents → Searches a knowledge base (e.g., a database, vector store).
3️⃣ Augment Context → Retrieved documents are added to the input.
4️⃣ Generate Response → An LLM processes the combined input and produces an answer.
https://github.com/gurezende/Basic-Rag
If you liked this content and want to learn more about my work, here is my website, where you can also find all my contacts.
https://gustavorsantos.me
https://cloud.google.com/use-cases/retrieval-augmented-generation
https://www.ibm.com/think/topics/retrieval-augmented-generation
https://youtu.be/T-D1OfcDW1M?si=G0UWfH5-wZnMu0nw
https://python.langchain.com/docs/introduction
https://www.geeksforgeeks.org/how-to-get-your-own-openai-api-key
Written By
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Enterprise Document Intelligence [Vol.1 #B1] – Three sources of one problem. User typos, fast-typing transcription noise, OCR character errors. Classical spell-check handles one of them. Embeddings carry the rest
It didn't turn out as expected.
Boost your Colab Notebook's GPU
Easily segment your data into training, validation, and test sets
Conformal Prediction, LLMs and HuggingFace - Part 1
A practical guide to connecting and coordinating multiple AI agents
Seeking Advice from a Panel of Specialists
How to build from scratch a recommender and boost its accuracy while keeping it simple
Controlling the Temperature Control Lab with an LSTM
Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.
