Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
By
As AI adoption accelerated across industries, deploying a chatbot to support a newly launched internal application seemed like a logical decision. However, the application itself challenged conventional user expectations. It introduced new workflows built on emerging technology unfamiliar to most users.
To reduce friction and improve adoption, the chatbot was designed to answer questions about the application and the underlying technology. The goal was to help users understand not only what to do, but also why the system behaved the way it did. We believed that providing contextual explanations would accelerate learning and reduce confusion.
From the beginning, the AI agent was conceived as a limited-scope solution. It was designed strictly to support documentation and provide user assistance. Conceptually, the chatbot was intended to serve as a dynamic replacement for a traditional Frequently Asked Questions document, offering a conversational, searchable and continuously available interface with extended functionality beyond static content.
To integrate the agent into the organization’s internal chat environment, we needed to understand how structured messages were rendered, how conversation history was stored and how the system identified participants within threads. This allowed us to determine the core variables required to begin processing user questions.
Large language models are powerful, but without contextual anchoring they are prone to hallucinations. To address this, we implemented a vector embedding technique.
User guides, internal documentation and the product vision were transformed into numerical vector representations of text. These embeddings captured semantic meaning, allowing the system to match concepts rather than rely on simple keyword matching.
When a user asked a question, the system converted the query into a vector representation and compared it to the stored embeddings. It retrieved the most semantically relevant documents and injected them into the model’s prompt. The model then generated a response grounded in those specific documents, often summarizing the relevant information.
This approach significantly improved response accuracy. Instead of generating answers based purely on general knowledge, the model responded using our organization’s own documentation as context.
It was essential to include conversation history in the prompt so the bot could interpret follow-up questions and maintain continuity. Without history, interactions became fragmented and repetitive. Users often refine their questions incrementally, and without context, the bot could not interpret references like “that option” or “the previous step.”
However, including too much history created a different problem: token limits. These occur when language models truncate inputs that exceed their maximum context window. If a question or conversation became too long, important information could be lost. This did not produce an explicit error, but rather degraded response quality or affected retrieval accuracy.
To mitigate this, we implemented strategies to control prompt size, prioritize relevant content and monitor question length. We experimented with summarizing older messages and selectively including only the most relevant parts of the conversation. Context was critical but it had to be carefully managed.
Beyond answering documentation-based questions, we extended the bot’s capabilities by adding backend functions that could extract certain public information directly from the application. This allowed users to retrieve data from the chat without logging into the app itself. The idea was to reduce friction and reinforce the chatbot as a useful interface, not only a static knowledge layer.
This extension created confusion for some users though. Once the bot began retrieving live data, users started asking it to execute actions that required direct interaction inside the platform. They assumed the chatbot could replace operational steps, including those requiring authentication or deliberate execution inside the platform.
The bot was never designed to perform those actions, but the distinction between informational assistance and operational execution was not always clear.
Integrating live data also introduced new technical considerations. We needed to define when a question should go through embedding-based retrieval and when it should trigger a backend call. That decision logic required careful design. Additionally, we had to tune responses to gracefully handle technical exceptions and avoid exposing raw system errors to users.
During testing, we realized that the bot consistently performed better in English than in other languages used within Jalasoft. The primary reason was structural: most of the documentation used to generate embeddings was written in English, and the embedding model we selected was optimized for English semantic similarity.
It did not support cross-lingual retrieval or semantic comparison across languages. As a result, non-English queries often retrieved less relevant documentation, leading to weaker responses.
This highlighted an important insight: multilingual capability is not automatic.
To control usage costs, we implemented a daily limit on the number of questions users could ask. However, we did not explicitly restrict the scope of those questions. Users were free to ask anything.
This openness led to unexpected usage patterns. Some users began interacting with the bot for personal or exploratory purposes unrelated to the application. Over time, expectations outgrew the bot’s intended role, creating a gap between what users hoped it could do and what it was designed to support.
This misalignment gradually reduced its perceived usefulness. Usage declined, and the chatbot was eventually deprecated, with efforts redirected toward redesigning the application itself to make it more intuitive and easier to use.
From an engineering standpoint, the system worked reasonably well. It retrieved documentation, incorporated conversation history, reduced hallucinations through embeddings, handled backend calls, and managed prompt size. The architecture functioned as intended.
But it lacked intentional interaction design.
The bot did not clearly shape conversations. It did not consistently reinforce its scope. It did not guide users with structured examples of what it could and could not do. It answered questions, but it did not set expectations.
We learned that conversational AI systems require more than strong models and structured data. They require carefully designed expectations. Users need clarity about the agent’s role, its boundaries and its strengths. The system must proactively provide example prompts, clarify limitations and redirect out-of-scope questions consistently.
Without this intentional framing, even a technically sound implementation can struggle to sustain value. Users may overestimate capabilities or disengage when unstated expectations are not met.
The core insight is simple but powerful.
Strong context, accurate retrieval and robust architecture are necessary, but not sufficient. The effectiveness of the system depends equally on how it defines its role, communicates its boundaries and shapes user expectations.
Technology alone does not guarantee adoption. Clear interaction design does.
Angie Navia is a Full-Stack Developer at Jalasoft with five years of experience building production applications and integrating AI capabilities into software solutions. She completed the IBM Generative AI for Software Developers Specialization and applies AI tools in her daily development workflow.
Your Agent Isn’t Just a Chatbot Anymore—So Why Are You Still Treating It Like One?
LLM-as-a-Judge: A Scalable Solution for Evaluating Language Models Using Language Models
Exploring Amazon’s New AI Chatbot: How Rufus is Changing the Shopping Experience
Will Meta’s Bold Move Towards User-Created Chatbots Work?
Why Do AI Chatbots Hallucinate? Exploring the Science
Researcher Develops Domain-Specific Scientific Chatbot
Advertiser Disclosure: Unite.AI is committed to rigorous editorial standards to provide our readers with accurate information and news. We may receive compensation when you click on links to products we reviewed.
Copyright © 2026 Unite.AI
