Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Latest news
Editions
Picking the best AI chatbot from a ranked list means borrowing someone else’s prompts and someone else’s taste. This tutorial builds the alternative: a blind arena where you and your teammates vote on anonymous answers to your own questions. The lineup is the current set of flagships as of November 9, 2026. It covers OpenAI’s GPT-6 Sol (the GPT-6 family also includes a lighter Luna model), Anthropic’s Claude Opus 5.5, xAI’s Grok 4.7, and Google’s Gemini 4 Argon, which reached developers, enterprises and eligible Google AI subscribers on October 13, 2026.
Major model refreshes were released across all four leading chatbots, including GPT-6 Sol and Luna, Claude Opus 5.5, Gemini 4 Argon, and Grok 4.7; no verified officially announced unreleased flagship entrant was found.
Industry insights, the latest tech news, and special interviews, all in the Tech Insider Newsletter.
One email a week. No spam, unsubscribe anytime.
Last updated 09 November 2026: prices, availability and picks re-checked.
The finished project is a Flask app, a SQLite file and a ranking script, about 500 lines of Python in all. Budget 60 minutes: 15 for setup, 10 for collecting answers, 30 for voting and 5 for scoring. If you want a quick opinion first, our ranked list of the best AI chatbots covers the general picks. This guide tests whether those picks survive contact with your own work.
Automated LLM evaluation scores answers with code or with a judge model. A blind vote measures something different, which is the reply a person would rather read. Most chatbot use ends exactly there, with a human deciding whether the answer helped. That makes the vote a useful partner for the automated checks, and the one signal you cannot get from a leaderboard.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
Before the build, here is the buying picture as of November 9, 2026. All four chatbots below have a free entry point, all four sell a roughly 20 dollar paid tier (Grok starts lower), and all four now charge between 100 and 250 dollars a month for heavy use. The differences that matter sit in what each flagship does best and which ecosystem you already live in.
Treat the table as a shortlist, not a verdict. Prices and plan limits move often, and our map of which plan unlocks the newest model tracks the changes. The sections below cover each chatbot in turn, and the rest of the guide shows how to test them on your own work.
ChatGPT remains the default answer for people who want one subscription to cover everything. It handles multimodal chat, web research, coding, image generation, agent tasks and office integrations in a single product. The current lineup is built around GPT-6 Sol and GPT-6 Luna, and the breadth of the surrounding feature set is the main reason it tops many shortlists.
Pricing. The Free tier costs nothing and is the right place to start. Plus costs 20 dollars a month and unlocks the newest model for most individual users. The Pro tier comes in three steps at 100, 200 and 500 dollars a month, aimed at people who hit usage limits daily, such as full-time developers and analysts. OpenAI’s pricing page lists Sol as unavailable on Free, available on Plus and available with expanded access on Pro, so the plan you buy decides which model answers.
Who should pick it. Choose ChatGPT if you want one tool for writing, research, images and light automation, or if your team already uses its office integrations. Skip the top Pro tiers unless you can show, with logged usage, that Plus limits slow you down. A blind test like the one in this guide is the cheapest way to check whether the upgrade earns its price.
Watch for. Free accounts do not get the flagship, so judging ChatGPT from a free session understates it. The reverse also holds: a Sol answer through the API tells you what the model can do, not what every plan serves.
Claude is the pick for long-context text, careful analysis and coding. Claude Opus 5.5 is especially strong for agentic coding and knowledge work, and Anthropic says it costs less to run than its predecessor. Claude Code, the command-line coding agent, and the wider set of agentic workflows are the main reasons developers choose it over the alternatives.
Pricing. Free gives you a taste of the product. Pro costs 20 dollars a month. Max comes in two steps, 100 and 200 dollars a month, for people who run long agentic sessions and would otherwise hit limits. Anthropic’s release notes list Opus 5.5 across its paid plans and the API, so you do not need the most expensive tier just to reach the flagship.
Who should pick it. Choose Claude if your work is document-heavy, code-heavy, or built around multi-step agents. Writers who like a measured tone also tend to favor it, though the blind vote in this guide is the right way to confirm that for your own prompts rather than trusting a reputation.
Watch for. Opus 5.5 no longer runs with thinking switched off, so every answer carries thinking tokens. That shows up in latency and in API cost, and it is worth knowing before you compare it against faster models on a simple question.
Gemini 4 Argon is now generally available, and it makes Gemini a leading general-purpose competitor. It combines multimodal reasoning, long context, image and video capabilities, and the deepest integration with Google Search and Workspace of any chatbot. If your documents, mail and calendar already live in Google, that integration is the feature that matters most.
Pricing. The basic tier is free. Google AI Pro costs 19.99 dollars a month, and Google AI Ultra costs 249.99 dollars a month, the highest entry fee among the four. In the consumer Gemini app, Argon access is limited to Google AI Pro and Ultra subscribers, so the free tier will not show you the flagship. On the API side, the introductory $2 and $10 rates run through November 30, 2026, and the standard $4 and $20 rates apply after that.
Who should pick it. Choose Gemini if you want answers grounded in Google Search, summaries of Drive files and mail, or image and video generation in the same app. For Workspace teams the paid tier is often the cheapest way to add a chatbot to tools people already use.
Watch for. Reports say the API sits behind the paid tier in AI Studio, so a free key may return an access error. The Ultra price is steep, so confirm that AI Pro limits are actually a problem before you jump.
Grok 4.7 remains a competitive fourth major chatbot. Its edge is real-time web and X context, which makes it useful for current events and anything driven by social-platform conversation. It also offers multimodal chat, coding and long context, and the API exposes a 500,000-token window with configurable reasoning effort from low to xhigh.
Pricing. Consumer access runs through X subscriptions that range from about 8 to 200 dollars a month, and the SuperGrok tiers vary, so check the current plan page before you pay. The API price is $2 and $6 per million tokens under 200,000 prompt tokens, and $4 and $12 above that, which makes it the cheapest of the four flagships to call.
Who should pick it. Choose Grok if you follow fast-moving news, track conversations on X, or want a low-cost API model for coding and long documents. It is the least likely of the four to be someone’s only chatbot, but it earns a place as a second opinion.
Watch for. Grok 4.7 Fast is limited to Cursor and Grok Build and cannot be called through the public API. Plan names and prices differ between the chat apps and the API, so do not assume one tracks the other.
No single flagship wins every job, so match the chatbot to the work rather than the headline. Use these rules as a starting point and then verify them with the arena below.
The honest caveat is that the gap between the four flagships is smaller than the gap between a well-written prompt and a lazy one. A leaderboard cannot see your prompts, which is why the rest of this guide builds a test you control.
Public leaderboards answer a population question. They tell you which model a crowd prefers across millions of prompts you will never see. Your question is narrower: which model handles your email drafts, your SQL reviews and your meeting summaries. A crowd average can hide a model that wins your tasks and loses everyone else’s. Our breakdown of how LMArena and Artificial Analysis score models explains why the two leaderboard styles often disagree.
The arena borrows the crowd method and shrinks it to your scale. Each rater sees one prompt and two anonymous answers. They pick a winner, call a tie, or mark both as bad, then move on. Behind the scenes the app randomizes which model sits on the left, scrubs vendor names from the text, and never sends model identities to the browser. It uses the same logic as the AI image generator arena we built earlier, so the ranking code will look familiar if you tried that one.
Votes become a ranking through the Bradley-Terry model, a standard way to turn pairwise wins into one strength score per contender. We express that score on the familiar Elo scale. Unlike a running Elo update, Bradley-Terry ignores the order in which votes arrived, which matters when three people vote in parallel. A bootstrap then resamples your votes 300 times and gives every score a 95 percent interval. You can see at a glance whether two models are separated or tied by luck.
The method has one hard limit. A vote captures preference, not correctness, so a confident wrong answer can beat a hesitant right one. The fix costs almost nothing. Seed the prompt set with problems that have known answers and hand raters a cheat sheet, as step 10 shows.
You need a laptop, four API keys, and one or two other people willing to click for half an hour. The code uses the Python standard library plus three packages.
Each vendor bills separately, so confirm you have credit on all four accounts before step 7. Reports say the Gemini 4 Argon API sits behind the paid tier in AI Studio, so a free-tier key may return an access error. The endpoints in this tutorial are the vendors’ long-standing public API shapes. Vendors rename parameters between model generations, so check each API reference if a call fails. The troubleshooting table near the end covers the usual errors.
The table lists the newest model in each vendor’s flagship line (OpenAI’s lighter GPT-6 Luna is left out of the main run, and makes a natural fifth contender) as of November 9, 2026, plus the API identifier you will paste into the config file. Every vendor shipped its entry within about three weeks of the others, which is why a fresh test beats any list written last quarter.
OpenAI. The consumer name is GPT-6 Sol, and the API identifier gpt-6.1-sol is the newest Sol model at the time of writing; confirm it in OpenAI’s model list before you run. OpenAI introduced it as offering “near-Astra intelligence” for coding, computer use and professional work at one-fifth of GPT-6 Astra’s API prices. Its release notes describe the rollout from Pro to Plus, Business, Enterprise and Edu plans, and OpenAI’s help center lists a 500,000-token context window in ChatGPT Work and Codex. OpenAI’s pricing page lists Sol as unavailable on Free and Go, available on Plus, and available with expanded access on Pro.
Anthropic. Claude Opus 5.5 is the first model in the Claude 5.5 family. Anthropic says it costs 40 percent less to run than Opus 5 on typical workloads, and its release notes list availability across Pro, Max, Ultra, Team and Enterprise plans plus the API. One quirk matters for a fair test. Anthropic’s announcement says Opus 5.5 no longer runs with thinking switched off, so its answers always carry thinking tokens, and that shows up in latency and cost. Anthropic’s pages also describe Claude Sonnet 5.5 and Claude Haiku 5.5 as members of the same family, which makes them natural cheaper contenders later.
Google. Gemini 4 Argon uses the identifier gemini-4-argon and offers a 1-million-token context window according to Google’s developer material. The $2 and $10 rates are introductory through November 30, 2026. Lindy’s pricing summary covers the same rates, and Google’s own note says the standard $4 and $20 rates apply after that. In the consumer Gemini app, Argon access is limited to Google AI Pro and Ultra subscribers.
xAI. Grok 4.7 has been in the public API as grok-4.7 since September 21, 2026, with a 500,000-token context window and configurable reasoning effort from low to xhigh. A faster variant, Grok 4.7 Fast, is limited to Cursor and Grok Build and cannot be called through the public API, so leave it out of the config. Plan names and prices for the chat apps differ from API pricing, and our map of which plan unlocks the newest model tracks them.
The arena tests API models, not the chat apps. That gives the cleanest comparison, because app defaults depend on your plan and the vendor’s routing. If you rate chatbots mostly through the apps, paste a few real app answers into the database by hand (the FAQ shows how) and treat the API results as a baseline.
Open a notes file and list the four to six jobs you gave a chatbot over the last two weeks. Typical entries are drafting, code review, reasoning, summarizing, personal advice and explaining a concept. Write three prompts per job, which gives 18 prompts and 108 possible matchups (18 prompts times six model pairs). Add one line per prompt saying what a good answer contains. That cheat sheet is your defense against confident nonsense winning a vote.
Pick raters now, too. Three works well. If you write the prompts and also cast every vote, you will favor the style you expected, so recruit at least one person who has not seen the prompt list.
Run every later command from inside the arena folder with the environment active. The prompt should start with (.venv).
Create .env.example as a template, copy it to .env, and paste your real keys into the copy. Generate the session secret with the one-liner below so cookies survive restarts.
The .gitignore line keeps keys and your raw votes out of version control. Never paste keys into the Python files.
The registry holds one frozen record per contender: a short key for the database, the provider, the API identifier and the two prices. Prices feed the cost report in step 12. Update them when vendors change rates, and expect an edit once the introductory period ends on November 30, 2026, when the Gemini 4 Argon rates step from $2 and $10 to $4 and $20. Confirm the OpenAI row on OpenAI’s pricing page before you trust any cost figure.
Each prompt gets an ID, a category and plain text. Keep prompts self-contained so a rater can judge them without opening another file. Some of the reasoning prompts have exact answers you can check: the ball costs five cents, and the three meetings fit because they need 135 minutes plus two 15-minute breaks, or 165 of the 180 available.
Swap in your own prompts as soon as the pipeline works. If your real use is character chat or fiction, borrow prompt ideas from our roleplay chatbot test. Avoid questions about recent events, because models with different knowledge cutoffs will split on facts, and a rater cannot verify them in five seconds.
Each adapter sends one prompt and returns the text plus input and output token counts. OpenAI goes through the Responses API, Anthropic through the Messages API, Google through generateContent, and xAI through its chat completions endpoint. The Google adapter drops thought parts and counts thinking tokens as output, because you pay for them. The shared ask function retries rate limits and server errors with exponential backoff and fails fast on other client errors, printing the vendor’s message.
The adapters set no temperature and no reasoning effort. Reasoning models often fix those values, and the goal is to test each product as the vendor ships it. If you later want matched effort, the advanced tips cover it.
The generator runs every missing prompt-and-model pair, four at a time, and writes each answer to SQLite the moment it arrives. It is resumable, so a crash or a rate-limit storm costs you nothing. Run it, then rerun it until the header reports zero answers to collect. If you already operate one endpoint in front of several providers, the LLM gateway tutorial shows how to point all four adapters at it instead.
Expect output like the block below on your first run. The timings and costs are placeholders and yours will differ.
A FAIL line is harmless. Rerun the script and it fills the gaps.
The schema has two tables. answers holds one row per prompt and model, and votes holds one row per click, including how many seconds the rater took. A CHECK constraint limits winner to a, b, tie or bad, so a bug in the app cannot write garbage.
Models love to introduce themselves. A reply that starts with a vendor’s name ends the blindness in one glance, so scrub.py swaps vendor and model names for a neutral token. The trade-off is that legitimate mentions get scrubbed too, which is fine for a prompt set that does not discuss these companies.
The app picks the matchup with the fewest votes so far, skips pairs the current rater already judged, flips a coin for left and right, and stores the pending matchup in the signed session cookie. The vote form posts only the choice. If the model names rode along in hidden form fields, any curious rater could read them with view-source. The table below shows how each control maps to a bias.
Start it with python app.py and open http://127.0.0.1:5000. To let teammates on your network vote, run flask --app app run --host 0.0.0.0 instead, and only on a network you trust, because the page has no login. If you would rather build the front end in React, our Vercel AI SDK chatbot tutorial gives you a Next.js starting point, and the vote endpoint stays the same.
Give each rater the same one-minute briefing and the cheat sheet from step 1. Raters should follow these rules:
Each rater opens the page, types a first name, and votes until they run out of time or matchups. Plan on 20 to 30 seconds per vote, which puts 54 votes at roughly 20 to 25 minutes. Spread across a few raters, that gives each model pair only a modest number of votes. That sounds like plenty and it is not. With so few decisive votes, even a clear-looking win rate carries a wide margin either way. So close pairs will stay unresolved, and that is an honest result. Teams comparing chatbots for literature work can pair this step with our look at research chatbots and lab seats to decide who gets which account.
Do the first five votes yourself and watch for problems. Both answers should look like plain text with no vendor names. If a reply still announces its maker, add the phrase to the NAMES list in scrub.py and restart.
The script does six jobs. It drops speed-clicks under eight seconds. It fits a Bradley-Terry model, with one pseudo-tie per pair so a model with zero wins does not collapse to zero strength. It bootstraps 95 percent intervals. It builds a head-to-head matrix and per-category scores. It checks for length bias, and it prices each preferred answer. Both bad votes are counted but excluded from the strength fit, since they say nothing about which answer is better.
Test the whole pipeline before any human votes by running the built-in demo. It seeds an in-memory database with synthetic votes from three raters, using four placeholder models with hidden strengths. Model alpha is strongest overall and writes the longest answers, and model beta gets a large boost on coding prompts. If the ranking recovers that story, your scoring code works and you can trust it to find the best AI chatbot on real votes.
These numbers come from synthetic votes and say nothing about the real models. They exist so you can learn to read the report.
Write your decision rule before you look at the real results. A workable one has three lines. Pick the highest Elo if its lower bound clears the next model’s upper bound. If the intervals overlap, pick the lower cost per 100 preferred answers. Keep a second model as a specialist for any category where it leads by at least 15 points of score across ten or more prompts. Deciding the rule first stops you from reading the numbers to fit a favorite, and it keeps your pick of the best AI chatbot honest.
Then freeze the evidence. Copy the database into a dated folder and write down the model identifiers, the prices in config.py, and the prompt list. A result without those details cannot be repeated.
Plan the rerun now. Three triggers justify one: a new flagship from any vendor, a price change such as the end of Argon’s introductory pricing on November 30, and a quarterly calendar reminder. Anthropic’s own pages already describe Sonnet 5.5 and Haiku 5.5, so your lineup will move. Append rows to MODELS, rerun generate.py, and only the new answers get collected. Further model coverage lives in our AI and machine learning section.
When none of these match, rerun the failing adapter by itself with a short prompt and print the raw response. Nine times out of ten the field name changed.
The folder below is the whole project. Nothing else is required, and the only state lives in arena.db.
Before you run a real session, make sure the demo output matches the numbers shown above. The demo is deterministic, so any difference points to a typo in your copy of rank.py.
It depends on your prompts, which is the reason this tutorial exists. The four flagships here are the newest releases from their vendors as of November 9, 2026, and none of them wins every job. Run the arena with your own tasks and let the votes name the best AI chatbot for your work.
Aim for at least 50 decisive votes per model pair. At that size a 60 percent win rate still carries a margin of about 14 points, so close pairs stay uncertain. Add raters or prompts until the intervals in the ranking table stop overlapping for the models you care about.
Yes, but the page has no authentication, so do not expose it to the open internet. Put it behind a VPN or a reverse proxy with a password, or share the SQLite file and have each person vote locally, then merge the votes tables.
Insert a row into answers with a new model key such as sol-app and the pasted text, then add a matching Model row so generate.py does not try to call it. The matchup picker uses whatever keys exist in the table.
No. OpenAI’s pricing page lists GPT-6 Sol as unavailable on Free and Go, available on Plus, and available with expanded access on Pro. A test through the API shows what the model can do, not what a free account serves.
The prompts are short, so the bill stays small. The generator prints the cost of every answer using the prices in config.py, and the cost block in the report sums it up per model. Rates change, notably Argon’s introductory API price, which ends on November 30, 2026 and is followed by $4 and $20 per million tokens, so refresh the registry before each run.
Judge models are fast and cheap, but they lean toward their own writing style and toward longer answers. They work well as a pre-screen and poorly as the final word. The blind human vote tells you what your team prefers to read.
Rerun when a vendor ships a new flagship, when prices move, and at least once a quarter. Keep every dated report so you can see whether a model improved or slipped between runs.
Nadia Dubois is the AI & Innovation Editor at Tech Insider, where she tracks the rapid evolution of artificial intelligence, from foundation models to real-world enterprise deployment. She previously covered AI and startups for La Tribune and contributed to MIT Technology Review's European coverage. Nadia specializes in generative AI, AI regulation, and the intersection of technology and European industrial policy. She holds a dual degree in Computational Linguistics and Journalism from Sciences Po Paris.
The Tech Insider Newsletter
Industry insights, the latest tech news, and special interviews, every week.
One email a week. No spam, unsubscribe anytime.
Tech Insider delivers in-depth coverage of the technologies shaping the future: AI, cybersecurity, cloud computing, hardware, and the trends that matter.
Tech Insider delivers in-depth coverage of the technologies shaping our future. From AI and cybersecurity to cloud computing and hardware innovation, our editorial team covers the trends that matter.
Tech Insider delivers in-depth coverage of the technologies shaping our future. From AI and cybersecurity to cloud computing and hardware innovation, our editorial team covers the trends that matter.
English | Svenska | Français | Suomi
The Tech Insider Newsletter: industry insights, tech news, and special interviews.