Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Scientists have found a way to teach AI chatbots to read individual letters instead of word chunks, improving spelling skills without sacrificing overall performance.
An AI chatbot can write a whole essay about a word and still struggle to spell that word backward. That’s because most AI language models never see individual letters.
Before reading anything, they break text into chunks of whole words or pieces of words.
Researchers have now converted several existing models to read text in bytes, the small units computers use to represent text. Unlike word-based chunks, bytes allow models to process text at a much finer level.
The conversion took less than 1% of the training such a model normally needs. Until now, a model that read text this way had to be built from scratch, and those models lagged behind.
Benjamin Minixhofer of the Allen Institute for AI and the University of Cambridge led the work.
Valentin Hofmann of the Allen Institute and LMU Munich was the senior author, with co-authors at the University of Washington and Imperial College London.
The team tested openly available models with up to about 8 billion parameters, the adjustable values a model learns in training.
Before a language model reads a sentence, a separate program splits it into tokens. A common word usually becomes one token, and a rarer word gets broken into a few pieces.
The model learns from those pieces and never directly sees the letters inside them.
Chunking makes models efficient, but it has costs.
Models built this way have trouble with tasks that depend on single characters, which matters for computer code and for DNA and other biological sequences.
A fixed vocabulary of chunks also tends to favor English.
Models that read raw bytes avoid those problems. But byte-level models trained from scratch have trailed the chunk-based kind, and every leading chatbot model still reads in chunks.
Minixhofer’s team didn’t start over. They kept the core of each finished model and added new, smaller parts that read bytes and group them into chunks the core can work with. They call the process byteification.
In the first stage, only the new parts were trained, with the goal of making the converted model behave exactly like the original. In the second stage, the whole model learned to use the letter-level detail.
The whole process used about 49 billion tokens of text, less than 1% of the training data typically used to build a comparable model from scratch.
The team converted three openly available models this way: Qwen3, Llama 3 and Olmo 3. The converted versions got new names, Bwen, Blama and Bolmo.
On a set of tests covering science, math, coding, reading and general knowledge, the converted models landed close to the models they came from.
Bolmo averaged 62.5, against 62.3 for the original Olmo 3. Bwen averaged 69.8, against 70.3 for Qwen3.
In other words, the converted models retained nearly all of their general abilities.
The converted models also beat every earlier byte-reading model of similar size.
On science, technology, engineering and math questions, Bolmo scored 16.5 points higher than one such model trained from scratch.
The team could also give a converted model new skills without more training.
Copying over the changes that had taught Olmo 3 to follow instructions raised Bolmo’s instruction-following score by 36 percentage points, roughly matching the instruction-trained Olmo 3.
In speed, the converted models came close to the originals. They can also be trained to read bigger chunks at once, which makes them faster at some cost in accuracy.
The biggest gains came on letter-level tasks, such as spelling out a word, reversing it, or swapping and deleting characters.
On those tests, Bolmo scored 78.1 against 54.3 for the original Olmo 3, and Blama scored 81.0 against 57.2 for Llama 3.
Hofmann, a junior professor at LMU Munich who studies language processing with AI, described the gap directly.
“Traditional LLMs struggle with tasks that require character-level capabilities, such as spelling a word backwards,” Hofmann said. “Our byteified models are substantively better at this.”
Part of that gain came from practice rather than from reading bytes. The team added about 75 million tokens of training text built around letter tasks, roughly 0.04% of the training data.
An unconverted copy of Olmo 3 trained on the same mix reached 70.7, so the byte-reading design added the rest.
Not every score went up. The converted models were less likely than the originals to write working computer code on the first try. Given many tries, though, they more often produced at least one answer that worked.
A split result like that could mean the converted models write more varied code, but differences in the settings used to generate answers could also explain it.
The letter-practice text was all in English, and the models tested stopped at about 8 billion parameters.
Whether the conversion works as well on much larger models hasn’t been tested. Neither has the idea that reading bytes would help with languages other than English, which the team lists as a potential benefit.
The team has released its models, code and training data, so other groups can run those checks themselves.
The full study was published in the journal Nature.
—–
Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates.
Check us out on EarthSnap, a free app brought to you by Eric Ralls and Earth.com.
—–
