How Well Do AI Models Handle Arabic

Arabic AI models have improved enough that a Dubai business can now put one in front of Arabic-speaking customers, but not enough that you can skip testing. The models are strong on Modern Standard Arabic, the formal register used in news and government documents. They are weaker on the Gulf dialect your customers actually type into a WhatsApp message, and weakest on the mixture of Arabic, English and Latin-letter Arabic that real people write.

The gap that catches most teams is not the model at all. It is the layer around it: how the text is stored, how it is rendered right to left, and how much the same sentence costs in tokens. This article covers what has genuinely improved, what the benchmark numbers mean, and the specific checks worth running before an Arabic assistant answers a customer.

Key takeaways

• The ISO 639-3 registry treats Arabic as a macrolanguage with 30 individual languages inside it, with Standard Arabic (arb) listed separately from Gulf Arabic (afb) and Egyptian Arabic (arz).
• Abu Dhabi’s Technology Innovation Institute announced Falcon-H1 Arabic on 5 January 2026 in 3B, 7B and 34B sizes, and reports it as the highest scorer on the Open Arabic LLM Leaderboard.
• Benchmark averages are reported by the organisation that built the model, so treat them as a reason to shortlist, not as evidence it handles your customers.
• In a small test on 30 September 2026, Arabic sentences used about 1.2 times the tokens of their English equivalents, and nearly twice as many tokens per word.
• Right-to-left display is controlled by the HTML dir attribute, which W3C treats as a requirement for scripts such as Arabic. No model setting fixes a page that omits it.

Arabic is 30 languages wearing one name

This is the part that explains most disappointing results. The ISO 639-3 registry, maintained by SIL International, classifies Arabic (code ara) as a macrolanguage rather than a single language. Its published macrolanguage mapping table lists 30 individual languages inside it. Standard Arabic has its own code, arb. Gulf Arabic is afb. Egyptian Arabic is arz, Levantine is apc, Mesopotamian is acm, Moroccan is ary and Hijazi is acw.

A registry is not a verdict on how different those varieties feel to a speaker, but it does tell you something useful about tooling: when a model card, a dataset or an API says it supports “Arabic”, that word is doing a lot of work. Most training data is Modern Standard Arabic, because that is what exists in volume as published text. Very little of what a Dubai customer types is Modern Standard Arabic.

The practical effect shows up in tone before it shows up in errors. An assistant trained mostly on formal text answers a casual Khaleeji question in the register of a government circular. It is not wrong, but it reads as though the company does not quite speak to its own market, which is the same trap covered in our note on Arabic and English marketing for GCC audiences. A trading company in Deira testing an Arabic support bot found the answers technically correct and oddly stiff, which is the usual first result.

Code-switching and Arabizi

Two habits break naive setups. The first is code-switching, where a single message mixes Arabic and English, often putting the English product name or job title inside an Arabic sentence. The second is Arabizi, Arabic written in Latin letters with digits standing in for sounds that have no Latin equivalent, so that the digit 3 stands in for the letter ain and 7 for haa.

Neither is unusual in the UAE, and neither is well represented in formal training text. If your test set is written by a marketing colleague in clean Modern Standard Arabic, it will not surface either problem. Pull real messages from your own inbox instead.

What the benchmark scores actually tell you

The reference point for Arabic models is the Open Arabic LLM Leaderboard, usually shortened to OALL, which scores models across Arabic understanding and reasoning tasks.

The Technology Innovation Institute in Abu Dhabi announced Falcon-H1 Arabic on 5 January 2026, built on a hybrid Mamba-Transformer architecture and released in 3B, 7B and 34B parameter sizes. In its own announcement TII reports OALL averages of 61.87 percent for the 3B model, 71.47 percent for the 7B and 75.36 percent for the 34B, and states that the 34B result is ahead of considerably larger systems including Qwen2.5 72B and Llama-3.3 70B. It also cites results on three narrower benchmarks: 3LM for STEM reasoning, ArabCulture for cultural and contextual understanding, and AraDice for dialect comprehension.

Two things are worth holding on to. The first is that a regional model built specifically for Arabic is now a serious option rather than a patriotic one, and that a 7B model you can run yourself scores in the same range as models an order of magnitude larger. The second is that those figures come from the organisation that built the model. That is normal practice and not a criticism, but vendor-reported benchmark averages are a reason to shortlist a model, never a substitute for testing it on your own content.

The narrower benchmarks matter more than the headline average for most businesses. A model that does well on STEM reasoning is not necessarily the one that understands a Sharjah customer asking about a delivery in dialect. If dialect handling is what your use case depends on, the dialect benchmark is the number to look at, and your own test set matters more than either.

Arabic costs more tokens than English

This one is invisible until the bill arrives. Language models charge by token, and Arabic does not tokenise as efficiently as English in general-purpose encodings.

On 30 September 2026 we ran three matched sentences of ordinary Dubai business English and their Arabic equivalents through the o200k_base encoding used by recent GPT models. Across the three, the Arabic versions came to 55 tokens against 45 for the English, about 1.2 times as many. Measured per word the gap is wider: 1.15 tokens per English word against 1.96 per Arabic word, because Arabic packs prefixes, suffixes and the definite article into single written words that the tokenizer then splits.

Three sentences is a small sample and the ratio moves with the encoding and the subject matter, so treat that as an indication rather than a planning figure. The point that holds is directional and easy to check yourself: run a few hundred of your own real messages through the tokenizer for the model you are considering, in both languages, before you budget.

The consequence is not only cost. Token count also consumes the context window, so a support assistant that holds ten previous messages in English may hold fewer in Arabic before it starts forgetting the beginning of the conversation. For a clinic in Jumeirah running a booking assistant, that is the difference between remembering the patient’s stated preference and asking for it twice.

Right to left is your developer’s problem, not the model’s

A large share of what looks like bad Arabic AI is ordinary front-end work that was never done. The model returns correct Arabic and the page displays it badly.

W3C’s internationalisation guidance is direct about the mechanism: the HTML dir attribute sets the base direction of text for display, and it calls that attribute essential for supporting right-to-left scripts including Arabic. Direction is structural information and belongs in the markup, not only in a stylesheet.

For interfaces where the language of a given message is not known ahead of time, which is exactly the case for a chat window, W3C describes dir=”auto”: applied to the element wrapping each message, the first strongly typed character determines that message’s direction. That is the single most useful thing to know when building a bilingual chat interface, because it handles an English reply and an Arabic reply in the same thread without the developer having to detect the language.

One issue we often see is a chat widget where the bubbles are styled with a right-aligned CSS rule and no dir attribute at all. Pure Arabic looks acceptable. As soon as a message mixes Arabic with an English product code or a phone number, the punctuation and the digits land in the wrong place, and the customer sees a sentence that is subtly scrambled. The model did nothing wrong. The same applies to the surrounding site: our guide to hreflang for Arabic and English UAE websites covers the page-level side of running both languages.

What to test before Arabic AI faces a customer

The useful test set is not large. It is specific, and it comes from your own business rather than from a sample document.

1. Collect 50 to 100 real customer messages in Arabic from WhatsApp, the contact form or the support inbox, without cleaning them up.
2. Keep the dialect, the spelling mistakes, the missing diacritics and any Arabizi exactly as written, because those are the inputs the system will actually receive.
3. Include at least ten messages that mix Arabic and English in one sentence, since that is where both the model and the interface tend to fail.
4. Check how the model writes numbers, dates and currency, and whether it matches how your invoices and your website already present them.
5. Check proper nouns: company names, Emirati and other Arabic personal names, and area names such as Al Quoz or Al Barsha, which are frequently mangled in transliteration.
6. Have a native Gulf Arabic speaker read the replies for register, not only for accuracy, and ask them whether a customer would find the tone normal.
7. Test the rendering on a phone, in the real interface, with a message that mixes both languages.

Step six is the one teams skip, and it is the one that decides whether customers trust the channel. Accuracy is measurable by anyone. Register is only visible to someone who speaks the dialect, and a fluent but formal reply reads as a form letter.

Where Dubai businesses get caught

From a development perspective, three patterns come up repeatedly.

The first is treating Arabic as a translation layer over an English product. The interface is built and tested in English, Arabic is added at the end, and the result is a right-to-left page with a left-to-right layout underneath it. Retrofitting direction into a finished front end costs considerably more than designing for it from the start.

The second is assuming the model is the expensive part of the decision. Fine-tuning gets the attention while the data, the test set and the front-end work take most of the effort. For many businesses a well-tested general model with a good prompt and a proper right-to-left interface outperforms a fine-tuned model behind a broken one.

The third is deploying an Arabic assistant with no route to a human. A model that handles most Arabic questions well will still meet dialect it does not follow, and a customer who has to repeat a question twice in their own language will not try a third time. Deciding what to automate first is a separate question, and one we covered in what Dubai SMEs should automate first.

For a smaller business we would usually recommend starting with a narrow, high-volume task in Arabic, such as answering opening hours, location and booking questions, rather than a general assistant. The failure modes are easier to see and the testing set is small enough to build in an afternoon.

If you are weighing up which model to put behind an Arabic assistant, or want the interface checked before it meets customers, our LLM integration team can look at both. For a narrower starting point, see our AI chatbot development.

Figures and sources were checked on 30 September 2026 against the ISO 639-3 registry published by SIL International, the Technology Innovation Institute’s Falcon-H1 Arabic announcement and W3C internationalisation guidance. Benchmark percentages are as reported by the model’s developer. The token measurement is our own and based on a three-sentence sample, so treat it as an indication rather than a planning figure. Cover photo: Jebel Ali Dubai (2) by Producer, via Wikimedia Commons (CC BY 2.5).