Arabic AI voice agents: what works and what to test

The best public evidence is the Open Universal Arabic ASR Leaderboard, which tests open-source speech recognition models on multi-dialect Arabic datasets. Its December 2024 paper reports word error rates (WER, lower is better) for Whisper large-v3, a widely used model, on the SADA dataset by dialect:

The authors found that all models did best on MSA and declined significantly on dialects such as Egyptian and Khaleeji, which they attribute mainly to MSA dominating public training data. Background noise and music made every model worse.

Two cautions. These are open-source models on a deliberately hard dataset, not the commercial engines inside every voice platform, so the absolute numbers won't match your calls. And the field is moving: in July 2026 Cohere reported an Arabic-specific model with an average WER of 25.87 on the same leaderboard, against 36.86 for Whisper large-v3. The direction is what matters: dialect and noise are where Arabic agents fail first, and the model you choose makes a real difference.

Code-switching is normal in the Gulf: "Abgha appointment bukra, after Asr if possible." An agent has to understand the mixed sentence and answer in a way the caller is comfortable with. Check how your platform handles it. ElevenLabs, for example, documents a language detection tool that lets the agent switch to the caller's preferred language, while a language chosen in its web widget stays fixed for the call. Whatever the platform, test mixed-language calls explicitly; don't assume.

In our testing, the same handful of things break first:

• Names. Arabic names have many English spellings (Mohammed, Muhammad, Mohamad), and the agent may hear a name right but write it differently from your records.
• Phone numbers. Callers read numbers in Arabic, in English, or in a mix, and group digits differently. Use the caller ID where you can instead of asking.
• Dates and times. "Ba'd bukra" (the day after tomorrow), "after Maghrib", or a day of the week without a date all have to be turned into a real slot.
• Addresses. Directions are often given by landmark rather than street, which matters for home visits and deliveries.
• Noise. Calls from cars and speakerphones are common, and they raise error rates.

The fix for all of these is the same: have the agent read the key details back (name, date, time, branch) and confirm them in writing on WhatsApp after the call.

Here is an illustrative extract from the kind of call our agents handle. The clinic, caller and times are made up; the structure is what we test for: understand the mixed request, offer real slots, read the details back, confirm in writing.

• Agent: مرحباً، معك المساعد الآلي لعيادة النخيل. كيف أقدر أساعدك؟ (Hello, this is the automated assistant for Al Nakheel Clinic. How can I help?)
• Caller: أبغى أحجز appointment مع دكتورة سارة، بكرة بعد العصر إذا ممكن. (I want to book an appointment with Dr Sara, tomorrow after Asr if possible.)
• Agent: عندها موعد الساعة 4:30 أو 5:15 مساءً بكرة الثلاثاء. أي وقت يناسبك؟ (She has 4:30 or 5:15 pm tomorrow, Tuesday. Which suits you?)
• Caller: 4:30 تمام. (4:30 is fine.)
• Agent: تم. موعدك مع د. سارة يوم الثلاثاء الساعة 4:30 مساءً في فرع البرشاء. بيوصلك تأكيد على الواتساب الحين. (Done. Your appointment with Dr Sara is Tuesday at 4:30 pm at the Al Barsha branch. A WhatsApp confirmation is on its way.)

This is the protocol we use. Adapt the numbers to your call volume.

1. Collect real examples: listen to a sample of your clinic's calls (with consent) and list the call types and phrases people actually use.
2. Script at least 30 to 50 test calls covering each call type, at least three dialects, code-switching, fast and slow speakers, and calls from a car or speakerphone.
3. Include the hard fields in every script: an Arabic name, a phone number, a relative date and a time given by prayer or part of day.
4. Score each call on task completion (was the right appointment booked?), errors in the key fields, correct hand-off when the agent should give up, and whether the pauses felt natural.
5. Agree the sign-off rule with the clinic before testing, for example: every booking detail correct, or the call handed to a person.
6. Go live after hours first, and review every transcript for the first two weeks.

Clinical questions, complaints, distressed callers, anything about results, and any call where the agent has asked the caller to repeat themselves twice. Hand-off should mean a live transfer during opening hours and, after hours, a promise of a callback with the full transcript waiting for the team in the morning.

Open every call by saying, in the caller's language, that they're speaking with an automated assistant, and say if the call is recorded. In the UAE, recording a private phone conversation without consent can be an offence under Article 431 of the Penal Code, so get the consent wording right; our guide to patient data rules in the UAE covers the wider picture. Keep the opening short: a long legal preamble is where callers hang up.