Arabic RAG Build Sheet: Normalization Rules, PDF Extractors, and the Embedding Models to Test First

A RAG pipeline tuned for English does not crash on Arabic. It retrieves the wrong clause, misses a decree number, and quietly inflates the bill. The failures sit in the plumbing: normalization, PDF extraction, the keyword index, and the choice of embedding model. This sheet lists the fixes in the order you apply them.

It is the engineering companion to our buyer-facing post on the four places Arabic breaks an English-first pipeline, and how to budget for them. That post explains the problem to the person signing off. This one is for the person building it.

Tokenization: measure it, do not quote a ratio

Older English-first tokenizers cut Arabic words into many more pieces than English words. Newer tokenizers from the major model families have narrowed that gap. By how much depends on the tokenizer version and on your text. Contracts, chat messages and call transcripts all tokenize differently. Any single ratio you read online was measured on somebody else's corpus.

Measure your own instead. The method takes an afternoon:

1. ▸Pull 200 documents or messages from real traffic. Include dialect and mixed Arabic-English text, not just formal prose.
2. ▸Count words. Then count tokens with each candidate tokenizer. OpenAI publishes its tokenizers in the open-source tiktoken library. Open-weight models ship their tokenizer files with the weights, and the Hugging Face tokenizers library loads them.
3. ▸Divide tokens by words, once per tokenizer. That is your tokens-per-word figure for each model.

Put the figure into the cost and latency formulas in the buyer post. The gap between rows is your real tokenization cost, in your currency. Record the number on the build sheet. It also sets your chunk sizes, because a chunk budget in tokens holds fewer Arabic words than English ones.

Normalize before you embed or index

The same Arabic word is written several ways across one document, and differently again in the user's query. Without normalization, keyword search misses matches and hybrid retrieval degrades. The rule that matters most is consistency. Apply the same function at index time and at query time. A mismatch between the two silently cuts recall.

Keep the original text for display and citations. Normalize a copy for the index.

Three toolkits implement these rules: CAMeL Tools from NYU Abu Dhabi, Farasa from QCRI, and PyArabic. Run Unicode NFKC normalization first, as defined in Unicode Standard Annex 15. It converts Arabic presentation forms, the joined letter shapes some PDFs store, back into standard letters.

PDF and RTL extraction: where most Arabic pipelines fail

Default PDF extractors mangle Arabic. Words come out reversed, letters disconnected, diacritics stranded, table cells scrambled. The chunker then stores garbage, and the embeddings are embeddings of garbage.

Tools to test, roughly in the order they tend to get Arabic right first time:

• ▸PyMuPDF for born-digital Arabic PDFs. It reorders right-to-left text natively. The pdftext project names that as the main extraction-quality difference between PyMuPDF and pdfium on right-to-left documents.
• ▸pdfplumber for tables, with your own right-to-left post-processing on cell text.
• ▸Docling and Unstructured for mixed enterprise documents. Both keep layout better than plain extractors. Validate on your own files.
• ▸Marker and MinerU for complex layouts with equations and figures.
• ▸Azure Document Intelligence for scans. Microsoft lists Arabic for printed and handwritten text in its language support table. Amazon Textract's FAQ lists six supported languages, and Arabic is not among them, so leave it off the shortlist.
• ▸Tesseract with the Arabic model from tessdata as the open-source OCR fallback for closed environments. Expect weaker results on poor scans and handwriting.

For scanned archives, the choice is wider than one OCR engine. The multimodal post compares layout-aware parsing versus page-image retrieval for scans.

Before you commit, extract 20 real PDFs with each candidate and compare the output to the page by eye. If the first paragraph reads right to left and the diacritics sit on the correct letters, the extractor works. If not, no embedding model will rescue the pipeline.

Chunking: the risk is the reference, not the word

A common worry is that chunking cuts Arabic words in half. It rarely does. Recursive splitters break on paragraphs, then lines, then spaces. They almost never split inside a word, in any language. Only a raw character splitter with no separators does that, and it should not be in the pipeline for English either.

The real Arabic risk is splitting inside a legal reference or a table. A clause such as المادة (12) من المرسوم الملكي رقم م/19 must stay in one chunk with the number it cites. A dosage table must stay intact with its header row.

Practical rules:

• ▸Split on structure first: headings, articles, numbered clauses and table boundaries.
• ▸Then split on Arabic sentence punctuation: the full stop, the Arabic question mark ؟ and the Arabic semicolon ؛. CAMeL Tools includes a sentence splitter.
• ▸Keep one sentence of overlap between neighboring chunks, so a reference that closes one chunk also opens the next.
• ▸Never separate a table row from its header. Store the table as its own chunk, or chunk it row by row with the header repeated.
• ▸Store the article or section path in the chunk metadata. A retrieved clause then arrives with the decree it belongs to.

Hybrid retrieval: dense vectors plus clitic-aware keyword search

Dense vectors carry meaning. Keyword search (BM25) carries exact strings: decree numbers, national IDs, case citations, product codes, Hijri dates. You need both. For Arabic keyword search to work, the index must understand clitics, the short words Arabic attaches to the front and back of a word. To a reader, العقد, والعقد, بالعقد and عقدنا are one word. To a naive index they are four.

Two mature paths:

• ▸The Lucene Arabic analyzer, built into Elasticsearch and OpenSearch. Elastic documents it among its language analyzers. It normalizes, strips common prefixes such as ال and و and ب and ل, and applies a light stemmer. It is cheap and well understood, and the right default.
• ▸CAMeL Tools morphological analysis feeding a custom analyzer. It is more accurate on inflected forms and slower at index time. Use it when the default misses too many forms in your test queries.

Merge the two result lists with Reciprocal Rank Fusion. It needs no score calibration between the dense and sparse systems. Qdrant documents it in its hybrid queries guide, and Elasticsearch and Weaviate support it too. Pass the top 50 to 100 candidates to the reranker.

Embedding and reranker shortlist

Not every "multilingual" model was tested on Arabic. The models below have documented Arabic coverage. Test them in this order, on your own set of paired queries and passages.

Embeddings:

• ▸BGE-M3 (BAAI). Dense, sparse and multi-vector output from one model. Open weights. The default when retrieval has to run on-prem.
• ▸multilingual-e5-large and multilingual-e5-large-instruct (Microsoft). A smaller footprint than BGE-M3, and reliable.
• ▸Cohere Embed multilingual. Hosted. Cohere's models page lists Arabic among the supported languages.
• ▸OpenAI text-embedding-3-large. Hosted, documented in the embeddings guide. Reasonable on Arabic. Test it against BGE-M3 before choosing it.
• ▸Jina Embeddings v3. Open weights with a hosted option. Arabic is in its documented language list.

Rerankers:

• ▸BGE Reranker v2 M3 and BGE Reranker v2 Gemma. Multilingual cross-encoders with open weights.
• ▸Cohere Rerank. Hosted, pay per query, documented in the Rerank overview.
• ▸Jina Reranker v2 multilingual. Open weights with a hosted option.

Never put an English-only cross-encoder in front of Arabic candidates. It buries the best match below the top five.

Keep one embedding model for Arabic, English and mixed queries. Let the keyword index carry the language-specific normalization. Two embedding models in one index means two geometries that never quite agree.

Evaluation: a test set with Arabic in it

Automated metrics catch regressions. They do not tell you whether an answer is right. Build the 100-question test set, with Arabic questions in the share you see in traffic, before you tune anything. Draw the questions from real user queries, not from questions written during development.

Make sure five categories are present: exact-string lookups such as decree numbers and dates, paraphrased queries, mixed Arabic-English queries, dialect queries, and questions the documents do not answer. Score faithfulness, answer relevance and context recall with an Arabic-capable judge model. Then have a native speaker read a sample of 50 to 100 answers per release. Mixed-script and dialect queries fail first, so weight them in the sample.

The on-prem stack for Gulf deployments

Many Gulf engagements require the embeddings, the vector store and the generation model inside the country, or on the customer's own hardware. Saudi Arabia's Personal Data Protection Law, published by SDAIA, restricts transfers of personal data outside the Kingdom. The UAE's Federal Decree-Law No. 45 of 2021 sets conditions for cross-border transfers. Sector rules in health and finance add more. Our PDPL compliance checklist works through what each one means for an AI system.

The open-weight stack that meets those rules:

• ▸Generation: a Qwen, Llama, Mistral or Jais model in the 8B to 32B class, served with vLLM or SGLang. Go larger only if your test set shows the gap.
• ▸Embeddings: BGE-M3 or multilingual-e5-large on a dedicated GPU, or a CPU host with batching.
• ▸Reranker: BGE Reranker v2 M3. It runs on a modest GPU.
• ▸Vector store: Qdrant or pgvector. Both do hybrid search. The trade-offs at scale are in Qdrant vs pgvector at 10M vectors.
• ▸OCR: Tesseract with Arabic data inside closed environments, or an in-country Azure endpoint if policy permits.

Latency is the usual on-prem complaint. Measure time to first token and retrieval time on your own hardware with real Arabic prompts. Then work through the serving-stack levers in order. Whether the fixed cost beats per-token pricing at your volume is a separate question, and the breakeven math is in our on-premise ROI post.

Frequently asked questions

Do we still need to translate Arabic documents to English before indexing? No. Current multilingual embeddings and rerankers map Arabic directly. Translating first adds errors, latency and cost. It also breaks citations, because the retrieved text no longer matches the source page.

Can one pipeline serve Modern Standard Arabic and Gulf dialects? Partly. MSA is well covered by the models above. Dialects are weaker across every model. Put dialect queries in the test set first. If recall is low, add a dialect-to-MSA rewrite step on the query side, or fine-tune the embedding model on in-domain pairs.

This week, run two of the tests above on your own corpus. Extract 20 real PDFs with your current tool and compare them to the page by eye. Then search your index for one term in each alef variant and count the paragraphs each search misses. Record the misses. That list is the first page of your build sheet, and it tells you which section above to start with.