Clinical Protocol RAG On-Prem: Architecture for Gulf Hospitals

A physician looking up a sepsis protocol at 3 AM cannot stop to check whether the answer was invented. They need the guideline the hospital approved and has in force today, with a citation they can open. A general-purpose chatbot cannot promise either.

The pattern that fits is a private knowledge base running inside the hospital. It reads the hospital's own protocols and answers only from them. The technical name is retrieval-augmented generation, or RAG: the system finds the relevant passage first, then writes a short answer from that passage alone.

The build is well understood, apart from one part that usually goes wrong: protocols change, and the system keeps quoting the old version. Most of the detail below is spent on that problem.

What the system does, and what it must never do

A clinical protocol system is a lookup tool with a summary on top. It takes a question and finds the matching section in approved documents. It then shows that section with a short plain-language summary drawn only from the retrieved text. If the approved documents do not answer the question, it says so and stops.

It does not diagnose. It does not recommend treatment for a specific patient. It does not fill gaps with general medical knowledge the model absorbed during training.

Each of those three is how a useful lookup tool becomes a clinical risk. The rest of the architecture exists to enforce that boundary.

Why it runs on-prem in the Gulf

A protocol document on its own may contain no patient data. The questions usually do. A clinician types: "54-year-old male, CKD stage 3, penicillin allergy, what is our protocol for community-acquired pneumonia?" Send that to an external API and patient health data has left the building.

Two legal facts shape the decision:

• ▸Saudi Arabia. The Personal Data Protection Law treats health data as sensitive. It restricts transfers outside the Kingdom, and SDAIA's implementing regulations set the conditions for any transfer. SDAIA publishes the law's fines of up to SAR 5 million, which may be doubled for a repeat violation. Our PDPL checklist for AI deployments turns the articles into build requirements.
• ▸UAE. Federal Law No. 2 of 2019 on the use of ICT in health fields requires health data to be stored and processed inside the country, with narrow exceptions set by ministerial decision. Abu Dhabi facilities also answer to DoH Abu Dhabi's ADHICS security standard, and Dubai facilities to DHA requirements.

The rules differ by country and emirate, so have your compliance team confirm the specifics. The practical answer is the same everywhere: keep the search index and the language model inside the hospital perimeter. A certified in-country sovereign cloud is the alternative. "In our data center" is far easier to defend to a regulator than "in a foreign region under a zero-retention contract."

Open-weights model families such as Qwen and Llama run well on hospital-owned hardware. Summarizing a retrieved passage is a narrow task. The quality gap with the largest hosted models is small here, but measure it on your own test set rather than assume it.

Three engineering problems that separate clinical RAG from a demo

1. Reading real clinical PDFs correctly

Clinical guidelines are rarely clean text. They contain decision flowcharts, dosage tables, footnotes and scanned approval pages. Basic text extraction flattens tables into one stream of numbers. A pediatric dose can end up read as part of the adult column. The system then holds an incorrect dose that no one wrote.

Use a layout-aware parser or a vision-capable model at ingestion. Tables stay tables. Flowcharts become numbered steps. Section headings survive. Then have a person spot-check every table-heavy document before it goes live. Ingestion errors cannot be fixed later in the pipeline.

2. Exact matches for drug names and codes

Meaning-based search is good with concepts. It knows "high blood pressure" and "hypertension" mean the same thing. It is unreliable with identifiers. A query for one protocol code can return a different code that merely looks similar.

The fix is hybrid search. Meaning-based search handles concepts. Keyword search, typically BM25, guarantees exact hits on drug names, codes and abbreviations. The two result lists are merged, and a reranking model reorders the best candidates. This step costs little and removes a whole class of wrong-document errors. The pgvector documentation describes pairing it with Postgres full-text search and merging results with Reciprocal Rank Fusion or a cross-encoder.

3. Splitting documents along clinical sections

The system stores documents in pieces, called chunks. Split them at real section boundaries: Indications, Contraindications, Dosage, Monitoring. Each chunk should carry its parent protocol's title and section path. A contraindications passage must never appear without the protocol it belongs to.

The hard problem: superseded protocols

Here is the failure that ships quietly. The 2023 sepsis guideline still sits in the shared drive next to the 2026 revision. Both get indexed. Search returns whichever one matches the wording of the question better. Sometimes that is the old one, and the clinician sees no warning.

Deleting old versions is not the answer. Audit and medico-legal obligations mean you must show what guidance applied on a given date. The system has to keep history while making superseded content unreachable for live questions. That takes three mechanisms.

Mechanism 1: required metadata on every chunk

Ingestion rejects any document missing these fields. No exceptions, including for "urgent" uploads.

These fields are the clinical form of the general metadata schema for dates, access, status and lineage that any enterprise RAG system needs. The clinical version adds scope and approver and makes every field mandatory.

The scope field is the one teams forget. Multi-site networks often keep a network policy plus local variants. A question from the ICU at one hospital should get that hospital's ICU protocol if one exists, then fall back to the network default.

Mechanism 2: filter before search, not after

Every query carries a reference date, defaulting to today. The search runs only over chunks that pass this rule:

status = active
AND effective_from is on or before the reference date
AND (effective_to is empty OR effective_to is after the reference date)
AND scope matches the user's facility or is network-wide

Qdrant, pgvector and Weaviate all apply this kind of filter inside the search itself. Filtering after search is wrong. A strong match on a withdrawn protocol can push the current one out of the top results, and then there is nothing left to filter in.

Audit users can set the reference date to the past and see what applied then. Clinicians never can.

Mechanism 3: a publish workflow, not a folder

Approving a new version is one logged action, not a file dropped into a shared drive. That action does three things together. It marks the previous version superseded. It sets the old version's end date to the new version's start date. It activates the new version's chunks. The approver's identity is recorded.

If any step fails, none of them apply. A half-published protocol, with both versions active, is the exact state you are building to prevent.

Every answer then shows the protocol name, version number and effective date beside the citation. Clinicians learn to glance at it, and a stale answer becomes visible instead of silent.

Build this on day one. Retrofitting it onto a live index means backfilling metadata for every chunk. It also means finding approval records for old documents that may not exist. The general version of that backfill problem is covered in our metadata schema post.

Hardware and running cost

Answering questions needs far less compute than training a model. As a planning range, a mid-sized network with a few hundred clinical users is usually served by one server. The GPU memory it needs follows from the formula in our on-prem LLM speed guide: weight memory is parameters times bytes per parameter, plus working memory for concurrent users. A 32B model at 8-bit needs about 32 GB for weights alone. A 70B model at 4-bit needs roughly 35 to 40 GB. That is why two data-center GPUs with 40 to 80 GB of memory each is the usual planning range. Confirm the sizing against your expected peak load before purchase. Typical software: Qdrant or pgvector for search, vLLM or SGLang to serve the model, Langfuse to log every query and answer.

Compare on-prem against cloud with your own numbers:

Monthly cloud cost = queries per day × 30 × (average input tokens × input rate + average output tokens × output rate)

Clinical RAG questions are input-heavy, because each one carries several retrieved passages. Set that monthly figure against server cost spread over its life, plus power and support. Your traffic, not a vendor slide, tells you where the lines cross. The same guide covers getting more throughput from the same hardware.

Evaluation before a clinician sees it

A medical board needs evidence, not a demo. Build a test set of a few hundred real questions, each paired with the protocol section that should answer it. Write the questions with clinicians, not engineers. Our evaluation post covers how to build and gate a 100-question test set; a clinical set is larger and adds the gates below. Set pass thresholds with your clinical governance committee before any results come in.

Four gates matter:

1. ▸Faithfulness. Every claim in the answer traces to retrieved text. Outside knowledge, even correct knowledge, counts as a failure.
2. ▸Context recall. Retrieval found the right section. If a pediatric asthma question returns the adult protocol, nothing downstream can recover.
3. ▸Version correctness. A set of questions whose answers changed between versions. The system must return the current version every time. One miss means the filter logic is broken.
4. ▸Correct refusal. Questions the protocols do not cover. The system must say it has no approved answer, not improvise one.

Rerun the full set whenever protocols, models or search settings change.

FAQ

Does the system make medical decisions? No. It retrieves and summarizes approved protocols, with citations. It replaces searching a PDF directory, not clinical judgment. Keep that boundary explicit in training and in the interface.

Can we run it on CPUs instead of GPUs? Not for interactive clinical use. A model capable of reliable summarization typically takes tens of seconds per answer on CPUs. Clinicians need an answer within a few seconds, so plan for GPUs.

How do we handle protocols in both Arabic and English? Use a multilingual search model so an Arabic question can retrieve an English protocol and the reverse. The answer comes back in the user's language, with the original-language citation. Build the test set in both languages, because retrieval quality is rarely equal across them. Treat any Arabic summary of an English dosage section as needing extra clinical review before launch.

The decision in front of you is not which model to pick. It is whether your protocol library has clean version control today. If it does not, fix that first. The AI will repeat whatever your document process gets wrong, and say it with confidence.