You deployed AI. Does it actually work?
An independent, fixed-fee audit of a chatbot, knowledge system, or AI agent you already run - whether we built it or someone else did. We test it against your real documents and real traffic patterns, then hand you a written report your leadership and compliance team can act on.
Fixed fee, quoted before work begins · Written report in 10 business days · NDA signed same day
When you need this
- A vendor built you a chatbot or RAG system and you have no independent verification of its accuracy
- You are about to renew, scale, or expand an AI system and need evidence it deserves the budget
- Compliance or legal asked "how do we know it doesn't make things up?" and nobody has a documented answer
- Answers look fluent but customers or staff report wrong information you cannot reproduce
- You are evaluating an acquisition or vendor proposal that claims AI capability
What we test
Grounding & hallucination rate
We run the system against a question set built from your real documents and score every answer: supported by the source, partially supported, or invented. You get the rate, not an anecdote.
Security exposure
Prompt injection, system-prompt extraction, data leakage across users or tenants, and whether the system can be manipulated into unauthorized actions or statements.
Cost & scaling economics
True cost per query today and projected at 10x volume, including hidden retry and context costs. Whether the current architecture survives your growth plan.
Failure modes & edge cases
What happens on ambiguous questions, out-of-scope requests, Arabic/English mixing, long documents, and adversarial phrasing. Where it should say "I don't know" but doesn't.
Retrieval quality (for RAG systems)
Whether the right documents are actually being found and cited. Wrong-document retrieval is the most common silent failure in deployed knowledge systems.
Operational readiness
Monitoring, logging, fallback behavior when the model provider has an outage, and whether anyone would notice quality degrading next month.
What you receive
- A scored written report: grounding rate, security findings by severity, cost analysis, and failure inventory
- An evidence appendix: every failed case documented with the exact question, answer, and source - reproducible by your team
- A prioritized fix roadmap: what to fix first, what it takes, and what can wait
- A 60-minute readout call with your technical and business stakeholders
How it works
Scoping call (30 min)
You describe the system and what worries you. We define the audit scope and quote a fixed fee. NDA signed the same day if needed.
Access & sampling
We get read access to the system (or a staging copy) and a sample of the real documents it should be answering from.
Testing (10 business days)
Structured evaluation runs, security probing, and cost analysis. No changes are made to your system.
Readout & roadmap
Written report delivered, walked through live. What you do with it - fix in-house, hire us, or hire anyone else - is entirely your call.
Why independent matters
The team that built a system should not be the one grading it. An audit from the original vendor is a progress report; an audit from an outside engineering team is evidence. Our method is public: the same grounding analysis powers our free Trust Checker tool, which you can try on your own documents right now, before ever talking to us.
Pricing
Fixed fee, quoted at the scoping call once we know the system count and scope. Typical single-system audits fall between $3,500 and $7,500. The fee is agreed in writing before any work begins - no hourly billing, no surprises.
Frequently asked questions
Do you audit systems built by other vendors?
Yes - that is the most common case. We audit systems regardless of who built them. The report is written so you can hand it to your existing vendor as a fix list, use it in a renewal negotiation, or bring it to any engineering team.
Do you need access to our production data?
We need read access to the system and a representative sample of the documents it answers from. Where data cannot leave your environment, the entire audit can run inside your infrastructure - we are an on-premise deployment firm and work under that constraint routinely.
Will the audit disrupt our live system?
No. Testing is read-only: we send questions and record answers, the same way a user does. Security probing is scoped and agreed in advance, and can run against a staging copy if you prefer.
What if the audit finds the system is fine?
Then you have documented, independent evidence that it works - which is exactly what compliance, leadership, or an acquirer wants to see. A clean audit is not a wasted audit.
Is this a sales pitch for a rebuild?
The report stands on its own and names the cheapest viable fix for each finding, including "keep your current vendor and have them fix these three things." If a rebuild is genuinely the right answer we will say so, and you are free to run that tender with anyone.
Can you audit Arabic-language systems?
Yes. We build Arabic and bilingual systems natively, and audit Arabic answer quality, dialect handling, and Arabic-specific failure modes that English-only teams miss.
Get evidence, not assurances
A 30-minute scoping call, a fixed fee in writing, and a report your board can read. Whether we ever do more work together is up to you.
Book a Scoping Call