CamoDocs: How Stealthy Poisoning Attacks Bypass RAG Security Filters

For anyone running a RAG stack in production, this is the kind of read that makes you want to grep your own vector store before lunch.

The attack hides adversarial content inside otherwise benign documents, and — here is the gotcha — it never puts the target query into the corpus directly. That single design choice sidesteps a whole class of defenses that scan for trigger keywords at retrieval time, which is why this paper deserves attention from anyone shipping retrieval into user-facing products.

What the paper actually tests

Per the abstract on arXiv, the authors evaluated CamoDocs against seven RAG defenses, three open-weight models, and proprietary models. Code is released with the paper, so we can reproduce the numbers on our own hardware instead of trusting a single leaderboard cell — exactly the workflow we want when validating a security claim.

The interesting engineering question is which defense categories actually degrade. Without the full text in front of us, the working hypothesis is that retrieval-time anomaly scoring, lexical filters, and surface-level query matching will struggle, since the poisoned chunks are designed to blend into the topical neighborhood of the target. Semantic similarity scoring, provenance weighting, and hybrid filters are the categories worth measuring — not assuming.

A practical checklist for your pipeline

Before this gets buried under the next wave of model releases, here is what to action this week:

• Audit recent ingestions for documents that look topically tight but contain no expected terminology from your users' queries. That is the shape CamoDocs exploits — chunks that pass a topical relevance check but carry no recognizable trigger language.
• Run the released CamoDocs code against your own retriever with a held-out evaluation set. Log attack success rates per defense so the comparison is real, not vibes.
• Add a sanity check that flags chunks with unusually high similarity to many unrelated queries. Not a silver bullet, but a measurable signal worth instrumenting in production.
• Revisit your trust scoring. If you weight by source metadata or provenance, log those weights explicitly so poisoning attempts leave a trail we can actually trace back.
• Pin your stack versions and, if you are on a hosted retriever, ask the vendor what they have done about this attack class — silence is an answer.

We will dig into the released repository next week and post reproducible numbers once the suite is running locally on our cluster. Expect a follow-up with the actual defense-by-defense breakdown.