RAG in Production: What LinkedIn, Uber, Morgan Stanley and DoorDash Published

LinkedIn's retrieval augmented generation system cut median support resolution time by 28.6% over about six months. That is the smallest gain in its own results table. We read the RAG in production write-ups that LinkedIn, Uber, Grab, Morgan Stanley and DoorDash engineers published. Of the four we could open and read in full, none reports an evaluation set size, an absolute quality score and a production outcome together.

The short answer

Published RAG case studies show real gains, mostly reported as relative improvements on test sets of unstated size. The repeatable lessons are structural. Fix document parsing and metadata before the model, build a golden set with subject experts, calibrate an LLM judge against it, and route failed answers to a human or a simpler retriever.

Seven named RAG deployments, side by side

Vendor demos of retrieval augmented generation are everywhere. Engineering write-ups with an evaluation method and a number attached are rarer. We found seven named deployments worth tabulating.

RAG means the model answers from documents or records retrieved at query time, rather than from what it memorised in training. Nearly every team below spent its effort on the retrieval step. The table records what each built, how it measured quality, what it reported, and whether we could open the source.

Sources: company engineering blogs and the LinkedIn SIGIR paper, retrieved 28 October 2026. Rows marked unverified carry figures we could only see restated by third parties, which is one source repeated, not corroboration.

Two things stand out. Only one deployment, LinkedIn's support tool, compares users against a control group. And the three most-quoted figures in RAG marketing decks, DoorDash's 90% and 99% and Morgan Stanley's 98%, are secondary restatements or an adoption rate, not a quality measure.

LinkedIn's 28.6% is the most conservative number in its own results

The LinkedIn paper by Zhentao Xu and six colleagues is the strongest public evidence that enterprise RAG changes an operational outcome. It was presented at SIGIR 2024, a peer-reviewed information retrieval conference. That puts it a tier above any engineering blog here.

The system treats each historical support ticket as a small tree, then links tickets to each other through explicit references and title similarity. A query is matched to the closest tickets, rewritten into a graph query, and answered from the retrieved subgraph. If the graph query fails, the system falls back to plain text retrieval.

A randomised split, which almost nobody else ran

LinkedIn split its support team randomly into two groups, one using the tool and one working manually. That design is what makes the result worth quoting. Most RAG case studies compare a new pipeline with an old one on a fixed test set. That tells you about answer quality and nothing about the business.

The paper's results table gives three statistics for each group. We recomputed the reduction for each one, and the headline uses the smallest.

I think the authors led with the right statistic. A mean falling from 40 hours to 15 hours is pulled by a few very long tickets, and a vendor would have quoted it. The median is the honest choice, though it undersells where the tool helped most.

Retrieval scores near the ceiling, on a test set of unstated size

The offline numbers are striking. Mean reciprocal rank, which rewards putting the right ticket first, rose from 0.522 to 0.927, the 77.6% gain the abstract reports. Recall at 3 went from 0.640 to a perfect 1.000.

A perfect score should prompt a question about the sample. The paper describes a curated golden dataset of typical queries and authoritative solutions. It does not state how many queries it holds. It also gives no group sizes for the production split and no significance test. The indexing cost of building a graph like this is a separate question, covered in the comparison of GraphRAG and standard RAG indexing costs.

RAG architecture examples: the fix was usually the corpus, not the model

Read together, the seven write-ups show a pattern. The teams with the largest improvements changed how documents were parsed, split and labelled. Very few reported a gain from swapping the model.

LinkedIn's paper names the problem directly. Fixed-length chunking can separate a ticket's problem description from its solution, so retrieval returns half an answer. Turning each ticket into a structured tree was the fix. The generation model was GPT-4 in both the baseline and the new system.

Uber rebuilt parsing before it touched prompts

Uber's Genie answers engineers' questions about 40+ security and privacy policy documents. Its first RAG version gave answers the team described as incomplete or inaccurate. The cause was mundane: standard PDF loaders turned multi-page tables into stand-alone cells with no row or column context.

The rebuild moved to a custom Google Docs loader, converted tables to markdown with an LLM, and kept table chunks intact. Each document gained a summary, generated FAQs and keywords. Retrieval became a union of vector search and BM25 keyword search. That matches what independent benchmarks show about hybrid search beating pure vector retrieval.

Two agents were added before retrieval. One rewrites ambiguous queries and splits compound ones, and one narrows the search to the documents most likely to hold the answer. Similar chunks across different policies had been pulling in irrelevant context.

Grab and Pinterest retrieve queries and tables, not passages

Grab's Integrity Analytics team built something most people would not recognise as RAG. Its Data-Arks platform packages frequently used SQL queries and Python functions as APIs. The retrieval step chooses which pre-written query to run, and the LLM summarises the result in Slack.

Pinterest's text-to-SQL work, as described in secondary summaries, applies the same idea to table selection. A vector index of table summaries picks candidates before the model writes SQL. In both cases the thing retrieved is a trusted artefact. That removes a whole class of hallucination at source. I would start most internal analytics RAG projects this way rather than with document chat.

How production RAG teams evaluate: golden sets, judges and annotators

Every team that described its evaluation used some mix of three tools. A golden set of questions with expert answers, an LLM acting as judge, and humans reviewing live traffic. What differed was the balance and how much was disclosed.

The subject-expert golden set plus an LLM judge

Uber's approach is the most fully documented. Subject experts curated a golden set of 100+ test queries with their own answers. An LLM judge then compares each new answer against the expert answer and the source documents, scoring it from 0 to 5 with written reasoning.

The payoff was speed. Uber says automated evaluation cut experiment turnaround from weeks of expert review to minutes. That mattered because the team had plateaued after small improvements. Faster evaluation made more experiments possible.

An LLM judge you have not checked against your experts is a second unverified model, not an evaluation. The published studies on how often LLM judges agree with human raters show why that calibration step cannot be skipped.

Human annotation still sets the bar

LinkedIn's generative AI team ran evaluation at three speeds. Engineers got fast, coarse automated checks. Annotators scored up to 500 conversations a day on quality, hallucination rate, responsible AI violations, coherence and style, with about a one-day turnaround. Member feedback took three days or more per change.

The team called automatic evaluation the holy grail and said it was still a work in progress. It reported some success on hallucination detection. That was April 2024, and it remains the most candid statement any company here has made about its eval stack.

According to OpenAI's case study, Morgan Stanley tested each use case before launch. Advisors and prompt engineers graded answers for accuracy and coherence. The firm has not published the grading rubric, the sample, or the pass threshold. To design your own, see the guide to building an LLM eval suite.

Enterprise RAG quality plateaus early, and the last points cost the most

The most useful planning number in this whole set comes from LinkedIn. The team reached 80% of the basic experience it wanted within the first month. It then spent another four months attempting to surpass 95% of the full experience.

Put as a share of elapsed time, roughly 80% of the five-month build went on the last stretch of quality. The team wrote that quality scores shot up at first, then quickly plateaued. It had also underestimated how hard hallucination detection would be.

Uber described the same shape from the other side. Before its parsing rebuild, experiments yielded only slight accuracy improvements before plateauing. The plateau broke when the team changed the corpus, not when it changed prompts.

My recommendation for planning is blunt. If your proof of concept took one month, budget four more before you promise users production quality. The tests that separate a pilot from a production system belong in that second phase, not after it.

Guardrails and fallbacks decide what users see when retrieval fails

Every serious deployment here assumes retrieval will sometimes fail and designs a path for it. The teams differ in where that path leads.

LinkedIn's support system falls back to plain text retrieval when its generated graph query fails. LinkedIn's member-facing team found that about 10% of model responses had malformed parameters. A defensive parser brought those errors down to about 0.01%. Neither fix is glamorous, and both are the reason the systems stayed up.

DoorDash, according to secondary summaries of its September 2024 post, checks each answer with an online guardrail before a Dasher sees it. Answers that fail are not shown, and the conversation defaults to a human agent. That is the right instinct, and it matches human-in-the-loop designs that survive production.

Guardrails cost latency, and LinkedIn quantified a version of that cost. After adding a chain-of-thought step, the team found that for a 200-token reasoning step, a 10ms rise in time between tokens adds about 2 seconds. It triggered a timeout alert on a public ramp. Any check that runs before the user sees an answer sits directly on that clock.

The disclosure gap: what no RAG case study reports

We scored the four write-ups we could read in full against four disclosures you would need to check a claim. The result is the original finding of this post: no team discloses eval set size, an absolute quality score and a measured production outcome together.

This is not a complaint about the engineers. A blog post is not a filing, and nobody owes the public a test set. It does mean every RAG case study, good ones included, shows only that a method can work. None is a benchmark you can hold your own system or a vendor's to.

Use this table as a procurement checklist. Studies of which vendor case studies carry verified outcomes find the same gaps in AI marketing generally. RAG is not special here; it is just newer.

Where this evidence is weakest

The argument above rests on a small, self-selected set of documents. Three problems limit how far it can be pushed.

Published write-ups are a survivor sample

Companies publish engineering posts about systems that worked well enough to be proud of. Abandoned RAG projects do not get blog posts. So the plateau-then-breakthrough story in this set may be common precisely because projects that never broke through are invisible.

The counter-case deserves a fair hearing. Missing disclosures may reflect legal review or competitive caution rather than weak results. A bank like Morgan Stanley has good reasons not to publish its grading rubric. Its 98% adoption figure is not a quality measure, but it signals that advisors found the tool useful enough to keep opening.

Three of seven primary sources could not be opened

DoorDash's and Pinterest's original posts, and OpenAI's Morgan Stanley case study, refused automated access during this research. The Morgan Stanley adoption figure appears in the firm's own 2025 awards release and in OpenAI's case study. We confirmed both from indexed page text. DoorDash's 90% and 99% figures are widely repeated. Every copy we found traces back to the same unread post, so we treat them as unverified.

The verified evidence is also concentrated. Two of the four fully read sources are from LinkedIn, and every relative figure from Uber lacks a published baseline. Treat the lessons as directional patterns from a handful of engineering teams, not as measured industry rates. If your problem is long documents rather than many, the trade-offs in diagnosing RAG retrieval failures against long context may matter more.

Frequently asked questions

Where to start this week

Before you change a retriever, a chunk size or a model, write 100 questions your users actually ask and get a subject expert to answer each one. That is Uber's golden set size, and it takes days rather than a quarter. Without it, every later improvement is a guess.

Then record your baseline as a distribution. Log the median, mean and 90th percentile of whatever outcome you care about, resolution time, escalation rate or answer acceptance, before the system ships. LinkedIn's paper shows those three can tell very different stories. You only get to measure the before state once.

Related reading