AI Website Agents: Grading 14 Pipeline Claims From Qualified and Intercom

We opened every pipeline figure that Qualified, Intercom and HubSpot publish for their AI website agents and graded 14 of them. Not one is a controlled test. The strongest is a single before-and-after rate: Demandbase says its qualified meeting rate rose from 2.6% to 8.8%. An AI website agent may well lift conversion on your site, but the published evidence cannot tell you by how much.

What AI website agents claim to change

An AI website agent is software that greets a B2B site visitor and answers product questions. It decides whether the visitor fits your target profile and books a meeting with a rep. It replaces the static demo form and the SDR-staffed chat widget. Vendors sell it as an AI SDR, short for sales development representative.

Vendors publish case-study figures, not experiments. They show customers reporting more meetings and pipeline after buying. They do not show how much of that change the agent caused, because no published claim has an unexposed control group.

The category is consolidating. Salesforce signed an agreement to acquire Qualified on 17 December 2025 and completed it on 1 April 2026. Intercom extended its Fin support agent into sales, which SiliconANGLE reported as a new role on 24 April 2026.

So we graded the evidence, using the rubric below. It judges the design of each claim, not whether the number is true.

A figure such as "visitors who engage are 8x more likely to convert" compares people who chose to talk to the agent with people who did not. Those people were already more interested. That is selection, not lift.

Qualified Piper case studies: what the numbers actually measure

Qualified publishes more customer figures than either rival, which is why most of the 14 claims are its. We read 6 Qualified pages in full: four customer stories, Salesforce's write-up of its own deployment, and the customers index.

Demandbase: the best figure in the set

The Demandbase page says the qualified meeting rate grew from 2.6% to 8.8%, a 3.4x increase year over year. The arithmetic checks: 8.8 divided by 2.6 is 3.38. It is a rate, so traffic growth alone cannot inflate it, and it has a baseline.

It is still a grade B, not an A. Neither period is dated, and the denominator is not stated. Anything else Demandbase changed that year, including targeting, pricing or offers, sits inside the 3.4x. The same page claims "$10M+ in pipeline influenced by Qualified programs in the last year", which is a grade D because "influenced" has no stated definition. The Qualified customers index card for Demandbase says "2X increase in pipeline generation", which matches neither number on the detail page.

Greenhouse and Asana: one strong count, two soft claims

The Greenhouse case study carries a subhead claiming meetings booked rose 130%. Its body text says Greenhouse "saw a 91% increase in meetings booked" after switching from Drift. Two numbers for one outcome on one page is a grade D. The same story says that in its first year Piper drove 15,000 conversations, influenced $27M in pipeline and generated $4M in closed-won revenue.

Closed-won revenue is the most useful number here, because a generous meeting definition cannot inflate it. Divide $4M by 15,000 conversations and you get about $267 of closed-won revenue per conversation. That figure is ours, not Greenhouse's, and it inherits an undefined attribution rule. Grade C.

Asana's case study says missed chats, "which had previously totaled 800 over a four-month period", were eliminated entirely. I find this easiest to believe, because an agent that answers every chat cannot miss one. It earns a B. The same page reports "an incremental 22% increase in pipeline" without defining incremental against what, so that one is a C.

Quantum Metric and Salesforce: the vendor grading itself

The Quantum Metric story is headlined 101X ROI, while its results list says "90–100x ROI most months". It also says visitors who engage with Piper are 8x more likely to convert. Both are grade D: the ROI has two values and no cost basis, and the 8x compares self-selected engagers with everyone else.

Salesforce published, on 29 July 2026, an account of running Piper on its own site. It says that since going live in April 2026, Piper achieved a 6% conversion rate and booked more than 60 meetings per week. April 2026 is the month the Qualified acquisition closed. The product's owner is reporting its results.

That makes the figures grade C, not false: the 6% has no stated denominator and no before figure. The customers index card for Salesforce says it "generates $7 million in early-stage pipeline", a number absent from the article it links to. Separately, one person, a SaaStr executive, is credited on index cards for at least six different customers, including Asana, whose own case study names a different contact.

Intercom Fin for Sales: the vendor's own funnel as evidence

Intercom publishes far fewer sales figures than Qualified, but its best one is better designed. Fin for Sales grew out of Fin, the support agent whose economics are covered in the analysis of Fin's per-resolution pricing.

Fin on fin.ai: a dated workload change, with a catch

Intercom's story about its own sales team says Fin has run on fin.ai since November 2025. It reports that the conversations SDRs had to answer fell from 1,275 a week in November to 675 in June. First-response SLA attainment rose from around 75% to 95%, with headcount held steady. That is a dated before-and-after on defined counts. Grade B.

In the same period, the team changed its MQL rule from "anyone who left an email address qualified" to "only prospects meeting sales-led criteria". A narrower definition removes conversations from the SDR queue by itself. The workload fall is real, but some of it is a definition change, and the page does not split the two.

The headline figures are weaker. The page claims "$3M in influenced closed-won revenue, +95% YoY pipeline ARR growth and +44% increase in sales qualified pipeline", with no window and no definition of influenced. Because the qualification rule changed, a year-over-year comparison of qualified pipeline compares two different things. Grade D. The claim that Fin handles around 85% of inbound sales conversations without a human is a coverage figure with no window. Grade C.

Attio: counts you can turn into a rate

SiliconANGLE reported on 24 April 2026 that Attio's agent "handled more than 1,600 inbound conversations, generating more than 50 sales-qualified leads". Divide one by the other and you get a sales-qualified rate of roughly 3.1% per conversation. Both inputs are floors, and there is no baseline, so this is a grade C. It is still more useful than any multiplier, because you can compare it with your own form-to-SQL rate.

What Fin's own dashboard can and cannot report

Intercom's reporting documentation, dated 10 July 2026, lists the metrics the product exposes: conversation volume, contact capture rate, completion rate, a qualification funnel by playbook outcome, and meeting booking conversion where a native calendar integration exists.

Two things are absent: pipeline or revenue attribution, and an attribution window for meetings. So any pipeline figure in a Fin case study was assembled in a CRM by the customer, under rules the case study does not state.

HubSpot: no published website agent pipeline figure

HubSpot is the hardest to grade, because its website agent pages publish support metrics instead of pipeline. The Customer Agent page claims a 70% average resolution rate and prices messaging at $0.50 per resolution. Those are service figures. We found no pipeline or meeting figure for the website agent on the pages we opened.

The nearest sales figures sit on HubSpot's Prospecting Agent page, an outbound email product. Its headline stats claim 76% more qualified leads, 80% more meetings and a 26% higher win rate, with no customer, sample or period attached. Grade D, and the wrong product for this question. One named testimonial there is more useful: RevenueWell says it went from booking 22% of meetings with sequences to 40%.

A secondary claim circulating in search results credits a HubSpot customer with a 28% rise in meetings booked. That figure did not appear on the version of the page we read, so we did not grade it. If your stack is already HubSpot, the credit economics matter more, as covered in the breakdown of Breeze credit repricing.

All 14 AI website agent claims, graded by evidence strength

Each row gives the claim as published, what it measures and its grade. Read the third column before the second.

The three B-grade rows are plotted below.

Independent evidence on AI chat agent conversion

Two independent sources point in opposite directions.

The stronger one is academic. Isabella, Almeida, Duran and Gabler published a paper in the Journal of Business Research in 2025 comparing chatbots with conventional landing pages for B2B lead generation. Across two field experiments with over 16,000 participants, the authors report that chatbots generated significantly more leads, of higher quality, than landing pages.

Three limits apply. The chatbots were WhatsApp-based, not website agents. The effect sizes are not in the abstract, and the paywalled full text went unchecked. And the authors found the effect moderated by purchase complexity and desire for control, exactly what differs between a $500 tool and a six-figure platform.

The counter-case is small and old. In 2020, the agency Articulate Marketing ran a chatbot on its own HubSpot site for two months. Chat converted at 0.2% against about 0.5% for the site as a whole, all nine chatbot contacts were disqualified, and page weight rose by 25%. The agency switched the bot off. That was a decision-tree bot, not a language-model agent, but the page-weight cost applies to any script you add.

Conversational capture probably beats a static form for some buyers in some contexts. Nobody has published how big that effect is for an AI website agent on a B2B software site. The same claimed-versus-measured gap runs through most AI return claims, as covered in the forensic read of AI ROI case studies.

Where this grading is weakest

Case studies are marketing documents. Grading them as research holds them to a standard they never claimed, and a vendor could fairly say so. A missing control group on a web page does not prove none exists. Qualified or Intercom may run holdouts with customers and not publish them.

The sample is also a choice. We graded 14 claims with a detail page or a dated report behind them. Qualified's customers index alone shows dozens more cards, and some may sit on better evidence than the ones we read. Two pages were partly summarised by our fetch tool rather than read verbatim; we marked those claims or left them out.

The rubric is mine, and the boundaries are judgement calls. Asana's missed-chat figure could reasonably be a C, because it measures coverage rather than pipeline. Greenhouse's $4M closed-won could be a B if the attribution rule were published. Move three rows and the chart changes shape, though the empty top row does not.

Finally, the independent evidence is thin. One peer-reviewed paper on WhatsApp bots and one agency blog from 2020 are not a literature. Concluding that AI website agents do not work would overread this post. The finding is narrower: the size of the effect is unknown, so you have to measure it yourself.

What your own AI website agent A/B test should measure

If the published evidence cannot size the effect, your own test has to. Website agents are unusually testable, because the vendor's script loads per visitor. Most vendors can suppress it for a random share of traffic. Ask for that in the contract before the pilot.

Randomise visitors, not weeks

Assign each eligible visitor at random to the agent or to your current form and chat. Do not compare March with April. A before-and-after design picks up seasonality, campaign launches and pricing changes, the weakness of every grade B claim above.

Then count everyone you assigned, including visitors who never opened the chat. Statisticians call this intention to treat. This rule turns the Quantum Metric "engagers convert 8x" claim from a selection artefact into a real comparison.

The metrics that matter, in order

Booked meetings are the metric vendors report and the one most easily gamed by a loose qualification rule. Measure further down the funnel, where an AE has to agree the lead was real. The approach mirrors the measurement problems in the post on attribution and dark discovery.

How much traffic you need

This is where most pilots fail quietly. Let p be your baseline conversion rate. Using the standard approximation for 80% power at a 5% significance level, detecting a 20% relative lift needs roughly 400 times (1 minus p) divided by p visitors per arm. At a 1% visitor-to-meeting rate that is about 39,600 visitors in each arm.

A site with 20,000 eligible visitors a month would therefore need around four months at a 50/50 split to read a 20% lift. That is longer than most vendor pilots. If your traffic is smaller, test a larger effect, or measure an earlier but still honest metric such as AE-accepted leads.

I would not sign an annual contract on any grade D figure above, or accept a pilot that cannot produce a holdout. If the vendor's agent works, a randomised test is the cheapest way for them to prove it. A vendor that resists one is telling you something about its own confidence. The same discipline applies to every vendor proof point, as argued in the piece on case studies with verified outcomes.

Frequently asked questions

Where to start before you sign

Pull your own baseline first. Write down last quarter's visitor-to-meeting rate, meetings held, and AE-accepted leads from your demo form, with the denominator stated. Without that number, every vendor figure in this post is unanswerable, and so is your own pilot.

Then put one clause in front of the vendor this week. Ask for a randomised holdout of at least 20% of eligible traffic for the pilot period, reported on qualified pipeline per 1,000 visitors. Use the sample-size arithmetic above to set the pilot length. If the vendor says the pilot cannot run that long, you have learned what its 30-day results are worth.

Related reading