When AI Agents Visit Your Website: Getting Ready for Agentic Browsing

For thirty years the assumption behind every website has been that a human would look at it. That assumption is quietly breaking. A growing share of first visits are made by software gathering facts on somebody's behalf, and the businesses that get shortlisted are increasingly the ones whose information can be extracted cleanly rather than the ones whose sites look best.

Why this matters now

The behaviour to picture is a buyer who says: find me three commercial cleaning companies covering Bristol, tell me who publishes pricing, and which ones handle out-of-hours work. Something then visits a set of websites, extracts what it can, and returns a comparison. The buyer reads the comparison, not the websites.

The consequence is a filter you never see. If your site was visited and nothing useful could be extracted, you are simply absent from the answer. There is no bounce to investigate, no impression logged in a report you read, no signal that anything happened. The first shortlist increasingly gets built without you, and the failure is silent by design.

What matters is that automated visitors are looking for facts, not impressions. They want to know what you do, where you do it, who for, roughly what it costs, how quickly you respond, and whether anything corroborates your claims. They are largely indifferent to your photography, your animation, your carefully worked headline and your brand voice. Those things still matter enormously for the human who arrives later, which is why this is an addition to your priorities rather than a replacement.

The good news is that this is not a new discipline requiring new skills. Everything that helps an automated visitor also helps a human in a hurry: plain statements of fact, information as text rather than pictures, no essential detail hidden behind a click. The work overlaps almost entirely with good clarity practice, which is why it is comparatively safe to do.

The related but distinct problem is what these systems say about you when they are not on your site at all, which draws on reviews, directories and third-party mentions. We covered that separately in what AI says about your business. This piece is about the visit itself.

Common mistakes to avoid

Almost every failure here is accidental. Nobody sets out to make their pricing unreadable. It happens as a side effect of decisions that looked reasonable at the time.

• Facts locked inside images. A price list, a service area map or an opening hours graphic contains no extractable text. This is the single most common failure and it also fails screen reader users, so it was already a problem before any of this.
• Key information behind an interaction. Details inside an accordion, a tab, a modal or a hover state may never be reached. If your service areas only appear after clicking Read more, assume they will be missed.
• Blocking automated visitors indiscriminately. Bot protection and aggressive robots.txt rules increasingly block the systems your buyers are using. Blocking scrapers and blocking your prospect's research assistant can be the same action, and most businesses have never checked which they are doing.
• Requiring a form for basic information. Gating your price guide or service list behind an email capture means it does not exist as far as an automated visitor is concerned. You are trading shortlist presence for a lead magnet, usually without realising.
• Vague qualifying statements. Competitive pricing, fast turnaround and nationwide coverage are unextractable. From 400 pounds, same-day response, and covering the M25 corridor are facts. The first set says nothing, the second gets you compared.

There is a subtler failure worth naming. Some businesses respond by writing pages aimed squarely at machines, full of flat declarative statements and no personality. This wins the extraction and loses the human who arrives next. The correct target is a page that states its facts plainly and then makes the case persuasively, which is what good service pages did anyway.

The deeper mistake is treating this as an optimisation exercise rather than a clarity one. If your service page cannot answer what you do, who for, where, roughly what it costs and what happens next, an automated visitor will fail to extract it, and so will a distracted human on a phone. Our website copy framework covers the underlying structure.

Step-by-step plan

This takes a fortnight and is mostly editorial rather than technical. Do it in order.

1. List the ten questions a buyer asks before shortlisting. What do you do, who for, where, how much roughly, how fast, what is the process, what qualifications, what proof, how to start, what happens next. Write them down. This list is the specification for everything below.
2. Answer each one in plain text on your site. Not in an image, not behind a tab, not in a downloadable PDF. Actual text on an actual page. If a question has no answer anywhere, that is a gap worth more than any technical fix.
3. Run the JavaScript-off test on your top five pages. Whatever disappears is invisible to a meaningful share of automated visitors. Move anything essential into the base HTML.
4. Get every fact out of images. Prices, service areas, opening hours, accreditations, phone numbers. Keep the graphic if it helps humans, and put the same information in text alongside it.
5. Audit what you are blocking. Check robots.txt and any bot protection or firewall rules. Decide deliberately which automated visitors you want to allow. Most businesses discover they are blocking things they would rather permit, purely by default.
6. Replace qualifying adjectives with numbers. Go through your service pages and change every vague claim into a specific one. Fast becomes a stated timeframe. Affordable becomes a starting price or a range. Nationwide becomes a named list of regions.
7. Add structured data for the basics. Organisation, LocalBusiness, Service and FAQ markup where genuinely applicable. This is helpful and it is secondary. Clean readable text matters more, and structured data describing things not visible on the page is worse than none.
8. Publish a real pricing statement. Not necessarily a price list. A pricing model, a starting point, or a worked example. Businesses that publish nothing get filtered out of comparisons that businesses publishing a range survive. Our guide to presenting prices without losing customers covers how to do this without commercial exposure.
9. Make contact routes machine-readable. A real tel link, a real email address, a form that does not require JavaScript to discover. Not a chat widget as the only route.
10. Test it. Ask an assistant to compare your business against two named competitors on the criteria a buyer would use. Whatever it gets wrong, or cannot find, is your remaining work.

Step ten is the honest audit and most businesses skip it because the result is uncomfortable. It is also the only step that tells you whether the previous nine worked. Do it before you start and again after you finish, and keep both answers.

Step seven deserves a caveat. Structured data is genuinely useful and it is not a substitute for the content itself. Marking up a price that appears nowhere on the visible page is the kind of shortcut that eventually gets treated as manipulation. Describe what is actually there.

Agentic readiness checklist

Run this against your main service pages.

• All ten pre-shortlist questions are answered in plain text somewhere on the site.
• No essential fact exists only inside an image.
• Key information is not hidden behind tabs, accordions or modals.
• The page still communicates its core facts with JavaScript disabled.
• robots.txt and bot protection rules have been reviewed deliberately.
• Vague claims have been replaced with numbers, ranges or named places.
• A pricing model or starting point is published.
• Contact routes include a real tel link and a real email address.
• Structured data describes only what is actually on the page.
• An assistant has been asked to compare you against two competitors, and the result reviewed.

How to measure impact

This is harder to measure than most work because the filter is invisible. Use a mix of direct testing and indirect signals, and do not expect a clean dashboard.

The comparison test, repeated. Ask an assistant to compare you against named competitors on buyer criteria, once a quarter, and save the answers. Whether you appear, and whether the facts are right, is the closest thing to a direct measure available.

Server logs for automated traffic. Your logs record what visited. Rising automated visits from assistant-associated sources indicate you are being included in research, which is the thing you are trying to achieve.

Extractable fact coverage. A simple internal count: of the ten pre-shortlist questions, how many are answered in plain text? Going from four to ten is real progress you can report even before any external signal moves.

Enquiry quality and specificity. Watch for enquiries that arrive already knowing your pricing model or process. That pattern suggests the research happened successfully before contact, which is exactly what you want.

Self-reported attribution. Keep asking how people found you. Answers mentioning an assistant have gone from rare to routine, and they are the most direct evidence you will get.

Key terms in plain English

Agentic browsing: Software visiting websites and taking actions on a person's behalf, rather than a person browsing directly.

Extraction: Pulling specific facts out of a page. What automated visitors do instead of reading.

Structured data: Machine-readable markup describing your business, services and content in a standard format. Helpful, secondary to clear text.

Rendering: Running JavaScript to produce the final page. Some automated visitors do this, many do not, so content requiring it may be invisible.

Shortlisting: The stage where options are narrowed before a human looks closely. Increasingly automated, and where silent failure happens.

robots.txt: A file telling automated visitors which parts of your site they may access. Frequently more restrictive than its owner realises.

Conclusion and next move

The change here is not that websites stop mattering. It is that a filter has appeared in front of them, and the filter reads facts rather than impressions. Businesses that state plainly what they do, where, for whom and roughly at what cost will pass it. Businesses whose key information lives in a graphic or behind a form will not, and will never see the enquiries they did not get.

Start with the JavaScript-off test on your main service page. It takes two minutes, requires nothing but a browser setting, and shows you immediately how much of your case survives when the presentation is stripped away. Whatever disappears is the work.