ShowUpLabsBot

You have probably arrived here from a line in your access logs. This page tells you exactly what our crawler does, and how to stop it in one line if you would rather it did not.

Stop it right now

Add this to /robots.txt. It takes effect on our next run, within about a day, and we cache your robots file for the length of a single run rather than indefinitely.

User-agent: ShowUpLabsBot
Disallow: /

You do not have to email us or fill in a form. We check before we fetch, so a disallowed page is one we never request, rather than one we request and discard. If you would rather tell a human, the contact form works too, but the robots line is faster and does not depend on us reading anything.

What it actually does

Two jobs, both small.

• Checks whether a business is named on a page. We measure which third-party pages influence what AI engines say about a category, then check whether a given company appears on them. That means fetching the page, searching the text for a name, and keeping the yes or no. We do not store your page.
• Reads a page to find who publishes it. When we are working out whether a roundup has an author or a submissions process, we read the byline and the contact links. Same fetch, nothing extra.

What it does not do: it does not crawl your site (typically one URL per domain per week), it does not follow every link it finds, it does not fetch anything requiring a login, it does not submit forms, it ignores anything behind a paywall rather than working around it, and your content is never used to train a model. We do not train models.

Rate

• At least 4 seconds between requests to the same host, and more if your Crawl-delay says so.
• Typically one page per domain, once a week. If you are seeing more than that from us, it is a bug and we would like to know.
• A 403, 401 or 429 is treated as a no. We record it and move on rather than retrying around it.
• If your robots.txt returns a 5xx we treat that as a no as well, because an ambiguous answer should not be read as permission.

Why we are careful about this

We sell crawler-access auditing. Our own product tells businesses whether AI crawlers can reach them and which ones they are blocking. Running a badly-behaved crawler while selling that would be indefensible, so the standard we hold ourselves to here is the one we would apply to anyone else.

There is a wider reason too. Bot and crawler traffic now exceeds human traffic on the web, and sites with something to lose have reasonably started shutting the door: roughly 60% of high-credibility news sites now block AI crawlers, and long-standing testing organisations disallow the major AI agents outright. That is a rational response to being scraped, and the crawlers that caused it are the ones that took without asking. We would rather be the kind that asks.

If we have got something wrong

If ShowUpLabsBot is hitting you more than described, ignoring your robots.txt, or appearing where you did not expect it, tell us and we will fix it and say what happened. We would rather be corrected than be confidently wrong, which is the same offer on every report we publish.

Questions, answered