SentientBot

SentientBot is the web crawler of Sentient AI Systems Inc. If you've found it in your server logs as SentientBot/1.0 (+https://uk.sentientaisystemsinc.com/bot), this page explains what it was doing and how to stop it.

What it reads

It visits UK business websites to keep a directory of companies accurate. On each site it reads at most five public pages: the home page, and the contact, about, legal (privacy or terms) and team pages linked from it. From those it takes business details that the site publishes:

• the company's registered name and number, and its VAT number
• business addresses, phone numbers and email addresses
• links to the company's own social media pages

It doesn't log in, fill in forms, follow links to other websites, or download files other than web pages.

How it behaves

• It reads your robots.txt first and obeys it. Rules for SentientBot take priority; if there are none, it follows the rules for *.
• It makes one request to your site at a time, a few seconds apart, and usually visits a site no more than once every few months.
• It always gives its name. It never pretends to be a web browser.

To block it with robots.txt:

User-agent: SentientBot
Disallow: /

Ask it not to visit your site

Enter your website below. SentientBot won't visit it again, and we'll delete the pages we have stored from it.