Glossary · Technical
robots.txt.
A file at the root of a domain telling crawlers which parts of the site they may request. Well-behaved crawlers honour it; it is not a security measure.
Why it matters.
Two mistakes are common: blocking the whole site during development and forgetting to unblock it at launch, and blocking CSS or JavaScript, which prevents search engines from seeing the page as a visitor does. It also now carries a decision most owners have not consciously made — whether AI crawlers may read the site.
How to check it yourself.
Open yourdomain.com/robots.txt. Anything disallowed there should be deliberate. A "Disallow: /" left over from a launch is a well-known way to disappear.
Questions
Straight answers.
How do I check robots.txt on my own site?
Open yourdomain.com/robots.txt. Anything disallowed there should be deliberate. A "Disallow: /" left over from a launch is a well-known way to disappear.
Now check your own
You have read the numbers.
Where does your site stand?
Type your address and we will open your site the way a visitor would, then tell you plainly what is wrong with it — including when the answer is nothing. Ten seconds, no email, nothing stored.
If you want
one built.
We build the site, you look at it, and you pay once it is finished and you are happy with it. Hosting, edits and the Google listing carry on monthly, cancellable any month.