Keep AI Crawlers or Search Engine Crawlers Out of Your Squarespace Website

You’ve created some great content for your website. Maybe you’ve also created some art or photographs to display or sell. Now you have to decide who gets access to it.

Do you want it to appear on search engines like Google or Bing? Do you want to allow AI crawlers to use your information? Do you only want your visitors to see it?

This is something a lot of us are thinking about right now. Most of us do want to be indexed on search engines, but as for AI, that’s another story. Many major websites are now blocking AI crawlers and some AI companies are being sued for using copyright protected materials.

Table of Contents

So, what are AI Crawlers and Search Engine Crawlers?

Crawlers are a type of bot that systematically scan website content all over the internet.

Web crawlers or search engine crawlers use this information to index web pages in their search results. So, when you go to Google and ask it a question it send you to the website with the information you requested.

AI crawlers use the information from the scanned content to train their AI, to improve their output.

---

How to Hide Your Squarespace Website from AI Crawlers

Would you prefer to have your website hidden from AI crawlers? Squarespace has recently added an easy way for you to do this for some AI crawlers: Anthropic AI, CCBot, Google Extended, GPTBot and ChatGPT-User.

1. From your Home Menu click Settings

2. In the Website menu click Crawlers (it’s at the bottom of the list)

3. Click the checkbox next to Block Known Artificial Intelligence Crawlers

4. Save

I’m hoping that at some point Squarespace decides to have AI crawlers blocked by default. I think you should have to agree to have your data used before they can start. Squarespace says that they don’t do this as it could impact your traffic, so they leave the decision to you.

Here is a list of the AI crawlers that will be blocked by clicking the checkbox:

• Anthropic AI

• Applebot-Extended

• CCBot

• Claude-Web

• cohere-ai

• FacebookBot

• Google Extended

• GPTBot and ChatGPT-User

• PerplexityBot

---

*Hiding your site from AI crawlers won’t remove any information from the crawlers that they received before you checked the box.

---

How to Hide Your Squarespace Website from Search Engine Crawlers

There are a couple of ways to hide your Squarespace website from web crawlers or search engine crawlers. Which way is best for you depends on who else can access your website.

To Hide Your Website Search Engines & Visitors

If you want access to your website to be accessible only to contributors (who can login to the website) you’ll want to set your website to private. This is the default setting for all unpublished websites.

1. From the Home Menu click Settings

2. Click on Site Availability

3. Choose Private and Save

To Hide Your Website Search Engines but Allow Certain Visitors

You can password protect your website to allow only visitors with the password to access your website. It will also be hidden from web crawlers. When they visit your website, visitors will see a lock page where they can enter the password.

Set your website to private:

1. From the Home Menu click Settings

2. Click on Site Availability

3. Choose Password Protected and enter a password into the box

4. Save

To Hide Your Website Search Engines but Allow All Visitors

This option will allow all visitors, but won’t allow the website to be crawled by search engine bots.

1. From your Home Menu click Settings

2. Click Crawlers

3. Click the checkbox next to Block Search engine crawlers

4. Save

---

*These options will stop your website from being indexed from this point on, but if your website has already been indexed by the search engines it can take a while before those pages get removed from search.

---

How to Hide Individual Pages from Search Engines

Specific pages on your website also have the option to be password protected, hidden from search engine crawlers or hidden from everyone. You can also create member only pages using Member Sites.

Parts of a collection (such as blog posts) can’t be hidden. You can only hide the entire collection.

Hide a Page from Visitors and Search Engines

To hide a page from both visitors and search engine crawlers you will need to disable the page. You can use this option if your page is under construction or you want to temporarily hide your page.

I generally prefer to use a password on my page if it’s under construction. It helps visitors know that the page is coming back and they won’t end up on the 404 page.

To disable a page:

1. From your Home Menu click Website

2. Hover over the page you want to disable and click the cog icon

3. In the Page Settings toggle Enable Page to the off position (grey)

4. Save

Hide a Page from Search Engines Only

This option leaves the page visible to all of your visitors, but doesn’t allow search engines to index your page.

1. From your Home Menu click Website

2. Hover over the page you want to disable and click the cog icon

3. In the Page Settings choose SEO

4. Toggle Hide Page from Search Results to the on position (green)

5. Save

Pssst, if you want to know all about password protecting a page, lock screens and more, check out my related post: How to Password Protect a Single Page in Squarespace.