Corys8646's avatar

Prevent indexing of staging site

Working on a new site currently, and have a local dev environment, staging site and production site. All 3 are working from the same github repo

I just happened to noticed that Google is crawling my staging site.

How do you handle this in your setup? I know I can create a robots.txt file, but that would be committed to the repo and sent to production.

0 likes
2 replies
MisstypingMeerkat's avatar

I think this is more of a GIT problem. I would add the robots.txt to gitignore, and then maintain a different robots.txt for each enviroment.

2 likes
ejdelmonico's avatar

@Corys8646 Place a robots.txt file in the root that contains

User-agent: *
Disallow: /

Some might ignore it but Google and Bing should not crawl your site.

2 likes

Please or to participate in this conversation.