Bots crawling

Hi,

I've just pushed my first website through Laravel Forge on a live server (DO droplet) a few days ago and I've got a lot of requests from bots crawling it for vulnerabilities already. They're trying all sorts of routes from /wp-admin to /db/mysql, /phpmyadmin and /.env

Is there anything I should do to further secure my app? It's a small site for a non-profit, so it'll have few visitors, but it does process payments through Stripe.

cwhite

3 years ago

Level 19

Do you have a robots.txt file in the public folder (i.e., web root)? You can set this up to tell most webcrawlers not to crawl all/portions of your site:

https://moz.com/learn/seo/robotstxt

However, do note that:

Some user agents (robots) may choose to ignore your robots.txt file. This is especially common with more nefarious crawlers like malware robots or email address scrapers.

Please or to participate in this conversation.