Rewrite robots.txt based on host with htaccess
I found my own answer. This works as desired.
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} ^(.*)?mydevserver(\.com)
RewriteRule ^robots\.txt$ robots-staging.txt [NS]
</IfModule>
You can use a more dynamic approach:
RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [NS]
And place your robots.txt files like follows:
- robots/domain.tld.txt
- robots/sub.domain.tld.txt
I came across this solution due to some multi website projects based on TYPO3 CMS and Neos CMS.