I need to set a rule in robots.txt to exclude a particular domain. My Composite c1 setup consists of multiple websites/domains... I need to exclude one of them.
Unfortunately, robots.txt needs to be located in the root directory of a domain, so I can not use the rule Disallow: / as it will disallow all the domains in the setup if I place it in the root of C1 install.
I need to be able to target just one domain with robots.txt. Is there a way to do this in Composite C1?
Unfortunately, robots.txt needs to be located in the root directory of a domain, so I can not use the rule Disallow: / as it will disallow all the domains in the setup if I place it in the root of C1 install.
I need to be able to target just one domain with robots.txt. Is there a way to do this in Composite C1?