Quantcast
Channel: C1 CMS Foundation - Open Source on .NET
Viewing all articles
Browse latest Browse all 2540

New Post: http://www.mysite.com/en-US/sitemap.xml is showing pages where "Menu item" is empty.

$
0
0
Sitemaps.xml tells the search engines what pages exist in your website, but ~/robots.txt tells them what to index. The two files are complementary. Almost all search engines will respect your robots.txt instructions.

http://www.robotstxt.org/robotstxt.html

You can also use rel="nofollow" attributes on hyperlinks, or include <meta name="robots" content="noindex, nofollow" /> in your page metadata to prevent search engines indexing specific individual pages.

Viewing all articles
Browse latest Browse all 2540

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>