New Post: Composite c1 and robots.txt

You would solve this using default asp.net tech - Composite C1 doesn't have a feature which solve this specific task. A good idea though - suggest a feature at https://compositec1.codeplex.com/workitem/list/basic

See for example http://www.kleenecode.net/2012/08/16/dynamic-robots-txt-with-asp-net-in-iis7/ for an asp.net fix.