Do you have more than one website, or rather lots of them? You can now reduce one hassle of the many that goes in maintaining lots of websites, blogs — the ability to place Sitemaps for multiple hosts on a single host through robots.txt.
We know how to include auto-discovery of Sitemaps using robots.txt. Google have announced a new way for Sitemap cross-submissions using Google Webmaster Tools, making it possible to submit Sitemaps for multiple hosts on a single dedicated host.
HOW TO DO IT
For instance, if I want to submit and maintain Sitemaps for each of my blogs —
and to make things easier, I want to host all the Sitemaps on brajeshwar.com.
I can make the best out of cross-submission support by telling the search engines (Google, Yahoo!, Microsoft) where the Sitemaps are with robots.txt —
- The robots.txt for brajeshwar.com would include:
Sitemap: http://www.brajeshwar.com/sitemap-www-brajeshwar.xml - Similarly, the robots.txt for odetoapple.com would include:
Sitemap: http://www.brajeshwar.com/sitemap-www-odetoapple.xml - And for the third one, the robots.txt for blog.oinam.com would include:
Sitemap: http://www.brajeshwar.com/sitemap-blog-oinam.xml
Here are a few other useful notes about the implementation (from Microsoft)
- robots.txt can have multiple “Sitemap:” references.
- Limit the size of robots.txt file to less than 1 MB.
- If multiple sitemaps for a domain include the same URL with conflicting metadata (i.e. priority, change frequency, etc), the metadata will be disregarded and just the URL will be considered.
- Individual sitemap files should never be larger than 10 MB when uncompressed. This includes all sitemap file formats: XML, RSS and Text.
References
- Yahoo! — Making Sitemaps Easier to Manage and Scale
- Microsoft — Microsoft to Support Cross-Domain Sitemaps
Note: For WordPress powered blogs, this may be nothing to worry about as there are good plugins that maintains the Sitemaps auto-magically for you.