Google Sitemaps Protocol

Google has introduced an XML schema for describing your site structure, and update frequency, it's called the Google Sitemap Protocol. The protocol is offered under a creative commons license.

The Sitemap Protocol allows you to inform search engine crawlers about URLs on your Web sites that are available for crawling. A Sitemap consists of a list of URLs and may also contain additional information about those URLs, such as when they were last modified, how frequently they change, etc.

The google sitemap protocol is especially useful for websites with Flash user interfaces because google may not otherwise index the links.

It is also handy because you can save bandwidth by telling google the last time the file has changed.

Here's an example sitemap xml file:

<?xml version="1.0" encoding="utf-8" ?>
<urlset xmlns="http://www.google.com/schemas/sitemap/0.84">
	<url>
		<loc>https://www.petefreitag.com/</loc>
		<lastmod>2005-06-06</lastmod>
		<changefreq>daily</changefreq>
		<priority>0.7</priority>
	</url>
	<url>
		<loc>https://www.petefreitag.com/tags/</loc>
		<priority>0.2</priority>
	</url>
</urlset>

A couple of notes on the sitemaps protocol:

• Sitemap files must use UTF-8 encoding
• You can compress sitemap files using gzip
• File can't be larger than 10mb uncompressed
• Sitemaps can't contain more than 50,000 url's
• The priority is relative only to pages on the same site
• The default priority is 0.5
• URL's must be XML Safe (use XMLFormat())

You can find the sitemap protocol specification here.

Google recommends that your store your sitemap in /sitemap.gz, you need manually add your sitemaps to the search engines. Google's page for that is here.