Google Sitemaps Protocol
Google has introduced an XML schema for describing your site structure, and update frequency, it's called the Google Sitemap Protocol. The protocol is offered under a creative commons license.
The Sitemap Protocol allows you to inform search engine crawlers about URLs on your Web sites that are available for crawling. A Sitemap consists of a list of URLs and may also contain additional information about those URLs, such as when they were last modified, how frequently they change, etc.
The google sitemap protocol is especially useful for websites with Flash user interfaces because google may not otherwise index the links.
It is also handy because you can save bandwidth by telling google the last time the file has changed.
Here's an example sitemap xml file:
<?xml version="1.0" encoding="utf-8" ?> <urlset xmlns="http://www.google.com/schemas/sitemap/0.84"> <url> <loc>https://www.petefreitag.com/</loc> <lastmod>2005-06-06</lastmod> <changefreq>daily</changefreq> <priority>0.7</priority> </url> <url> <loc>https://www.petefreitag.com/tags/</loc> <priority>0.2</priority> </url> </urlset>
A couple of notes on the sitemaps protocol:
- Sitemap files must use UTF-8 encoding
- You can compress sitemap files using gzip
- File can't be larger than 10mb uncompressed
- Sitemaps can't contain more than 50,000 url's
- The priority is relative only to pages on the same site
- The default priority is 0.5
- URL's must be XML Safe (use XMLFormat())
You can find the sitemap protocol specification here.
Google recommends that your store your sitemap in
/sitemap.gz, you need manually add your sitemaps to the search engines. Google's page for that is here.
- Pinging Search Engines when Sitemaps Change - July 18, 2007
- Sitemap hint in robots.txt - June 13, 2007
- Google Sitemaps Accepts RSS and Atom Feeds - September 12, 2005
- Google Site Verification - September 12, 2005
- Loss of traffic due to Google sitemaps - July 24, 2005
- ColdFusion returning empty response with server-error: true
- Careful applying CF11u16, CF2016u8, CF2018u2
- Sessions don't work in Chrome but do in IE
- csrfVerifyToken does not invalidate the token
- The cf_sql_ is optional in cfqueryparam
- Cookie Expires / Max-Age 1969-12-31T23:59:59.000Z
- Burst Throttling on AWS API Gateway Explained
- How to Resolve Java HTTPS Exceptions