Blocking Mozilla / Google Prefetch

web

Mozilla browsers support a feature called link prefetching, which allows a web page to tell the browser to prefetch a url if it is idle. Google has been using this technique in their search results, telling Mozilla to start loading the first result. I also noticed that MXNA 2.0 is including 3 prefetch tags.

How to you tell the browser to prefetch a url?

By using the following code:

<link rel="prefetch" href="http://url.to.prefetch/" />

How can I detect prefetching on my web site?

When mozilla does a prefetch, it sends a header X-moz: prefetch, you can then block based on that header. Google recommends sending a 404 back to block the prefetch.

How can I block prefetching?

Using mod_rewrite to send a 404:

RewriteEngine On
SetEnvIf X-moz prefetch HAS_X-moz 
RewriteCond %{ENV:HAS_X-moz} prefetch 
RewriteRule .* /prefetch-attempt [L]

This will redirect all prefetch-attempts to /prefetch-attempt as long as that file doesn't exist, the client will get a 404.

You could also block with a 503 Forbidden response:

RewriteEngine On
SetEnvIf X-moz prefetch HAS_X-moz
RewriteCond %{ENV:HAS_X-moz} prefetch 
RewriteRule .* [F,L]

In IIS using isapi rewrite:

RewriteCond X-moz: prefetch
RewriteRule .* /prefetch-attemtp [L]

Or with ColdFusion:

<cfset httpHeaders = GetHttpRequestData().headers>
<cfif StructKeyExists(httpHeaders, "X-moz") AND httpHeaders["X-moz"] IS "prefetch">
	<cfheader statuscode="404" statustext="Not Found">
	<cfabort>
</cfif>

How can I disable prefetching on mozilla browsers?

Add this line to your prefs.js file located in your Mozilla profile directory to disable prefetching in firefox and mozilla browsers.

user_pref("network.prefetch-next",false);

Some interesting things about prefetch:

  • Query Strings are ignored - if you have a page with a query string it will not be prefetched.
  • HTTP only - https pages are not prefetched by firefox. ftp and other protocols are also ignored
  • Cross domain cookie problems - With prefetching enabled, you may end up with cookies and web pages in your web browser's cache from web sites that you did not click on since prefetching happens automatically when you view pages.
  • A good way to preload images - prefetching is a good way to preload images, instead of using javascript to do this because your browser can do it while it is idle.


Related Entries

7 people found this page useful, what do you think?

Trackbacks

Trackback Address: 312/343C304E904B69DDF7818493EC2D2A6E

Comments

On 04/21/2005 at 5:42:07 PM EDT Joseph Yaroch wrote:
1
Using a mozilla blowswer, an easier way to disable prefetching is to type "about:config" in the address bar. This brings up a page of numerous preferences. Type "prefetch" in the filter, so that only settings that contain the string, prefetch, are shown. Then doulbe click on the prefence, and it will change from true to false.

On 04/21/2005 at 5:46:12 PM EDT Pete Freitag wrote:
2
Thanks Joseph,

That is a lot easier!

On 05/06/2005 at 2:23:09 PM EDT James Moberg wrote:
3
Your ColdFusion example only works with CFMX6 or 7... do you have a method that works with CF5?

On 05/06/2005 at 2:40:38 PM EDT Pete Freitag wrote:
4
James,

GetHttpRequestData() should work in CF5, what isn't working?

On 05/06/2005 at 3:05:53 PM EDT James Moberg wrote:
5
It's the syntax. I used CFDump to figure it out and it should be written like this (without GetHttpRequestData().headers):

<cfset httpHeaders = GetHttpRequestData()> <cfif StructKeyExists(httpHeaders.headers, "X-moz") AND httpHeaders.headers["X-moz"] IS "prefetch"> <cfheader statuscode="404" statustext="Not Found"><cfabort> </cfif>

Thanks! Your article is the only one I found concerning how to block prefetching. This is going to become more of a problem now that Google's Web Accelerator has this enabled. For more info, go to: http://37signals.com/svn/archives2/google_web_accelerator_hey_not_so_fast_an_alert_for_web_app_designers.php

On 05/09/2005 at 7:36:52 AM EDT John Seymore wrote:
6
Here's a related site

http://www.yafla.com/papers/Block_Google_Prefetching/block_prefetching.html

On 05/09/2005 at 5:55:18 PM EDT sophek wrote:
7
Where do you put to code? In the Application.cfm file?

<cfset httpHeaders = GetHttpRequestData().headers> <cfif StructKeyExists(httpHeaders, "X-moz") AND httpHeaders["X-moz"] IS "prefetch"> <cfheader statuscode="404" statustext="Not Found"> <cfabort> </cfif>

thanks Sophek

On 05/09/2005 at 6:01:43 PM EDT Sophek wrote:
8
You want this code to run before your page logic does, so yeah Application.cfm would probably be a good place to put it.

On 11/22/2005 at 2:22:39 AM EST JA wrote:
9
ITYM 403 Forbidden, not 503 Forbidden.

503 is Service (temporarily) Unavailable.

On 06/19/2006 at 6:23:59 PM EDT ikki wrote:
10
I tried the mod rewrite and it does not work. I also do not see the X-moz or anything regarding prefetching in the apache logs.

RewriteEngine On SetEnvIf X-moz prefetch HAS_X-moz RewriteCond %{ENV:HAS_X-moz} prefetch RewriteRule .* /prefetch-attempt [L]

On 04/27/2007 at 11:50:04 AM EDT Mike S wrote:
11
The above apache changes did not work for me either. The correct apache change is:

SetEnvIfNoCase X-moz prefetch has_x-moz=true RewriteCond %{ENV:has_x-moz} true RewriteRule . . [F,L]

On 04/27/2007 at 11:51:19 AM EDT sleman wrote:
12
lines got joined together.

SetEnvIfNoCase X-moz prefetch has_x-moz=true

RewriteCond %{ENV:has_x-moz} true

RewriteRule . . [F,L]

On 07/31/2008 at 5:13:15 PM EDT uoaabb wrote:
13
I also found that the original apache version didn't work for me. Mike S's version (with correct line breaks) would probably work, but I designed a simpler version that avoids the intermediate step of setting an environment variable, namely:

RewriteEngine on

RewriteCond %{HTTP:X-moz} prefetch

RewriteRule . . [F,L]

While I verified that this works, I didn't use it in the end because I found that the small number of prefetch requests we're getting with the X-moz header are well-behaved and have negligible impact on our server.

Meanwhile, I found another class of prefetch requests with a serious negative impact. Some companies now have proxy servers that use a prefetch engine which is poorly behaved (e.g, they prefetch URLs that include query parameters), and they prefetch URLs at such a high rate that it overloaded our server. These prefetch requests don't include the X-moz header, but can be identified by the User Agent string "Mozilla/4.0 (compatible;)". To block these requests, I'm using the apache code:

RewriteEngine on

RewriteCond %{HTTP_USER_AGENT} compatible;\)$

RewriteRule . . [F,L]

On 12/05/2008 at 2:33:40 PM EST davux wrote:
14
Actually, the "Cross domain cookie problems" part is not due to the prefetching, but to the allowing of third-party cookies. In the "Privacy" tab under Edit/Preferences, you can disallow third-party cookies.

On 09/24/2011 at 6:49:29 AM EDT Neespsypeceta wrote:
15
Howdy! Would you mind if I share your blog with my facebook group? There's a lot of folks that I think would really enjoy your content. Please let me know. Many thanks http://clarkpeterso513.edublogs.org/2011/09/19/devis-site-internet/

On 02/07/2012 at 11:33:30 PM EST suzanelyk wrote:
16
In my opinion it is obvious. I recommend to look for the answer to your question in google.com

Post a Comment




  



Spell Checker by Foundeo

Recent Entries



foundeo


did you hack my cf?