About the author

Vijay Kodali
E-mail me Send mail

Site Statistics

Site Meter

Recent comments

Authors

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2010

Detecting crawlers,bots,spiders in Asp.Net

Here is a way of detecting search engine (Google, live…) crawlers,spiders etc..

[code:c#]
            System.Web.HttpBrowserCapabilities clientBrowserCaps = Request.Browser;
            if (((System.Web.Configuration.HttpCapabilitiesBase)clientBrowserCaps).Crawler)
            {
                Response.Write ( "Browser is a search engine.");
            }
            else
            {
               Response.Write ("Browser is not a search engine.");
            } 

//Here is another approach..

            if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("Googlebot"))
            {
                //log Google bot visit
            }
            else if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("msnbot"))
            {
                //log MSN bot visit
            }
            else if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("Yahoo"))
            {
                //log yahoo bot visit
            }
            else
            {
                //similarly you can check for other search engines
            }

Currently rated 5.0 by 1 people

  • Currently 5/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Posted by vijay on Tuesday, July 15, 2008 11:51 AM
Permalink | Comments (0) | Post RSSRSS comment feed

Related posts

Add comment


(Will show your Gravatar icon)  

  Country flag

[b][/b] - [i][/i] - [u][/u]- [quote][/quote]



Live preview

Thursday, March 11, 2010 6:03 PM