About the author

Vijay Kodali
E-mail me Send mail

Site Statistics

Site Meter

Recent comments

Authors

Disclaimer

The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2017

Detecting crawlers,bots,spiders in Asp.Net

Here is a way of detecting search engine (Google, live…) crawlers,spiders etc..

[code:c#]
            System.Web.HttpBrowserCapabilities clientBrowserCaps = Request.Browser;
            if (((System.Web.Configuration.HttpCapabilitiesBase)clientBrowserCaps).Crawler)
            {
                Response.Write ( "Browser is a search engine.");
            }
            else
            {
               Response.Write ("Browser is not a search engine.");
            } 

//Here is another approach..

            if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("Googlebot"))
            {
                //log Google bot visit
            }
            else if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("msnbot"))
            {
                //log MSN bot visit
            }
            else if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("Yahoo"))
            {
                //log yahoo bot visit
            }
            else
            {
                //similarly you can check for other search engines
            }

[/code]


Posted by vijay on Tuesday, July 15, 2008 11:51 AM
Permalink | Comments (0) | Post RSSRSS comment feed

Add comment


(Will show your Gravatar icon)

  Country flag

biuquote
  • Comment
  • Preview
Loading