User-agent: * User-agent: Openbot/3.0+(robot- response@openfind.com.tw;+http://www.openfind.com.tw/robot.html) User-agent: Szukacz/1.5 (robot; www.szukacz.pl/jakdzialarobot.html; info@szukacz.pl) User-agent:!Susie (http://www.sync2it.com/susie) User-agent:+SitiDi.net/SitiDiBot/1.0 (+Have Good Day) User-agent:192.comAgent User-agent:4anything.com LinkChecker v2.0","4anything","http://www.4anything.com/","4anything", User-agent:A1 Sitemap Generator/1.0 (+http://www.micro-sys.dk/products/sitemap-generator/) miggibot/2006.01.24 User-agent:a1.google.com User-agent:a2.google.com User-agent:a2z.lycos.com User-agent:AbachoBOT User-agent:AbachoBOT (Mozilla compatible) User-agent:ABCdatos BotLink/5.xx.xxx#BBL User-agent:abcdatos_botlink/1.0.2 User-agent:Aberja Checkoma User-agent:abot/0.1 (abot User-agent:About/0.1libwww-perl/5.47 User-agent:accoona User-agent:Accoona-AI-Agent/1.1.1 (crawler at accoona dot com) User-agent:Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com) User-agent:Acme.Spider User-agent:AcoiRobot User-agent:Acoon Robot User-agent:Acoon Robot v1.50.001 User-agent:Acorn/Nutch-0.9 (Non-Profit Search Engine User-agent:Ad Muncher User-agent:ADB_Web/1.1 User-agent:add-url.altavista.com Scooter/1.0 User-agent:AESOP_com_SpiderMan User-agent:AESOP_com_SpiderMan User-agent:agadine/1.x.x (+http://www.agada.de) User-agent:AgentName/0.1 libwww-perl/5.48 User-agent:Agent-SharewarePlazaFileCheckBot/2.0+(+http://www.SharewarePlaza.com) User-agent:ah-ha.com crawler User-agent:Ahoy! The Homepage Finder User-agent:AIBOT/2.1 By +(www.21seek.com , A Real artificial intelligence search engine , China) User-agent:aipbot/1.0 (aipbot User-agent:Aladin/3.324 User-agent:Albert Indexer User-agent:Aleksika Spider/1.0 (+http://www.aleksika.com/) User-agent:Alkaline User-agent:AlkalineBOT/1.3 User-agent:AlkalineBOT/1.4 (1.4.0326.0 RTM) User-agent:Allesklar/0.1 libwww-perl/5.46 User-agent:AltaVista Intranet V1.0 AltaVista Search Intranet PX V1.0 User-agent:AltaVista Intranet V1.0 bluewindow.ch User-agent:AltaVista Intranet V1.0 pta.at User-agent:AltaVista Intranet V1.0 Re-Index User-agent:AltaVista Intranet V2.0 pta.at User-agent:AltaVista Intranet V2.0 Sear.ch User-agent:AltaVista Intranet V2.0 www.altavista.de User-agent:AltaVista-Intranet User-agent:Amfibibot/0.06 (Amfibi Robot User-agent:ananzi User-agent:AnnoMille spider 0.1 alpha - http://www.annomille.it User-agent:AnswerBus (http://www.answerbus.com/) User-agent:AnswerChase User-agent:AnthillV1.1 User-agent:AntiBot User-agent:antibot-V1.3.3.1/debian-linux-sarge User-agent:AnzwersCrawl/2.0 User-agent:AnzwersCrawl/2.0 (anzwerscrawl@anzwers.com.au User-agent:A-Online Search User-agent:Apexoo Spider 1.0 User-agent:Aport User-agent:appie 1.1 (www.walhello.com) User-agent:appie/1.1 User-agent:Arachnoidea User-agent:Arachnoidea (arachnoidea@euroseek.com) User-agent:Arachnophilia User-agent:Arale User-agent:Araneo User-agent:Aranha User-agent:AraybOt/1.0 (+http://www.araykoo.com/araybot.html) User-agent:ArchitextSpider User-agent:archive.org_bot User-agent:Aretha User-agent:ARIADNE User-agent:Arikus_Spider User-agent:arks/1.0 User-agent:Asahina-Antenna/1.x User-agent:Asahina-Antenna/1.x (libhina.pl/x.x User-agent:ask.24x.info User-agent:asked/Nutch-0.8 (web crawler User-agent:AskJeeves User-agent:Asp seek User-agent:ASpider (Associative Spider) User-agent:ASPSeek User-agent:ASPSeek/1.2.5 User-agent:ASPseek/1.2.9d User-agent:ASPSeek/1.2.x User-agent:ASPSeek/1.2.xa User-agent:ASPseek/1.2.xx User-agent:ASPSeek/1.2.xxpre User-agent:asterias User-agent:asterias/2.0 User-agent:ATA-Translation-Service User-agent:AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html) User-agent:ATN_Worldwide User-agent:Atomz/1.0 User-agent:augurfind User-agent:augurnfind V-1.x User-agent:AURESYS/1.0 User-agent:av-dev3.av.pa-x.dec.com Scooter/2.0 G.R.A.B V1.0 User-agent:av-dev4.pa.alta-vista.net Scooter/2.0 G.R.A.B. V1.1.0 User-agent:AVSearch User-agent:AVSearch-1.0(peter.turney@nrc.ca) User-agent:axadine/ (Axadine Crawler User-agent:AxmoRobot - Crawling your site for better indexing on www.axmo.com search engine. User-agent:BaboomBot/1.x.x (+http://www.baboom.us) User-agent:BackRub User-agent:BackRub/2.1 User-agent:Baiduspider+(+http://www.baidu.com/search/spider.htm) User-agent:Baiduspider+(+http://www.baidu.com/search/spider_jp.html) User-agent:BanBots/1.2 (spider@banbots.com) User-agent:bbot/0.100 User-agent:BeamMachine/0.5 (dead link remover of www.beammachine.net) User-agent:beautybot/1.0 (+http://www.uchoose.de/crawler/beautybot/) User-agent:BebopBot/2.5.1 ( crawler http://www.apassion4jazz.net/bebopbot.html ) User-agent:BeijingCrawler User-agent:Big Brother User-agent:BigCliqueBOT/1.03-dev (bigclicbot User-agent:BIGLOTRON (Beta 2 User-agent:Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider User-agent:BilgiBetaBot/0.8-dev (bilgi.com (Beta) User-agent:BilgiBot/1.0(beta) (http://www.bilgi.com/ User-agent:Bitacle bot/1.1 User-agent:Bitacle Robot (V:1.0 User-agent:Bizbot003 User-agent:BizBot04 kirk.overleaf.com User-agent:Bjaaland User-agent:BlackWidow User-agent:Blaiz-Bee/1.0 (+http://www.blaiz.net) User-agent:Blaiz-Bee/2.00.5502 (+http://www.blaiz.net) User-agent:Blaiz-Bee/2.00.8222 (BE Internet Search Engine http://www.rawgrunt.com) User-agent:BlitzBOT@tricus.net User-agent:BlitzBOT@tricus.net (Mozilla compatible) User-agent:BlogBot/1.x User-agent:Blogdimension/Alpha2 (Blogdimension BlogBot User-agent:Bloglines Title Fetch/1.0 (http://www.bloglines.com) User-agent:BlogSearch/1.0 +http://www.icerocket.com/ User-agent:BlogSearch/1.x +http://www.icerocket.com/ User-agent:blogWatcher_Spider/0.1 (http://www.lr.pi.titech.ac.jp/blogWatcher/) User-agent:BlogzIce/1.0 (+http://icerocket.com User-agent:BlogzIce/1.0 +http://www.icerocket.com/ User-agent:Bloodhound User-agent:Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags User-agent:boitho.com-dc User-agent:boitho.com-dc/0.xx (http://www.boitho.com/dcbot.html) User-agent:boitho.com-robot/1.x User-agent:boitho.com-robot/1.x (http://www.boitho.com/bot.html) User-agent:Borg-Bot User-agent:borg-bot/0.9 User-agent:bos-catalog1.bos.lycos.com User-agent:bos-catalog2.bos.lycos.com User-agent:bos-catalog4.bos.lycos.com User-agent:bos-fe1-1.bos.lycos.com User-agent:bos-fe11-1.bos.lycos.com User-agent:bos-fe12-1.bos.lycos.com User-agent:bos-fe13-1.bos.lycos.com User-agent:bos-fe2-1.bos.lycos.com User-agent:bos-fe3-1.bos.lycos.com User-agent:bos-fe4-1.bos.lycos.com User-agent:bos-fe5-1.bos.lycos.com User-agent:bos-fe6-1.bos.lycos.com User-agent:bos-fe6-4.bos.lycos.com User-agent:bos-fe8-1.bos.lycos.com User-agent:bos-fe9-1.bos.lycos.com User-agent:bos-spider1.bos.lycos.com User-agent:bos-spider10b.bos.lycos.com User-agent:bos-spider11b.bos.lycos.com User-agent:bos-spider2.bos.lycos.com User-agent:bos-spider3.bos.lycos.com User-agent:bos-spider4.bos.lycos.com User-agent:bos-spider5.bos.lycos.com User-agent:bos-spider6.bos.lycos.com User-agent:bos-spider7.bos.lycos.com User-agent:bos-spider9b.bos.lycos.com User-agent:BoxSeaBot/0.5 (http://boxsea.com/crawler) User-agent:BravoBrian SpiderEngine MarcoPolo User-agent:bright.net caching robot User-agent:brillo.av.pa-x.dec.com Scooter/2.0 G.R.A.B. V1.1.0 User-agent:BruinBot (+http://webarchive.cs.ucla.edu/bruinbot.html) User-agent:BSpider User-agent:BStop.BravoBrian.it Agent Detector User-agent:BTbot/0.x (+http://www.btbot.com/btbot.html) User-agent:BuildCMS crawler (http://www.buildcms.com/crawler) User-agent:BullsEye User-agent:BurstFindCrawler/1.1 (crawler.burstfind.com User-agent:Buscaplus Robi/1.0 (http://www.buscaplus.com/robi/) User-agent:c1.googlebot.com User-agent:c2.googlebot.com User-agent:c3.googlebot.com User-agent:CACTVS Chemistry Spider User-agent:Calif/0.6 (kosarev@tnps.net; http://www.tnps.dp.ua) User-agent:CaliforniaBrownSpider User-agent:Camcrawler (+http://www.camdiscover.com/crawler.html) User-agent:carleson/1.0","Carleson User-agent:Carnegie_Mellon_University_WebCrawler,http://www.andrew.cmu.edu/~brgordon/webbot/index.html User-agent:Cassandra User-agent:Catall Spider User-agent:Catall-Spider/3.3.3(www.Catall.de) User-agent:CazoodleBot/0.1 (CazoodleBot Crawler User-agent:ccubee/x.0 User-agent:cfetch/1.0 User-agent:Checkbot/x.xx LWP/5.x User-agent:ChristCRAWLER 2.0 User-agent:ChristCrawler.com User-agent:churl User-agent:cIeNcIaFiCcIoN.nEt User-agent:CipinetBot (http://www.cipinet.com/bot.html) User-agent:CipinetBot/1.0 (http://www.cipinet.com/bot.html) User-agent:ClariaBot/1.0 User-agent:Claymont.com User-agent:CloakDetect/0.9 (+http://fulltext.seznam.cz/) User-agent:Clushbot/2.x (+http://www.clush.com/bot.html) User-agent:Clushbot/3.31-BinaryFury (+http://www.clush.com/bot.html) User-agent:Clushbot/3.x-BinaryFury (+http://www.clush.com/bot.html) User-agent:Clushbot/3.xx-Ajax (+http://www.clush.com/bot.html) User-agent:Clushbot/3.xx-Hector (+http://www.clush.com/bot.html) User-agent:Clushbot/3.xx-Peleus (+http://www.clush.com/bot.html) User-agent:CMC/0.01 User-agent:CoBITSProbe User-agent:Collective User-agent:Combine System User-agent:Combine/2.0 User-agent:Combine/2.0 http://combine.it.lth.se/ User-agent:cometrics-bot, http://www.cometrics.de User-agent:ComputingSite Robi/1.0 User-agent:Comrite/0.7.1 (Nutch User-agent:Conceptbot User-agent:Confuzzledbot/X.X (+http://www.confuzzled.lu/bot/) User-agent:Convera Internet Spider V6.x User-agent:ConveraCrawler/0.2 User-agent:ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl) User-agent:ConveraMultiMediaCrawler/0.1 (+http://www.authoritativeweb.com/crawl) User-agent:CoolBot User-agent:cosmos/0.8_(robot@xyleme.com) User-agent:cosmos/0.9_(robot@xyleme.com) User-agent:CougarSearch/0.1 (+http://www.cougarsearch.com/faq.shtml) User-agent:Cowbot-0.1 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) User-agent:Cowbot-0.1.x (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) User-agent:C-PBWF-ip3000.com-crawler User-agent:crawl1.googlebot.com User-agent:CrawlConvera0.1 (CrawlConvera@yahoo.com) User-agent:Crawler (cometsearch@cometsystems.com) User-agent:crawler1.bos2.fast-search.net User-agent:crawler1.googlebot.com User-agent:crawler2.bos2.fast-search.net User-agent:crawler2.googlebot.com User-agent:crawler3.bos2.fast-search.net User-agent:crawler4.bos2.fast-search.net User-agent:crawler5.bos2.fast-search.net User-agent:CrawlerBoy Pinpoint.com User-agent:Crawllybot/0.1 (Crawllybot User-agent:CrocCrawler vx.3 [en] (http://www.croccrawler.com) (X11 User-agent:csci_b659/0.13 User-agent:Cuasarbot/0.9b http://www.cuasar.com/spider_beta/ User-agent:CurryGuide SiteScan 1.1 User-agent:Cusco User-agent:Custom Spider www.bisnisseek.com /1.0 User-agent:CyberSpyder User-agent:CyberSpyder (amant@www.cyberspyder.com) User-agent:CyberSpyder Link Test User-agent:CydralSpider/1.x (Cydral Web Image Search User-agent:CydralSpider/2.2 (Cydral Image Search User-agent:CydralSpider/2.4 (Cydral Image Search User-agent:DataFountains/DMOZ Downloader User-agent:DataFountains/DMOZ Feature Vector Corpus Creator (http://ivia.ucr.edu/useragents.shtml) User-agent:DataparkSearch/4.xx (http://www.dataparksearch.org/) User-agent:DataSpear/1.0 (Spider User-agent:DataSpearSpiderBot/0.2 (DataSpear Spider Bot User-agent:DaviesBot User-agent:DaviesBot/1.7 (www.wholeweb.net) User-agent:daypopbot/0.x User-agent:dbDig(http://www.prairielandconsulting.com) User-agent:deepak-USC/ISI User-agent:DeepIndex User-agent:DeepIndex ( http://www.zetbot.com ) User-agent:DeepIndex (www.en.deepindex.com) User-agent:Deepindex V2 User-agent:DeepIndexer.ca User-agent:deepweb User-agent:Denmex websearch (http://search.denmex.com) User-agent:dev-spider2.searchpsider.com/1.3b User-agent:DeWeb(c) Katalog/Index User-agent:DiaGem/1.1 (http://www.skyrocket.gr.jp/diagem.html) User-agent:Diamond/1.0 User-agent:DiamondBot User-agent:Die Blinde Kuh User-agent:-DIE-KRAEHE- META-SEARCH-ENGINE/1.1 http://www.die-kraehe.de User-agent:DienstSpider User-agent:Digger User-agent:Digger/1.0 JDK/1.3.0rc3 User-agent:Digimarc MarcSpider User-agent:Digimarc Marcspider/CGI User-agent:Digital Integrity Robot User-agent:DigOut4U User-agent:DIIbot User-agent:Direct Hit Grabber User-agent:DirectHitGrabber User-agent:DISCoFinder User-agent:DittoSpyder User-agent:DittoSpyder User-agent:dloader(NaverRobot)/1.0 User-agent:DNAbot User-agent:DoCoMo/1.0/Nxxxi/c10 User-agent:DoCoMo/1.0/Nxxxi/c10/TB User-agent:DoCoMo/2.0 P900iV(c100 User-agent:DoCoMo/2.0/SO502i (compatible User-agent:DownLoad Express User-agent:Download-Tipp Linkcheck (http://download-tipp.de/) User-agent:DragonBot User-agent:Drecombot/1.0 (http://career.drecom.jp/bot.html) User-agent:dtSearchSpider","DtSearchSpider User-agent:Dumbot(version 0.1 beta - dumbfind.com) User-agent:Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html) User-agent:Dumbot(version 0.1 beta) User-agent:DWCP (Dridus' Web Cataloging Project) User-agent:eApolloBot/1.0 (eApollo search engine robot User-agent:EARTHCOM.info/1.x [www.earthcom.info] User-agent:EARTHCOM.info/1.xbeta [www.earthcom.info] User-agent:EasyDL/3.xx User-agent:EasyDL/3.xx http://keywen.com/Encyclopedia/Bot User-agent:EbiNess User-agent:EchO!/2.0 User-agent:Echo2 User-agent:e-collector User-agent:egothor/3.0a (+http://www.xdefine.org/robot.html) User-agent:EgotoBot/4.8 (+http://www.egoto.com/about.htm) User-agent:EI*Net/0.1 libwww/0.1 User-agent:EIT Link Verifier Robot User-agent:elfbot/1.0 (+http://www.uchoose.de/crawler/elfbot/) User-agent:ELFINBOT User-agent:Emacs-w3 Search Engine User-agent:EmailWolf User-agent:EmeraldShield.com Web Spider (http://www.emeraldshield.com/webbot.aspx) User-agent:EMPAS_ROBOT User-agent:Enfish Tracker User-agent:EnigmaBot User-agent:enoola (http://www.enoola.com) User-agent:Enterprise_Search/1.0 User-agent:Enterprise_Search/1.0.xxx User-agent:Enterprise_Search/1.00.xxx User-agent:envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.php) User-agent:envolk[ITS]spider/1.6(+http://www.envolk.com/envolkspider.html) User-agent:EroCrawler User-agent:eruvo-bot 4.8.1 (http://www.eruvo.com) User-agent:ES.NET_Crawler/2.0 (http://search.innerprise.net/) User-agent:eseek-larbin_2.6.2 (crawler@exactseek.com) User-agent:e-sense User-agent:ESISmartSpider User-agent:e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/) User-agent:Esther User-agent:eStyleSearch 4 (compatible User-agent:Eule-Robot User-agent:EuripBot/0.2 (+http://www.eurip.com) User-agent:EuripBot/0.4 (+http://www.eurip.com) User-agent:EuripBot/0.4 (+http://www.eurip.com) User-agent:EuripBot/0.5 (+http://www.eurip.com) PreCheck User-agent:Evaal/0.7.1 User-agent:Evaal/0.7.1 (Evaal User-agent:EvaalSE - bot@evaal.com", User-agent:eventax/1.3 (eventax User-agent:Everest-Vulcan Inc./0.1 (R&D project User-agent:Evliya Celebi User-agent:Exabot/2.0 User-agent:Exabot/3.0 User-agent:Exabot-Images/1.0 User-agent:Exabot-Test/1.0 User-agent:ExaBotTest/2.0 User-agent:ExaBotTest/3.0 User-agent:ExactSeek Crawler/0.1 User-agent:exactseek.com User-agent:ExactSeek_Spider User-agent:exactseek-crawler-2.63 (crawler@exactseek.com) User-agent:exactseek-pagereaper-2.63 (crawler@exactseek.com) User-agent:Exalead NG/MimeLive Client (convert/http/0.120) User-agent:Excalibur Internet Spider User-agent:Excalibur Internet Spider V6.5.4 User-agent:Execrawl/1.0 (Execrawl User-agent:EyeCatcher (Download-tipp.de)/1.0 User-agent:EZResult User-agent:factbot : http://www.factbites.com/robots User-agent:Factbot 1.09 (see http://www.factbites.com/webmasters.php) User-agent:Fast User-agent:FAST Enterprise Crawler 6 / Scirus scirus-crawler@fast.no User-agent:FAST Enterprise Crawler 6 used by BilBasen ApS (michael@bilinfo.dk) User-agent:FAST Enterprise Crawler 6 used by Comperio AS (sts@comperio.no) User-agent:FAST Enterprise Crawler 6 used by Pages Jaunes (crawladmin@gmail.com) User-agent:FAST Enterprise Crawler 6 used by Pages Jaunes (fastadmin@pagesjaunes.fr) User-agent:FAST Enterprise Crawler 6 used by Pages Jaunes (pvincent@pagesjaunes.fr) User-agent:FAST Enterprise Crawler 6 used by suckling@un.org (UN OCHA) User-agent:FAST Enterprise Crawler 6 used by Verizon Superpages Powered By FAST (crawler_admin@superpages.com) User-agent:fast enterprise crawler 6 used by verizon superpages powered by fast (kevin.watters@fastsearch.com) User-agent:fastbot crawler beta 2.0 (+http://www.fastbot.de) User-agent:fastbot crawler beta 2.0 (+http://www.fastbot.de) User-agent:FastBug http://www.ay-up.com User-agent:FastCrawler User-agent:fastlwspider/1.0 User-agent:FAST-SoccerCrawler/2.2-pre-cvs (oyvinda@fast.no User-agent:FAST-WebCrawler/2.0.10 (crawler@fast.no User-agent:FAST-WebCrawler/2.0.9 (crawler@fast.no User-agent:FAST-WebCrawler/2.1.pre.2000-04-14.1 (ashen@looksmart.net) User-agent:FAST-WebCrawler/2.1.pre.2000-04-18.1 (crawler@fast.no User-agent:FAST-WebCrawler/2.1.prealpha.2000-04-07.1 (ashen@looksmart.net) User-agent:FAST-WebCrawler/2.1-pre10 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre11 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre12 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre13 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre14 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre2 (ashen@looksmart.net) User-agent:FAST-WebCrawler/2.1-pre4 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre5 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre6 (crawler@fast.no User-agent:FAST-WebCrawler/2.1-pre7 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre1 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre2 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre3 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre4 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre5 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre8 (crawler@fast.no User-agent:FAST-WebCrawler/2.2-pre9 (crawler@fast.no User-agent:FAST-WebCrawler/3.7/FirstPage (atw-crawler at fast dot no;http://fast.no/support/crawler.asp) User-agent:FAST-WebCrawler/3.8 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp) User-agent:favo.eu crawler/0.6 (http://www.favo.eu) User-agent:Faxobot/1.0 User-agent:Feed24.com User-agent:FeedChecker/0.01 User-agent:Feedfetcher-Google User-agent:Feedster Crawler/3.0 User-agent:Felix - Mixcat Crawler (+http://mixcat.com) User-agent:Felix IDE User-agent:FetchRover User-agent:fido User-agent:fido/1.0 Harvest/1.4.pl2 User-agent:Filangy/1.01 (Filangy User-agent:Findexa Crawler (http://www.findexa.no/gulesider/article26548.ece) User-agent:findlinks/0.901 (+http://wortschatz.uni-leipzig.de/findlinks/) User-agent:findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/) User-agent:findlinks/1.1.1 (+http://wortschatz.uni-leipzig.de/findlinks/) User-agent:findlinks/1.1.1-a2 (+http://wortschatz.uni-leipzig.de/findlinks/) User-agent:findlinks/1.1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/) User-agent:FineBot User-agent:Firefly/1.0 User-agent:Firefly/1.0 (compatible User-agent:Firefox_1.0.6 (kasparek@naparek.cz) User-agent:FirstGov.gov Search - POC:firstgov.webmasters@gsa.gov User-agent:firstsbot User-agent:Fish search User-agent:Flapbot/0.7.2 (Flaptor Crawler User-agent:FlickBot 2.0 RPT-HTTPClient/0.3-3 User-agent:Fluffy the spider User-agent:Fluid Dynamics Search Engine robot User-agent:Fooky.com/ScorpionBot/ScoutOut User-agent:Fouineur User-agent:Francis/1.0 (francis@neomo.de http://www.neomo.de/) User-agent:Freecrawl User-agent:freshmeat.net URL validator/1.1 User-agent:FreshNotes crawler, report problems to crawler-at-freshnotes-dot-com User-agent:FreshNotes crawler< report problems to crawler-at-freshnotes-dot-com User-agent:FuchsBot +http://www.fuchsbot.tld User-agent:FunnelWeb User-agent:FyberSpider (+http://www.fybersearch.com/fyberspider.php) User-agent:GAIS Robot/1.0B2 User-agent:GAIS Robot/1.0B2 User-agent:Gaisbot/3.0+(indexer@gais.cs.ccu.edu.tw User-agent:Gaisbot/3.0+(robot@gais.cs.ccu.edu.tw User-agent:Gaisbot/3.0+(robot06@gais.cs.ccu.edu.tw User-agent:GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html) User-agent:gamekitbot/1.0 (+http://www.uchoose.de/crawler/gamekitbot/) User-agent:gammaSpider, FocusedCrawler User-agent:GammaSpider/1.0 User-agent:gazz User-agent:gazz/x.x (gazz@nttrd.com) User-agent:GCreep User-agent:geckobot User-agent:Generic Mobile Phone (compatible User-agent:geniebot wgao@genieknows.com User-agent:GeonaBot 1.x User-agent:GetBot User-agent:GetterroboPlus Puu User-agent:GetURL User-agent:GigaBaz User-agent:gigabaz/3.1x (baz@gigabaz.com User-agent:GigaBazVStheWeb User-agent:Gigabot/2.0 User-agent:Gigabot/2.0/gigablast.com/spider.html User-agent:Gigabot/2.0att User-agent:Giskard User-agent:GNODSPIDER (www.gnod.net) User-agent:Goblin/0.9 (http://www.goguides.org/) User-agent:Goblin/0.9.x (http://www.goguides.org/goblin-info.html) User-agent:GoForIt.com User-agent:GOFORITBOT ( http://www.goforit.com/about/ ) User-agent:Golem User-agent:gonzo1[P] +http://www.suchen.de/popups/faq.jsp User-agent:gonzo1[P] mailto:crawleradmin.t-info@telekom.de User-agent:gonzo2[P] mailto:crawleradmin.t-info@telekom.de User-agent:Google WAP Proxy/1.0 User-agent:Googlebot User-agent:Googlebot/1.0 User-agent:Googlebot/1.0 (googlebot@googlebot.com http://googlebot.com/) User-agent:Googlebot/1.0 (googlebot@googlebot.com) User-agent:Googlebot/2.0 (+http://googlebot.com/bot.html) User-agent:Googlebot/2.0 beta (googlebot@googlebot.com) User-agent:Googlebot/2.1 User-agent:Googlebot/2.1 (+http://www.google.com/bot.html) User-agent:Googlebot/2.1 (+http://www.googlebot.com/bot.html) User-agent:Googlebot/2.1d User-agent:Googlebot/2.1w User-agent:Googlebot/2.1w (+http://googlebot.com/bot.html) User-agent:Googlebot-Image/1.0 User-agent:Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html) User-agent:Googlebot-Image1.0 User-agent:Googlebot-w/2.1 User-agent:Googlebot-w/2.1 (+http://googlebot.com/bot.html) User-agent:Google-Sitemaps/1.0 User-agent:Grapnel/0.01 Experiment User-agent:great-plains-web-spider/gpws (Flatland Industries Web Spider User-agent:griffon User-agent:GrigorBot 0.8 (http://www.grigor.biz/bot.html) User-agent:Gromit User-agent:Gromit/1.0 User-agent:grub-client User-agent:gsa-crawler (Enterprise User-agent:GSiteCrawler/v1.20 rev. 273 (http://gsitecrawler.com/) User-agent:Gulliver User-agent:Gulliver/1.1 User-agent:Gulliver/1.2 User-agent:Gulliver/1.3 User-agent:Gulper Bot User-agent:Gulper Web Bot 0.2.4 (www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot) User-agent:GurujiBot/1.0 (+http://www.guruji.com/WebmasterFAQ.html) User-agent:Hämähäkki User-agent:HamBot User-agent:HappyBot User-agent:HappyBot (gserver.kw.net) User-agent:Harvest User-agent:Harvest-NG/1.0.2 User-agent:Hatena Antenna/0.4 (http://a.hatena.ne.jp/help#robot) User-agent:HatenaScreenshot/1.0 (checker) User-agent:HatenaScreenshot/1.0 (checker) User-agent:havIndex User-agent:HeinrichderMiragoRobot (http://www.miragorobot.com/scripts/deinfo.asp) User-agent:Helix/1.x (+http://www.sitesearch.ca/helix/) User-agent:HenriLeRobotMirago (http://www.miragorobot.com/scripts/frinfo.asp) User-agent:HenryTheMiragoRobot User-agent:HenryTheMiragoRobot (http://www.miragorobot.com/scripts/mrinfo.asp) User-agent:heraSpider User-agent:heraSpider User-agent:HI (HTML Index) Search User-agent:Hi! I'm CsCrawler, my homepage: http://www.kde.cs.uni-kassel.de/lehre/ss2005/googlespam/crawler.html RPT-HTTPClient/0.3-3 User-agent:Hitwise Spider v1.0 http://www.hitwise.com User-agent:HKU WWW Octopus User-agent:holmes/3.10.1 (OnetSzukaj/5.0 User-agent:holmes/3.9 (OnetSzukaj/5.0 User-agent:holmes/x.x","Holmes","http://morfeo.centrum.cz/","Morfeo / Centrum", User-agent:HomePageSearch(hpsearch.uni-trier.de) User-agent:Homerbot: www.homerweb.com User-agent:Hometown Spider Pro User-agent:Honda-Search/0.7.2 (Nutch User-agent:HooWWWer/2.1.0 (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) User-agent:HooWWWer/2.1.3 (debugging run) (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) User-agent:ht://Dig User-agent:htdig/3.1.6 (http://computerorgs.com) User-agent:htdig/3.1.6 (unconfigured@htdig.searchengine.maintainer) User-agent:htdig/3.1.x (root@localhost) User-agent:html_analyzer User-agent:HTMLgobble User-agent:http://wortschatz.uni-leipzig.de/findlinks/ User-agent:http://www.almaden.ibm.com/cs/crawler User-agent:http://www.almaden.ibm.com/cs/crawler [hc5] User-agent:http://www.monogol.de User-agent:http://www.trendtech.dk/spider.asp), User-agent:Hubater User-agent:Hyper-Decontextualizer User-agent:I, Robot User-agent:i1searchbot/2.0 (i1search web crawler User-agent:ia_archiver User-agent:ia_archiver User-agent:iajaBot User-agent:iaskspider2 (iask@staff.sina.com.cn) User-agent:IBM_Planetwide User-agent:Ibot/1.0 libwww-perl/0.40 User-agent:ICC-Crawler(Mozilla-compatible User-agent:ICCrawler - ICjobs (http://www.icjobs.de/bot.htm) User-agent:ichiro/1.0 (ichiro@nttr.co.jp) User-agent:ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html) User-agent:IconSurf/2.0 favicon finder (see http://iconsurf.com/robot.html) User-agent:IconSurf/2.0 favicon monitor (see http://iconsurf.com/robot.html) User-agent:ICRA_label_spider/x.0 User-agent:icreport.lycos.com User-agent:icsbot-0.1 User-agent:ideare - SignSite/1.x User-agent:IIITBOT/1.1 (Indian Language Web Search Engine User-agent:ilial/Nutch-0.9-dev User-agent:IlTrovatore/1.2 (IlTrovatore User-agent:IlTrovatore-Setaccio User-agent:IlTrovatore-Setaccio/1.2 (It-bot User-agent:image.kapsi.net User-agent:Imagelock User-agent:IncyWincy User-agent:IncyWincy data gatherer(webmaster@loopimprovements.com,http://www.loopimprovements.com/robot.html) User-agent:IncyWincy page crawler(webmaster@loopimprovements.com,http://www.loopimprovements.com/robot.html) User-agent:IncyWincy(http://www.look.com) User-agent:IncyWincy(http://www.loopimprovements.com/robot.html) User-agent:IncyWincy/2.1(loopimprovements.com/robot.html) User-agent:IndexTheWeb.com Crawler7 User-agent:InelaBot/0.2 (+http://inelegant.org/bot) User-agent:Inet library User-agent:InfoFly/1.0 (http://www.versions-project.org/) User-agent:infoGIST User-agent:Informant User-agent:InfoSeek Robot 1.0 User-agent:Infoseek Sidewinder User-agent:InfoSeek Sidewinder/0.9 User-agent:InfoSpiders User-agent:Ingrid User-agent:Inktomi Slurp User-agent:Insitor.com search and find world wide! User-agent:Insitornaut User-agent:Inspector Web User-agent:IntelliAgent User-agent:InterGO User-agent:Internet Cruiser Robot User-agent:Internet Ninja User-agent:Internet Ninja x.0","Internet Ninja User-agent:Internet Shinchakubin User-agent:InternetAmi IOR User-agent:InternetArchive User-agent:InternetSeer User-agent:InternetSeer.com User-agent:IP2MapBot/1.1 http://www.ip2map.com User-agent:ip3000.com-crawler User-agent:ipiumBot User-agent:IPiumBot laurion(dot)com User-agent:IpselonBot/0.xx-beta (Ipselon User-agent:IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler) User-agent:IRLbot/2.0 (compatible User-agent:IRLbot/3.0 (compatible User-agent:Iron33 User-agent:iSEEKbot/iSEEKbot-0.9-dev (http://beta.iseek.com/iseekbot.html User-agent:Israeli-search User-agent:ITI Spider User-agent:Jabot/6.x (http://odin.ingrid.org/) User-agent:Jabot/7.x.x (http://odin.ingrid.org/) User-agent:Jambot/0.1.1 (Jambot User-agent:JavaBee User-agent:Jayde Crawler. http://www.jayde.com User-agent:JBot Java Web Robot User-agent:JCrawler User-agent:jeanie/3.3.3(www.sidedc.net/ User-agent:Jetbot/1.0 User-agent:JoBo Java Web Robot User-agent:Jobot User-agent:JoeBot User-agent:jongaimpi/2.10 (jonga User-agent:JumpStation User-agent:Jyxobot/1 User-agent:k2spider User-agent:KakleBot - www.kakle.com/0.1 (KakleBot - www.kakle.com User-agent:Katipo User-agent:Katriona User-agent:KDD-Explorer User-agent:KDDI-CA33 UP.Browser/6.2.0.10.4 (GUI) MMP/2.0 (compatible User-agent:KE_1.0/2.0 libwww/5.2.8 User-agent:Keegeebot/2.1 (+http://www.keegee.com/keegee/bot.html) User-agent:Kenjin Spider User-agent:Kevin http://dznet.com/kevin/ User-agent:Kevin http://websitealert.net/kevin/ User-agent:KFSW-Bot (Version: 1.01, powered by KFSW, www.kfsw.de) User-agent:kicktooBotV1.1 kictooBot@kictoo.com User-agent:Kilroy User-agent:kinjabot (http://www.kinja.com) User-agent:kinja-imagebot (http://www.kinja.com/) User-agent:KIT_Fireball/2.0 User-agent:KIT-Fireball User-agent:KnowItAll(knowitall@cs.washington.edu) User-agent:Knowledge.com/0.x User-agent:KO_Yappo_Robot User-agent:Konqueror/1.1.1 User-agent:Krugle/Krugle,Nutch/0.8+ (Krugle web crawler User-agent:KSbot/1.0 (KnowledgeStorm crawler User-agent:kulokobot www.kuloko.com kuloko@backweave.com User-agent:kuloko-bot/0.x User-agent:kulturarw3/0.1 User-agent:KummHttp/1.1 (compatible User-agent:LabelGrabber User-agent:LapozzBot/1.4 ( http://robot.lapozz.com) User-agent:larbin User-agent:larbin_2.1.1 larbin2.1.1@somewhere.com User-agent:larbin_2.2.0 (crawl@compete.com) User-agent:larbin_2.2.1_de_Viennot (Laurent.Viennot@inria.fr) User-agent:larbin_2.2.2 (sugayama@lab7.kuis.kyoto-u.ac.jp) User-agent:larbin_2.2.2_guillaume (guillaume@liafa.jussieu.fr) User-agent:larbin_2.6.2 (hamasaki@grad.nii.ac.jp) User-agent:larbin_2.6.2 (listonATccDOTgatechDOTedu) User-agent:larbin_2.6.2 (tom@lemurconsulting.com) User-agent:larbin_2.6.3 (ltaa_web_crawler@groupes.epfl.ch) User-agent:larbin_2.6.3 (wgao@genieknows.com) User-agent:larbin_2.6.3 larbin2.6.3@unspecified.mail User-agent:larbin_2.6.3 marzia.polito@intel.com User-agent:larbin_2.6_basileocaml (basile.starynkevitch@cea.fr) User-agent:LECodeChecker/3.0 libgetdoc/1.0 User-agent:legs User-agent:LEIA User-agent:LexiBot User-agent:LexiBot/1.00 User-agent:LG-C1500 UP.Browser/6.2.3 (GUI) MMP/1.0 (compatible User-agent:Libertech-Rover User-agent:LibertyW (+http://www.lw01.com) User-agent:libWeb/clsHTTP -- hiongun@kt.co.kr User-agent:libwww/5.3.2 User-agent:libwww-perl/5.41 User-agent:libwww-perl/5.45 User-agent:libwww-perl/5.65 User-agent:Link Validator User-agent:linklooker User-agent:linknzbot User-agent:Links4US-Crawler, (+http://links4us.com/) User-agent:LinkScan User-agent:LinkScan/11.0beta2 Unix User-agent:LinkScan/9.0g Unix User-agent:LinkScan/x.x Unix User-agent:LinkWalker User-agent:LinkWalker","LinkWalker User-agent:Llaut/1.0 (http://mnm.uib.es/~gallir/llaut/bot.html) User-agent:lmspider (lmspider@scansoft.com) User-agent:LNSpiderguy User-agent:LNSpiderguy User-agent:LocalBot/1.0 ( http://www.localbot.co.uk/) User-agent:Lockon User-agent:Lockstep Spider/1.0 User-agent:logikabot User-agent:logo.gif Crawler User-agent:Look.com User-agent:LookdirBot User-agent:luchs.at URL checker User-agent:lwp-trivial/1.27 User-agent:lwp-trivial/1.34 User-agent:Lycos User-agent:Lycos_Spider_(modspider) User-agent:Lycos_Spider_(T-Rex) User-agent:Lycos_Spider_(T-Rex) User-agent:Lycos_Spider_(T-Rex)/3.0 User-agent:lycosidae.lycos.com User-agent:lycosinc.NorthRoyalton.cw.net Mozilla/4.0 User-agent:Mac WWWWorm User-agent:Mackster( http://www.ukwizz.com ) User-agent:Magpie User-agent:mammoth/1.0 (+http://www.sli-systems.com/) User-agent:MantraAgent User-agent:MantraAgent User-agent:Mariner/5.1b [de] (Win95 User-agent:Marketwave Hit List User-agent:Martini User-agent:MARTINI User-agent:Marvin v0.3 User-agent:marvin/infoseek User-agent:MaSagool/1.0 (MaSagool User-agent:MasterSeek User-agent:Mata Hari User-agent:Mata Hari/2.00 User-agent:Mattie User-agent:maxomobot/dev-20051201 (maxomo User-agent:MediaCrawler-1.0 (Experimental) User-agent:MediaFox User-agent:Mediapartners-Google/2.1 User-agent:MediaSearch/0.1 User-agent:MegaSheep v1.0 (www.searchuk.com internet sheep) User-agent:Melbot WebSpider & RSS News Crawler www.melbot.info (V.2.42 by A.I.C.E.) User-agent:Mercator User-agent:Mercator-1.1 User-agent:Mercator-1.2 User-agent:Mercator-1.x User-agent:Mercator-2.0 User-agent:Mercator-Scrub-1.1 User-agent:Merritt/1.0 User-agent:MerzScope User-agent:metabot User-agent:Metaeuro Web Crawler/0.2 (MetaEuro Web Search Clustering Engine User-agent:MetaGer_PreChecker0.1 User-agent:MetagerBot/0.8-dev (MetagerBot User-agent:MetaGer-LinkChecker User-agent:Metaspinner/0.01 (Metaspinner User-agent:metatagsdir/0.7 (+http://metatagsdir.com/directory/) User-agent:MicroBaz User-agent:Microsoft Small Business Indexer User-agent:Microsoft URL Control - 6.00.8862 User-agent:Microsoft URL Control - 6.01.9782 User-agent:MindCrawler User-agent:Missigua Locator 1.9 User-agent:Mister Jack User-agent:Mister Pix II User-agent:Misterbot-Nutch/0.7.1 (Misterbot-Nutch User-agent:Miva (AlgoFeedback@miva.com) User-agent:MJ12bot/v1.0.7 (http://majestic12.co.uk/bot.php?+), User-agent:MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+) User-agent:MJ12bot/v1.1.1 (http://majestic12.co.uk/bot.php?+) User-agent:MJ12bot/v1.1.2 (http://majestic12.co.uk/bot.php?+) User-agent:MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+) User-agent:MJ12bot/vx.x.x (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php) User-agent:mnoGoSearch search engine software User-agent:MnogoSearch/3.2.xx User-agent:Mo College 1.9 User-agent:moget User-agent:moget/2.0 User-agent:moget/x.x (moget@goo.ne.jp) User-agent:mogimogi/1.0 User-agent:MojeekBot/0.x (archi User-agent:MOMspider User-agent:Monster User-agent:Moreoverbot/5.00 (+http://www.moreover.com) User-agent:Morris - Mixcat Crawler (+http://mixcat.com) User-agent:MOSES 2.0 Spider User-agent:Motor User-agent:MOT-V975/81.33.02I MIB/2.2.1 Profile/MIDP-2.0 Configuration/CLDC-1.1 (compatible User-agent:Mouse-House/7.4 (spider_monkey spider info at www.mobrien.com/sm.shtml) User-agent:mozDex/0.04-dev (mozDex User-agent:Mozdex/0.06-dev (Mozdex User-agent:Mozilla (Mozilla@somewhere.com) User-agent:Mozilla/2.0 (compatible User-agent:Mozilla/2.0 (compatible; EZResult -- Internet Search Engine) User-agent:Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E) User-agent:Mozilla/3.0 (compatible User-agent:Mozilla/3.0 (compatible; MuscatFerret/1.4.1; olly@muscat.co.uk) User-agent:Mozilla/3.0 (compatible; MuscatFerret/1.5.2; olly@muscat.co.uk) User-agent:Mozilla/3.0 (compatible; MuscatFerret/1.5.3; olly@muscat.co.uk) User-agent:Mozilla/3.0 (INGRID/3.0 MT User-agent:Mozilla/3.0 (INGRID/3.0 MT User-agent:Mozilla/3.0 (Slurp.so/Goo User-agent:Mozilla/3.0 (Slurp/cat User-agent:Mozilla/3.0 (Slurp/si User-agent:Mozilla/3.0 (Vagabondo/1.x MT User-agent:Mozilla/3.0 (Vagabondo/2.0 MT User-agent:Mozilla/3.01 (Compatible User-agent:Mozilla/3.01 (Win95; I) User-agent:Mozilla/4.0 User-agent:Mozilla/4.0 (agadine3.0) User-agent:Mozilla/4.0 (agadine3.0) www.agada.de User-agent:Mozilla/4.0 (compatible User-agent:Mozilla/4.0 (compatible: AstraSpider V.2.1 : astrafind.com) User-agent:Mozilla/4.0 (JemmaTheTourist User-agent:Mozilla/4.0 (Sleek Spider/1.2) User-agent:Mozilla/4.0 (vBSEO User-agent:Mozilla/4.0 compatible FurlBot/Furl Search 2.0 (FurlBot User-agent:Mozilla/4.0 compatible ZyBorg/1.0 (wn-16.zyborg@looksmart.net User-agent:Mozilla/4.04 [de] (Win95; I ;Kolibri gncwebbot) User-agent:Mozilla/4.04 [de] (Win95; I ;Nav; Kolibri gncwebbot) User-agent:Mozilla/4.6 [en] (http://www.cnet.com/) User-agent:Mozilla/4.72 [en] (BACS http://www.ba.be) User-agent:Mozilla/5.0 User-agent:Mozilla/5.0 (+http://www.sli-systems.com/) Mammoth/0.1 User-agent:Mozilla/5.0 (+http://www.toile.com/) ToileBot/0.1 User-agent:Mozilla/5.0 (Clustered-Search-Bot/1.0 User-agent:Mozilla/5.0 (compatible User-agent:Mozilla/5.0 (Linux User-agent:Mozilla/5.0 (Slurp/cat User-agent:Mozilla/5.0 (Slurp/si User-agent:Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) User-agent:Mozilla/5.0 (wgao@genieknows.com) User-agent:Mozilla/5.0 (Windows User-agent:Mozilla/5.0 (X11 User-agent:Mozilla/5.0 [en] (compatible User-agent:Mozilla/5.0 URL-Spider User-agent:Mozilla/5.0 usww.com-Spider-for-w8.net User-agent:Mozilla/5.0 wgao@genieknows.com User-agent:Mozilla/5.0 whoiam [http://www.axxus.de/] User-agent:MQbot http://metaquerier.cs.uiuc.edu/crawler User-agent:MQbot metaquerier.cs.uiuc.edu/crawler User-agent:MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler User-agent:MSIECrawler User-agent:msnbot User-agent:msnbot/0.3 (+http://search.msn.com/msnbot.htm) User-agent:msnbot/0.9 (+http://search.msn.com/msnbot.htm) User-agent:msnbot/1.0 (+http://search.msn.com/msnbot.htm) User-agent:msnbot-media/1.0 (+http://search.msn.com/msnbot.htm) User-agent:msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm) User-agent:MSProxy User-agent:MSRBOT (http://research.microsoft.com/research/sv/msrbot/) User-agent:MSRBOT (http://research.microsoft.com/research/sv/msrbot) User-agent:MultiText/0.1 User-agent:Muncher User-agent:Muscat Ferret User-agent:MusicWalker2.0 (+http://www.somusical.com) User-agent:Mwd.Search User-agent:my-bytebot/1.0 (+http://spider.my-byte.de/info) User-agent:MyCNNSpider User-agent:Mylinea.com Crawler 2.0 User-agent:Naamah 1.0.1/Blogbot (http://blogbot.de/) User-agent:Naamah 1.0a/Blogbot (http://blogbot.de/) User-agent:NABOT/5.0 User-agent:nabot_1.0 User-agent:NASA Search 1.0 User-agent:NationalDirectoryAddURL/1.0 User-agent:NationalDirectory-WebSpider/1.3 User-agent:NatzanBot, http://www.natzan.com User-agent:NaverBot_dloader/1.5 User-agent:NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / nhnbot@naver.com) User-agent:NaverBot-1.0 (NHN Corp. / +82-31-784-1989 / nhnbot@naver.com) User-agent:NavissoBot User-agent:NavissoBot/1.7 (+http://navisso.com/) User-agent:NCSA Beta 1 (http://vias.ncsa.uiuc.edu/viasarchivinginformation.html) User-agent:Nebulla/V1.0 Spider (http://spider.nebulla.de) User-agent:Nebullabot/2.2 (http://bot.nebulla.info) User-agent:NEC-MeshExplorer User-agent:Nederland.zoek User-agent:Nelian Pty Ltd - Spider v2.1 ( http://pcaccessoriesparts.com ) User-agent:NetCarta WebMap Engine User-agent:NetCruiser User-agent:Netluchs/0.8-dev ( User-agent:Netluchs/0.8-dev ( User-agent:NetMechanic User-agent:Netprospector JavaCrawler User-agent:NetResearchServer User-agent:NetResearchServer(http://www.look.com) User-agent:NetResearchServer/4.0(loopimprovements.com/robot.html) User-agent:NetResearchServer/x.x(loopimprovements.com/robot.html) User-agent:NetScoop User-agent:NetSprint -- 2.0 User-agent:NetWhatCrawler/0.06-dev (NetWhatCrawler from NetWhat.com User-agent:NetZippy User-agent:newscan-online User-agent:News-gathering agent User-agent:NextGenSearchBot 1 (for information visit http://about.zoominfo.com/About/NextGenSearchBot.aspx) User-agent:NextGenSearchBot 1 (for information visit http://www.eliyon.com/NextGenSearchBot) User-agent:NextGenSearchBot 1 (for information visit http://www.zoominfo.com/NextGenSearchBot) User-agent:NextopiaBOT (+http://www.nextopia.com) distributed crawler client beta v0.x User-agent:NextopiaBOT (+http://www.nextopia.com) distributed crawler client beta v1.1 User-agent:NG/2.0","Exabot User-agent:NG/4.0.1229 User-agent:NG-Search/0.9.8 (http://www.ng-search.com) User-agent:NG-Search/0.90 (NG-SearchBot User-agent:NHSE Web Forager User-agent:NMG Spider/0.3 (szukanko.com) User-agent:Noago Spider User-agent:Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible User-agent:Nokia6820/2.0 User-agent:Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible User-agent:Nokia9500 (compatible User-agent:Nokia-WAPToolkit/1.2 googlebot(at)googlebot.com User-agent:NokodoBot/1.x (+http://nokodo.com/bot.htm) User-agent:Nomad User-agent:Norbert the Spider(Burf.com) User-agent:noxtrumbot/1.0 (crawler@noxtrum.com) User-agent:noyona_0_1 User-agent:NP/0.1 (NP User-agent:NPBot User-agent:NPBot (http://www.nameprotect.com/botinfo.html) User-agent:NPBot-1/2.0 User-agent:nttdirectory_robot/0.9 (super-robot@super.navi.ocn.ne.jp) User-agent:nuSearch Spider www.nusearch.com (compatible User-agent:Nutch","Nutch","http://lucene.apache.org/","Lucene", User-agent:NutchCVS/0.06-dev (Nutch User-agent:NutchCVS/0.0x-dev (Nutch User-agent:NutchCVS/0.7.1 (Nutch running at UW User-agent:NutchCVS/0.8-dev (Nutch running at UW User-agent:NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2. User-agent:NutchOrg/0.0x-dev (Nutch User-agent:NWSpider 0.9 User-agent:NZBot User-agent:nzexplorer User-agent:obidos-bot (just looking for books.) User-agent:ObjectsSearch/0.01-dev (ObjectsSearch User-agent:oBot ((compatible User-agent:Occam User-agent:Ocelli/1.3 (http://www.globalspec.com/Ocelli) User-agent:Ocelli/1.x (http://www.globalspec.com/Ocelli) User-agent:Octora Beta - www.octora.com User-agent:Octora Beta Bot - www.octora.com User-agent:Offline Explorer User-agent:OGspider User-agent:OmniExplorer_Bot/1.0x (+http://www.omni-explorer.com) Internet Categorizer User-agent:OmniExplorer_Bot/1.0x (+http://www.omni-explorer.com) Job Crawler","OmniExplorer User-agent:OmniExplorer_Bot/1.1x (+http://www.omni-explorer.com) Torrent Crawler User-agent:OmniExplorer_Bot/x.xx (+http://www.omni-explorer.com) WorldIndexer User-agent:Onet.pl SA, http://szukaj.onet.pl User-agent:Open Text Index Robot User-agent:Openbot/3.0+(robot-response@openfind.com.tw User-agent:Opencola User-agent:Openfind data gatherer, Openbot/3.0+(robot-response@openfind.com.tw User-agent:Openfind Robot/1.1A2 User-agent:OpenISearch/1.x (www.openisearch.com) User-agent:OpenTaggerBot (http://www.opentagger.com/opentaggerbot.htm) User-agent:OpenTextSiteCrawler/2.9.2 User-agent:OpenWebSpider/x User-agent:OpidooBOT (larbin2.6.3@unspecified.mail) User-agent:ORA_checksite User-agent:Oracle Ultra Search User-agent:OrangeSpider User-agent:Orb Search User-agent:Orbiter/T-2.0 (+http://www.dailyorbit.com/bot.htm) User-agent:Overture-WebCrawler/3.8/Fresh (atw-crawler at fast dot no User-agent:Oxxbot1 User-agent:ozelot/2.7.3 (Search engine indexer User-agent:Pack Rat User-agent:page_verifier http://www.securecomputing.com/goto/pv User-agent:PageBitesHyperBot/600 (http://www.pagebites.com/) User-agent:PageBoy User-agent:ParaSite User-agent:ParaSite/1.0b (http://www.ianett.com/parasite/) User-agent:Patric User-agent:Patwebbot (http://www.herz-power.de/technik.html) User-agent:PEERbot www.peerbot.com User-agent:pegasus User-agent:PerlCrawler 1.0 User-agent:Phantom User-agent:Phoaks User-agent:PhpDig User-agent:phpMySearch-Crawler User-agent:PICgrabber User-agent:PicoSearch/1.0 User-agent:PictureOfInternet www.malfunction.org/poi User-agent:PictureOfInternet/3.0 (http://www.malfunction.org/poi User-agent:Piffany_Web_Scraper_v0.x User-agent:Piffany_Web_Spider_v0.x User-agent:PiltdownMan User-agent:Pimptrain.com's robot User-agent:Pingdom GIGRIB v1.1 (http://www.pingdom.com) User-agent:PintaSpider User-agent:Pioneer User-agent:pipeLiner/0.xx (PipeLine Spider User-agent:Pita User-agent:Pita (Chub.Stanford.EDU) User-agent:PitSpyder Thread0 User-agent:PJspider/3.0 (pjspider@portaljuice.com User-agent:PlantyNet_WebRobot_V1.9 dhkang@plantynet.com User-agent:PluckFeedCrawler/2.0 (compatible User-agent:PlumtreeWebAccessor User-agent:PolyBot User-agent:polybot 1.0 (http://cis.poly.edu/polybot/) User-agent:Pompos User-agent:Pompos/1.1 http://dir.com/pompos.html User-agent:Pompos/1.2 http://dir.com/pompos.html User-agent:Pompos/1.3 http://dir.com/pompos.html User-agent:Pompos/1.x http://dir.com/pompos.html User-agent:Pompos/1.x pompos@iliad.fr User-agent:Popdexter/1.0 User-agent:Poppi User-agent:Popular Iconoclast User-agent:Portal Juice Spider User-agent:PortalB Spider User-agent:PortalBSpider/2.0 (spider@portalb.com) User-agent:PrivacyFinder Cache Bot v1.0", User-agent:PrivacyFinder/1.1 User-agent:PROBE! (Probert Encyclopaedia Research Robot V1.0 http://www.probertencyclopaedia.com) User-agent:processor/0.0ALPHA libwww-perl/0.20 User-agent:Project XP5 [2.03.07-111203] User-agent:PROve AnswerBot 4.0 User-agent:ProWebGuide Link Checker (http://www.prowebguide.com) User-agent:psbot User-agent:psbot/0.1 (+http://www.picsearch.com/bot.html) User-agent:psycheclone User-agent:PureSight User-agent:PycURL User-agent:Python-urllib/1.16 User-agent:QPCreep Test Rig ( We are not indexing, just testing ) User-agent:qube/2.0 (+http://qube.qelix.com/v2)","Qube User-agent:QuepasaCreep ( crawler@quepasacorp.com ) User-agent:QuepasaCreep v0.9.1x User-agent:QueryN Metasearch User-agent:Qweery_robot.txt_CheckBot/3.01 (http://qweerybot.qweery.com) User-agent:QweeryBot/3.01 ( http://qweerybot.qweery.nl) User-agent:QweeryBot/3.02 ( http://qweerybot.qweery.nl) User-agent:RaBot User-agent:ramBot xtreme x.x User-agent:RAMPyBot - www.giveRAMP.com/0.1 (RAMPyBot - www.giveRAMP.com User-agent:RAMPyBot/0.8-dev (Nutch User-agent:Rational SiteCheck (Windows NT) User-agent:Raven Search User-agent:RBSE Spider User-agent:Reaper [2.03.10-031204] (http://www.sitesearch.ca/reaper/) User-agent:Reaper/2.0x (+http://www.sitesearch.ca/reaper) User-agent:recrawler3.bos2.fastsearch.net User-agent:recrawler4.bos2.fastsearch.net User-agent:RedBot/redbot-1.0 (Rediff.com Crawler User-agent:RedCarpet/1.2 (http://www.redcarpet-inc.com/robots.html) User-agent:RedCell/0.1 (InfoSec Search Bot (Coming Soon) User-agent:RedCell/0.1 (RedCell User-agent:RedKernel WWW-Spider 2/0 (+http://www-spider.redkernel-softwares.com/) User-agent:res5.eng.pgh.lycos.com User-agent:Resume Robot User-agent:Road Runner: The ImageScape Robot User-agent:RoadHouse Crawling System User-agent:Robbie the Robot User-agent:RoboCrawl (http://www.canadiancontent.net) User-agent:RoboCrawl (www.canadiancontent.net)","CanadianContent Search User-agent:RoboCrawl Spider User-agent:RoboFox User-agent:RoboPal (http://www.findpal.com/) User-agent:Robot Francoroute User-agent:Robot/www.pj-search.com User-agent:Robot@SuperSnooper.Com User-agent:Robot2.0(PingSoft) User-agent:Robozilla User-agent:Robozilla/1.0 User-agent:Rotondo/3.1 libwww/5.3.1 User-agent:Roverbot User-agent:RufusBot (Rufus Web Miner User-agent:RuLeS User-agent:Rumours-Agent User-agent:SafetyNet Robot User-agent:SandBot User-agent:SandCrawler - Compatibility Testing User-agent:savvybot/0.2 User-agent:SBIder/0.7 (SBIder User-agent:SBIder/0.8-dev (SBIder User-agent:SBIder/SBIder-0.8.2-dev (http://www.sitesell.com/sbider.html) User-agent:ScanWeb User-agent:schibstedsokbot (compatible User-agent:ScholarUniverse/0.8 (Nutch User-agent:ScollSpider/2.0 (+http://www.webwobot.com/ScollSpider.php) User-agent:Scooter/1.0 User-agent:Scooter/1.0 scooter@pa.dec.com User-agent:Scooter/1.1 (custom) User-agent:Scooter/2.0 G.R.A.B V1.0 User-agent:Scooter/2.0 G.R.A.B. V1.1.0 User-agent:Scooter/2.0 G.R.A.B. X2.0 User-agent:Scooter/3.3 User-agent:Scooter/3.3.QA.pczukor User-agent:Scooter/3.3.vscooter User-agent:Scooter/3.3_SF User-agent:Scooter_bh0-3.0.3 User-agent:Scooter_Merc_lk-1.0 User-agent:Scooter_trk3-3.0.3 User-agent:Scooter2_Mercator_lcb-1.0 User-agent:Scooter2_Mercator_x-x.0 User-agent:Scooter-3.0.EU User-agent:Scooter-3.0.FS User-agent:Scooter-3.0.g12-23 User-agent:Scooter-3.0.HD User-agent:Scooter-3.0.VNS User-agent:Scooter-3.0QI User-agent:Scooter-3.2 User-agent:Scooter-3.2.BT User-agent:Scooter-3.2.DIL User-agent:Scooter-3.2.EX User-agent:Scooter-3.2.JT User-agent:Scooter-3.2.NIV User-agent:Scooter-3.2.SF0 User-agent:Scooter-3.2.snippet User-agent:Scooter-3.3dev User-agent:Scooter-ARS-1.1 User-agent:Scooter-ARS-1.1-ih User-agent:scooter-venus-3.0.vns User-agent:Scooter-W3.1.2 User-agent:Scooter-W3-1.0 User-agent:scoutmaster User-agent:Scrubby/2.x (http://www.scrubtheweb.com/) User-agent:sdcresearchlabs-testbot/0.8-dev (www.shopping.com/bot.html User-agent:search.at V1.2 search.at User-agent:Search.Aus-AU.COM User-agent:search.ch V1.4 User-agent:search.ch V1.4.2 (spiderman@search.ch User-agent:Search/1.0 (http://www.innerprise.net/es-spider.asp) User-agent:SearchByUsa/2 (SearchByUsa User-agent:SearchdayBot User-agent:Search-Engine-Studio User-agent:SearchExpress Spider0.99 User-agent:SearchGuild_DMOZ_Experiment (chris@searchguild.com) User-agent:Searchit-Now Robot/2.2 (+http://www.searchit-now.co.uk) User-agent:Searchmee! Spider v0.98a","Searchmee! Spider User-agent:SearchProcess User-agent:SearchSight/2.0 (http://SearchSight.com/) User-agent:SearchSpider.com/1.1 User-agent:Searchspider/1.2 (SearchSpider User-agent:Seekbot/1.0 (http://www.seekbot.net/bot.html) HTTPFetcher/0.3 User-agent:Seekbot/1.0 (http://www.seekbot.net/bot.html) HTTPFetcher/2.2 User-agent:Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.0 (XDF) User-agent:Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.2 User-agent:Seeker.lookseek.com User-agent:Semager/1.0 (http://www.semager.de) User-agent:Senrigan User-agent:Sensis.com.au Web Crawler (search_commentsatsensisdotcomdotau) User-agent:sexsearcher Sexsearcher User-agent:SeznamBot/1.0 User-agent:SeznamBot/1.0 (+http://fulltext.seznam.cz/) User-agent:SGH-Z130 SHP/VPP/R5 SMB3.1 SMM-MMS/1.1.0 profile/MIDP-2.0 configuration/CLDC-1.0 (compatible User-agent:SGH-Z130 SHP/VPP/R5 SMB3.1 SMM-MMS/1.1.0 profile/MIDP-2.0 configuration/CLDC-1.0 (compatible User-agent:SG-Scout User-agent:ShagSeeker User-agent:Shai'Hulud User-agent:sherlock/1.0 User-agent:Shim-Crawler(Mozilla-compatible User-agent:Shim-Crawler(NICT)(Mozilla-compatible User-agent:ShopWiki/1.0 ( +http://www.shopwiki.com/) User-agent:ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot) User-agent:Shoula.com Crawler 2.0 User-agent:SIE-SX1/05 UP.Browser/7.0.0.1.181 (GUI) MMP/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 (compatible User-agent:SIE-SX1/05 UP.Browser/7.0.0.1.181 (GUI) MMP/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 (compatible User-agent:SietsCrawler/1.1 (+http://www.siets.biz) User-agent:Sift User-agent:Siigle Orumcex v.001 Turkey (http://www.siigle.com) User-agent:SilentSurf User-agent:silk/1.0 User-agent:silk/1.0 (+http://www.slider.com/silk.htm)/3.7 User-agent:Simmany Robot Ver1.0 User-agent:Site Searcher User-agent:Site Valet User-agent:SiteBar/3.3.8 (Bookmark Server User-agent:SiteSpider +(http://www.SiteSpider.com/) User-agent:SiteTech-Rover User-agent:SiteTruth.com site rating system User-agent:sjc-accipiter1f.sjc.lycos.com User-agent:sjc-fe10-1.sjc.lycos.com User-agent:sjc-fe12-1.sjc.lycos.com User-agent:sjc-fe13-1.sjc.lycos.com User-agent:sjc-fe2-2.sjc.lycos.com User-agent:sjc-fe3-1.sjc.lycos.com User-agent:sjc-fe3-2.sjc.lycos.com User-agent:sjc-fe4-1.sjc.lycos.com User-agent:sjc-fe4-2.sjc.lycos.com User-agent:sjc-fe5-1.sjc.lycos.com User-agent:sjc-fe5-2.sjc.lycos.com User-agent:sjc-fe8-1.sjc.lycos.com User-agent:sjc-fe9-1.sjc.lycos.com User-agent:Skymob.com User-agent:SLCrawler User-agent:Sleek User-agent:Slider_Search_v1-de User-agent:SlowBot User-agent:Slurp User-agent:Slurp.so/1.0 User-agent:Slurp.so/1.0 (slurp@inktomi.com; http://www.inktomi.com/slurp.html) User-agent:Slurp/2.0 User-agent:Slurp/2.0 (slurp@inktomi.com; http://www.inktomi.com/slurp.html) User-agent:Slurp/2.0-KiteHourly User-agent:Slurp/2.0-OwlWeekly User-agent:Slurp/3.0-AU User-agent:Slurp/si User-agent:Slurpy Verifier/1.0 User-agent:SlySearch User-agent:Smart Spider User-agent:SmiffyDCMetaSpider/1.0 User-agent:Snapbot/1.0 User-agent:Snooper User-agent:Snoopy v1.2 User-agent:SnykeBot/0.6 (http://www.snyke.com) User-agent:SoftHypermarketFileCheckBot/1.0+(+http://www.softhypermaket.com) User-agent:Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07) User-agent:sohu-search User-agent:Solbot User-agent:SonyEricssonP910c/R2A SEMC-Browser/Symbian/3.0 Profile/MIDP-2.0 Configuration/CLDC-1.0 (compatible User-agent:SpaceBison User-agent:speedfind ramBot xtreme 8.1 User-agent:Speedy Spider User-agent:Speedy Spider (Beta/1.0 User-agent:Speedy Spider (Beta/x.x User-agent:Speedy Spider (Entireweb User-agent:Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/) User-agent:Speedy_Spider (http://www.entireweb.com) User-agent:spider.yellopet.com - www.yellopet.com User-agent:Spider/maxbot.com admin@maxbot.com User-agent:spider_monkey User-agent:spider4.srv.pgh.lycos.com User-agent:SpiderBot User-agent:Spiderline Crawler User-agent:SpiderMan User-agent:SpiderMonkey/7.0x (SpiderMonkey.ca info at http://spidermonkey.ca/sm.shtml) User-agent:Spider-Sleek/2.0 (+http://search-info.com/linktous.html) User-agent:SpiderView(tm) User-agent:SpotOn User-agent:sproose/0.1-alpha (sproose crawler User-agent:sproose/1.0beta (sproose bot User-agent:Spry Wizard Robot User-agent:spyder3.microsys.com User-agent:SQ Webscanner User-agent:Squid User-agent:Sqworm User-agent:Sqworm/2.9.81-BETA (beta_release User-agent:srd06.ma.lycos.com User-agent:srd07.ma.lycos.com User-agent:srd08.ma.lycos.com User-agent:srd09.ma.lycos.com User-agent:srd10.ma.lycos.com User-agent:StackRambler/2.0 User-agent:StackRambler/2.0 (MSIE incompatible) User-agent:StackRambler/x.x User-agent:StatFetcher/1.0 User-agent:StatFetcher/1.0 User-agent:Steeler/1.x (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) User-agent:Steeler/3.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) User-agent:Strategic Board Bot (+http://www.strategicboard.com) User-agent:suchbaer.de User-agent:suchbaer.de (CrawlerAgent v0.103) User-agent:Suchknecht(Der_Knecht) User-agent:Suchknecht.at/Spider User-agent:suchpad/1.0 (+http://www.suchpad.de) User-agent:Suke User-agent:SummyCrawler User-agent:suntek search engine User-agent:Surfbot User-agent:SurfControl User-agent:Surfnomore Spider v1.1 User-agent:SurveyBot/2.3 (Whois Source) User-agent:Sven User-agent:SwissSearch V1.2 User-agent:Swooglebot/2.0. (+http://swoogle.umbc.edu/swooglebot.html) User-agent:SycaBoT/1.0 User-agent:SycaBoT-Audio User-agent:SycaBoT-Image User-agent:SycaBoT-Programme User-agent:SycaBoT-Video User-agent:SygolBot http://www.sygol.com User-agent:SygolBot http://www.sygol.net User-agent:SynooBot/Mozilla/5.0 (compatible User-agent:Syntryx ANT Scout Chassis Pheromone User-agent:Szukacz/1.5 (robot User-agent:Szukacz/1.5 (robot User-agent:Szukacz/1.x (robot User-agent:TACH Black Widow User-agent:Tagword (http://tagword.com/dmoz_survey.php) User-agent:Talkro Web-Shot/1.0 (E-mail: webshot@daumsoft.com, Home: http://222.122.15.190/webshot) User-agent:Tarantula User-agent:tarspider User-agent:TaWWWantula User-agent:TCDBOT/Nutch-0.8 (PhD student research User-agent:Tcl W3 Robot User-agent:TeacherSoft/1.0 libwww/2.17 User-agent:TechBOT User-agent:Technoratibot/0.7 User-agent:TECOMAC-Crawler/0.x User-agent:Tecomi Bot (http://www.tecomi.com/bot.htm) User-agent:TelenetDigger/1.0 User-agent:Templeton User-agent:Teoma MP User-agent:teoma_agent1 User-agent:teomaagent crawler-admin@teoma.com User-agent:teomaagent1 [crawler-admin@teoma.com] User-agent:TeraCrawl User-agent:Teradex Mapper User-agent:TeragramCrawler User-agent:TeragramWebcrawler/1.0 User-agent:TerrawizBot/1.0 (+http://www.terrawiz.com/bot.html) User-agent:TestCrawler/Nutch-0.9 (Testing Crawler for Research User-agent:The Informant User-agent:The Jubii Indexing Robot User-agent:The NorthStar Robot User-agent:The NWI Robot User-agent:The Peregrinator User-agent:The Python Robot User-agent:The TkWWW Robot User-agent:The Web Moose User-agent:The Web Wombat User-agent:The Webfoot Robot User-agent:the World Wide Web Wanderer User-agent:The World Wide Web Worm User-agent:Theme Spider (+http://www.themespider.com/spider.html) User-agent:TheSuBot/0.1 (www.thesubot.de) User-agent:thumbshots-de-Bot (Version: 1.02, powered by www.thumbshots.de) User-agent:timboBot/0.9 http://www.breakingblogs.com/timbo_bot.html User-agent:TITAN User-agent:TitIn User-agent:tivraSpider/1.0 (crawler@tivra.com) User-agent:Tkensaku/x.x(http://www.tkensaku.com/q.html) User-agent:TLSpider User-agent:Topodia/1.2-dev (Topodia - Crawler for HTTP content indexing User-agent:Toutatis 2.5-2 User-agent:Toutatis x.x (hoppa.com) User-agent:Toutatis x.x-x User-agent:Toutatis x-xx.x (hoppa.com) User-agent:traazibot/testengine (+http://www.traazi.de) User-agent:Trampelpfad-Spider User-agent:Trampelpfad-Spider-v0.1 User-agent:Turnitin User-agent:TurnitinBot User-agent:TurnitinBot/x.x (http://www.turnitin.com/robot/crawlerinfo.html) User-agent:Turnpike Emporium LinkChecker/0.1 User-agent:TutorGig/1.5 (+http://www.tutorgig.com/crawler) User-agent:Tutorial Crawler 1.4 (http://www.tutorgig.com/crawler) User-agent:Twiceler www.cuill.com/twiceler/robot.html User-agent:Twiceler-0.9 http://www.cuill.com/twiceler/robot.html User-agent:TygoBot User-agent:TygoProwler User-agent:UCmore User-agent:UCSD Crawl User-agent:UdmSearch User-agent:UdmSearch User-agent:UKWizz/Nutch-0.8.1 (UKWizz Nutch crawler User-agent:Ultraseek User-agent:unido-bot, http://mobicom.cs.uni-dortmund.de/bot.html User-agent:unnamed User-agent:UOLCrawler (soscrawler@uol.com.br) User-agent:updated/0.1-alpha (updated crawler User-agent:updated/0.1beta (updated.com User-agent:Uptimebot User-agent:UptimeBot(www.uptimebot.com) User-agent:URL Check User-agent:URL Spider Pro User-agent:URL Spider Pro/x.xx (innerprise.net) User-agent:URL_Spider_Pro/x.x","URL_Spider_Pro User-agent:URL_Spider_Pro/x.x+(http://www.innerprise.net/usp-spider.asp) User-agent:URLBlaze User-agent:UrlScope User-agent:User-Agent: Mozilla/4.0 (SKIZZLE! Distributed Internet Spider v1.0 - www.SKIZZLE.com) User-agent:User-agent:vscooter.av.pa-x.dec.com vscooter/2.0 G.R.A.B. V1.1.0 User-agent:USyd-NLP-Spider (http://www.it.usyd.edu.au/~vinci/bot.html) User-agent:UtilMind HTTPGet User-agent:Vacobot User-agent:Vagabondo User-agent:Vagabondo/1.x MT (webagent@wise-guys.nl) User-agent:Vagabondo/2.0 MT User-agent:Vagabondo/2.0 MT (webagent at wise-guys dot nl) User-agent:Vagabondo/2.0 MT (webagent@NOSPAMwise-guys.nl) User-agent:Vagabondo/3.0 (webagent at wise-guys dot nl) User-agent:Vagabondo-WAP/2.0 (webcrawler at wise-guys dot nl User-agent:Vakes/0.01 (Vakes User-agent:Valkyrie User-agent:VCI WebViewer User-agent:versus 0.2 (+http://versus.integis.ch) User-agent:versus crawler eda.baykan@epfl.ch User-agent:Verticrawl User-agent:VeryGoodSearch.com.DaddyLongLegs User-agent:verzamelgids.nl - Networking4all Bot/x.x User-agent:Verzamelgids/2.2 (http://www.verzamelgids.nl) User-agent:Vespa Crawler User-agent:Victoria User-agent:VIPr/0.4 (VIProbot User-agent:Visbot/1.0 (+http://www.visvo.com/bot.html User-agent:VisBot/2.0 (Visvo.com Crawler User-agent:Vision Research Lab image spider at vision.ece.ucsb.edu User-agent:vision-search User-agent:VMBot/0.7.2 (VMBot User-agent:VMBot/0.9 (VMBot User-agent:Voilabot User-agent:Voilabot BETA 1.2 User-agent:Vortex/2.2 (+http://marty.anstey.ca/robots/vortex/) User-agent:Voyager User-agent:voyager/1.0 User-agent:vscooter/2.0 G.R.A.B. V1.1.0 User-agent:vspider User-agent:vspider/3.x User-agent:VWbot User-agent:w@pSpider by wap4.com User-agent:W3C_Validator/1.432.2.10 User-agent:W3C-checklink/4.2 [4.20] libwww-perl/5.803 User-agent:w3index User-agent:w3index User-agent:W3M2 User-agent:w3mir User-agent:W3SiteSearch Crawler_v1.1 http://www.w3sitesearch.de User-agent:wadaino.jp-crawler 0.2 (http://wadaino.jp/) User-agent:Walhello appie User-agent:WallPaper User-agent:Wavefire/0.8-dev (Wavefire User-agent:WAVETools User-agent:Waypath development crawler - info at waypath dot com User-agent:Waypath Scout v2.x - info at waypath dot com User-agent:WeatherBot v1.4 http://www.ezweather.net User-agent:Web Core / Roots User-agent:Web Hound User-agent:Web Magnet User-agent:Web Snooper User-agent:WebAlta Crawler/1.2.1 (http://www.webalta.ru/bot.html) User-agent:WebAlta Crawler/1.3.15 (http://www.webalta.ru/bot.html) (Windows User-agent:WebAlta Crawler/1.3.18 (http://www.webalta.net/ru/about_webmaster.html) (Windows User-agent:WebarooBot (Webaroo Bot User-agent:Webbandit User-agent:WebBandit Web Spider User-agent:webbandit/4.xx.0 User-agent:Webbase User-agent:webbot(+http://webbot.com/bot.htm)","Webbot","http://webbot.com/bot.htm","Webbot", User-agent:WebCatcher User-agent:Webclipping.com User-agent:webcollage User-agent:WebCompass User-agent:WebCopy User-agent:WebCorp/1.0 User-agent:webcrawl.net User-agent:webcrawl.net User-agent:WebCrawler/3.0 Robot libwww/5.0a WebCrawler User-agent:WebCrawler-AddURL/2.0 WebCrawler User-agent:webfetcher User-agent:WebFindBot(http://www.web-find.com) User-agent:WebGenie User-agent:Webglimpse 2.xx.x (http://webglimpse.net) User-agent:webGobbler/1.x.x User-agent:Webinator User-agent:weblayers User-agent:WebLinker User-agent:Weblog Attitude Diffusion 1.0 User-agent:Weblog Monitor User-agent:webmeasurement-bot, http://rvs.informatik.uni-leipzig.de User-agent:webmeasurement-crawler, http://mobicom.cs.uni-dortmunde.de/webmeasurement/ User-agent:WebMiner User-agent:WebMirror User-agent:Webpush User-agent:WebQuest User-agent:webrank User-agent:WebReaper User-agent:webs User-agent:WebSearch.COM.AU/3.0.1 (The Australian Search Engine User-agent:WebSearchBench WebCrawler v0.1(Experimental) User-agent:WebSearchBench WebCrawler V1.0 (Beta) User-agent:Website eXtractor User-agent:WebsiteWorth v1.0 User-agent:Websnarf User-agent:WebSpider User-agent:Webspinne/1.0 webmaster@webspinne.de User-agent:Websquash.com (Add url robot) User-agent:WebStat/1.0 (Unix User-agent:Webster v0.3 ( http://webster.healeys.net/ ) User-agent:WebStolperer User-agent:WebSymmetrix User-agent:WebVac User-agent:WebVac (webmaster@pita.stanford.edu) User-agent:Webverzeichnis.de - Telefon: 01908 / 26005 User-agent:webwalk User-agent:WebWalker User-agent:webwasher User-agent:WebWatch User-agent:WebZinger User-agent:WebZinger User-agent:WEP Search 00 User-agent:WeRelateBot/0.9 (WeRelate User-agent:Wget User-agent:whatUseek Winona User-agent:WhizBang! Lab User-agent:WhosTalking User-agent:WhoWhere Robot User-agent:Wild Ferret Web Hopper #1, #2, #3 User-agent:Willow Internet Crawler by Twotrees V2.1 User-agent:WinkBot/0.06 (Wink.com search engine web crawler User-agent:WIRE WebRefiner: User-agent:WIRE/0.11 (Linux User-agent:WIRE/0.x (Linux User-agent:Wired Digital User-agent:WISEbot/1.0 (WISEbot@koreawisenut.com User-agent:WiseWire WiseWire User-agent:WiseWire-Alpha-1.0 WiseWire User-agent:WiseWire-Alpha12-Spider971219a WiseWire User-agent:WiseWire-Alpha-Spider WiseWire User-agent:WiseWire-HotSpider-1.0 WiseWire User-agent:WiseWire-Spider WiseWire User-agent:WiseWire-Spider-1.0 WiseWire User-agent:WiseWire-Spider2 WiseWire User-agent:WiseWire-Widow-1.0 WiseWire User-agent:WiseWire-Widow-1.0-ALPHA12 User-agent:WiseWire-Widow-1.0r WiseWire User-agent:wobot/1.0 User-agent:wobot/1.0 from 206.214.202.45 User-agent:worio heritrix bot (+http://worio.com/)","Worio heritrix bot User-agent:WorldLight User-agent:WorldWideWeb-X/3.1 (+http://www.worldwideweb-x.com/) User-agent:Wotbox/alpha0.6 (bot@wotbox.com User-agent:Wotbox/alpha0.x.x (bot@wotbox.com User-agent:WSB WebCrawler V1.0 (Beta), cl@cs.uni-dortmund.de User-agent:WSB, http://websearchbench.cs.uni-dortmund.de User-agent:WSCbot User-agent:wume_crawler/1.1 (http://wume.cse.lehigh.edu/~xiq204/crawler/) User-agent:Wwlib/Linux User-agent:WWW Collector User-agent:www.arianna.it User-agent:www.petitsage.fr site detector 0.4 User-agent:WWWC Ver 0.2.5 User-agent:WWWeasel Robot v1.00 (http://wwweasel.de) User-agent:www-jp.lycos.com User-agent:wwwster/1.4 (Beta, mailto:gue@cis.uni-muenchen.de) User-agent:Xaldon WebSpider User-agent:X-Crawler User-agent:XGET User-agent:xirq/0.1-beta (xirq User-agent:xirq/0.1-beta (xirq User-agent:XupiterToolbar User-agent:XYLEME Robot User-agent:Y!J/1.0 (http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html) User-agent:Y!J-BSC/1.0 (http://help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html) User-agent:Y!J-SRD/1.0 User-agent:yacy (www.yacy.net User-agent:yacybot (i386 Linux 2.6.17-2-686 User-agent:yacybot (i386 Linux 2.6.17-gentoo-r7 User-agent:yacybot (x86 Windows XP 5.1 User-agent:Yahoo! Mindset User-agent:Yahoo-Blogs/v3.9 (compatible User-agent:YahooFeedSeeker/1.0 User-agent:YahooFeedSeeker/1.0 (compatible User-agent:YahooFeedSeeker/2.0 (compatible User-agent:Yahoo-MMCrawler/3.x User-agent:Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com) User-agent:YahooSeeker/CafeKelsa-dev (compatible User-agent:Yahoo-Test/4.0 User-agent:Yahoo-VerticalCrawler-FormerWebCrawler/3.9 User-agent:Yahoo-VerticalCrawler-FormerWebCrawler/3.9 crawler at trd dot overture dot com User-agent:YahooYSMcm/2.0.0 User-agent:Yandex User-agent:Yandex/1.01.001 (compatible User-agent:yarienavoir.net/0.2 User-agent:Yellopet-Spider User-agent:Yeti User-agent:Yeti/0.01 (nhn/1noon, yetibot@naver.com, check robots.txt daily and follows it) User-agent:yggdrasil/Nutch-0.9 (yggdrasil biorelated search engine User-agent:YodaoBot/1.0 (http://www.yodao.com/help/webmaster/spider/ User-agent:yoogliFetchAgent/0.1 User-agent:yoono/1.0 web-crawler/1.0 User-agent:YooW!/1.8.9 RC1 (+http://www.yoow.eu) User-agent:YottaCars_Bot/4.12 (+http://www.yottacars.com) Car Search Engine User-agent:YottaShopping_Bot/4.12 (+http://www.yottashopping.com) Shopping Search Engine User-agent:yxobot/x User-agent:Z-Add Link Checker (http://w3.z-add.co.uk/linkcheck/) User-agent:Zao/0.1 (http://www.kototoi.org/zao/) User-agent:Zao-Crawler User-agent:Zao-Crawler 0.2b User-agent:ZBot/1.00 (icaulfield@zeus.com) User-agent:Zearchit User-agent:ZeBot_lseek.net (bot@ze.bz) User-agent:ZeBot_www.ze.bz (ze.bz@hotmail.com) User-agent:zebra.eng.pgh.lycos.com User-agent:zedzo.digest/0.1 (http://www.zedzo.com/) User-agent:zerxbot/Version 0.6 libwww-perl/5.79 User-agent:Zeus ThemeSite Viewer Webster Pro V2.9 Win32 User-agent:Zeus xxxxx Webster Pro V2.9 Win32 User-agent:ZipppBot/0.xx (ZipppBot User-agent:ZIPPPCVS/0.xx (ZipppBot/.xx User-agent:Zippy v2.0 - Zippyfinder.com User-agent:ZoomSpider - wrensoft.com User-agent:zspider/0.9-dev http://feedback.redkolibri.com/ User-agent:ZyBorg User-agent:ZyBorg User-agent:ZyBorg/1.0 (ZyBorg@WISEnut.com Disallow:/conf/ Disallow:/stats/ Disallow:/lib/ Disallow:/js/ Disallow:/cgi-bin/ Disallow:/css/ Disallow:/smarty/ Disallow:/lib/ Disallow:/lang/ Disallow:/templates_c/ Disallow:/pop_up/ Disallow:/class/