How to block backlink checking sites (Majestic, Ahrefs, SEMRush), from spying your website?

Now go to Google and type in your focus keyword — your money keyword, the most important keyword in your SEO strategy. See where you rank and see where your competitors rank. And if you ranking first you don;t want to share the income with competitors, so you need to hide all your SEO job.

 

Most important bots to block if you like to do:

Majestic SEO -> User-agent: MJ12bot
MOZ OpenSiteExplorer -> User-agent: rogerbot
Ahrefs -> User-agent: Ahrefs

Robots.txt:

  1. Useragent: Rogerbot
  2. Useragent: Exabot
  3. Useragent: MJ12bot
  4. Useragent: Dotbot
  5. Useragent: Gigabot
  6. Useragent: AhrefsBot
  7. Useragent: BlackWidow
  8. Useragent: ChinaClaw
  9. Useragent: Custo
  10. Useragent: DISCo
  11. Useragent: Download\ Demon
  12. Useragent: eCatch
  13. Useragent: EirGrabber
  14. Useragent: EmailSiphon
  15. Useragent: EmailWolf
  16. Useragent: Express\ WebPictures
  17. Useragent: ExtractorPro
  18. Useragent: EyeNetIE
  19. Useragent: FlashGet
  20. Useragent: GetRight
  21. Useragent: GetWeb!
  22. Useragent: Go!Zilla
  23. Useragent: GoAheadGotIt
  24. Useragent: GrabNet
  25. Useragent: Grafula
  26. Useragent: HMView
  27. Useragent: HTTrack
  28. Useragent: Image\ Stripper
  29. Useragent: Image\ Sucker
  30. Useragent: Indy\ Library
  31. Useragent: InterGET
  32. Useragent: Internet\ Ninja
  33. Useragent: JetCar
  34. Useragent: JOC\ Web\ Spider
  35. Useragent: larbin
  36. Useragent: LeechFTP
  37. Useragent: Mass\ Downloader
  38. Useragent: MIDown\ tool
  39. Useragent: Mister\ PiX
  40. Useragent: Navroad
  41. Useragent: NearSite
  42. Useragent: NetAnts
  43. Useragent: NetSpider
  44. Useragent: Net\ Vampire
  45. Useragent: NetZIP
  46. Useragent: Octopus
  47. Useragent: Offline\ Explorer
  48. Useragent: Offline\ Navigator
  49. Useragent: PageGrabber
  50. Useragent: Papa\ Foto
  51. Useragent: pavuk
  52. Useragent: pcBrowser
  53. Useragent: RealDownload
  54. Useragent: ReGet
  55. Useragent: SiteSnagger
  56. Useragent: SmartDownload
  57. Useragent: SuperBot
  58. Useragent: SuperHTTP
  59. Useragent: Surfbot
  60. Useragent: tAkeOut
  61. Useragent: Teleport\ Pro
  62. Useragent: VoidEYE
  63. Useragent: Web\ Image\ Collector
  64. Useragent: Web\ Sucker
  65. Useragent: WebAuto
  66. Useragent: WebCopier
  67. Useragent: WebFetch
  68. Useragent: WebGo\ IS
  69. Useragent: WebLeacher
  70. Useragent: WebReaper
  71. Useragent: WebSauger
  72. Useragent: Website\ eXtractor
  73. Useragent: Website\ Quester
  74. Useragent: WebStripper
  75. Useragent: WebWhacker
  76. Useragent: WebZIP
  77. Useragent: Wget
  78. Useragent: Widow
  79. Useragent: WWWOFFLE
  80. Useragent: Xaldon\ WebSpider
  81. Useragent: Zeus
  82. Disallow: /

.htaccess:

  1. SetEnvIfNoCase UserAgent .*rogerbot.* bad_bot
  2. SetEnvIfNoCase UserAgent .*exabot.* bad_bot
  3. SetEnvIfNoCase UserAgent .*mj12bot.* bad_bot
  4. SetEnvIfNoCase UserAgent .*dotbot.* bad_bot
  5. SetEnvIfNoCase UserAgent .*gigabot.* bad_bot
  6. SetEnvIfNoCase UserAgent .*ahrefsbot.* bad_bot
  7. SetEnvIfNoCase UserAgent .*sitebot.* bad_bot
  8. <Limit GET POST HEAD>
  9. Order Allow,Deny
  10. Allow from all
  11. Deny from env=bad_bot
  12. </Limit>

 

 

Leave a Comment