Site Search
sitemap • search tips • faqs 
Frequently Asked Question #42
Can I prevent FusionBot from indexing my alternate file types?
If you have a paid FusionBot Account and you do not wish to have your Adobe Acrobat PDF, RTF, or your Microsoft Office documents indexed, you can create a robots.txt file and place it in the root / default directory of your website, or login to your FusionBot account, click on the 'Spider' tab, and select the 'Exclude Pages & Directories' link to create an internal robots deny list via the Robots Exclusion Form feature.

The robots deny list contains special instructions for the FusionBot spider to use when visiting your site and creating an index of its searchable content. To instruct the FusionBot spider to exclude an alternate file type, construct your robots.txt file or enter into your Exclusion Form the following:

User-Agent: fusionbot
Disallow: /*.pdf
Disallow: /*.doc
Disallow: /*.xls
Disallow: /*.ppt
Disallow: /*.rtf

This special syntax will prevent the indexing of ALL of the specificed alternate file types within your site. As this is not a standard implementation for the robots.txt specification, this will only work with your FusionBot account.

This syntax works for denying ALL of the specified file types within your site and/or mini-portal.

To prevent the contents of certain directories from being indexed, regardless of file type, or to prevent a specfic page from being indexed, please reference the following FAQ.

After making your changes, you must click the 'Request Spider' link, available via the 'Spider tab, to rebuild your index.

If you did not find an answer to your question, please return to the Support Page and fill out the form provided.

 Featured Customers:
  • Coca-Cola Enterprises
  • Rezidor Hotel Group
  • Covenant Health Products
  • American Electric Power
  • Everywherechair
  • Fleet Safety Equipment, Inc.
  • HandsOnTools
  • Chemglass Life Sciences
  • MerckSerono
  • R-JetTek Inkjet Technology Solutions
  • Madison Gas and Electric
  • TwitterGadget
  • True Beer
  • Capital Tractor
  • adamanta
  • BBQ Island Outdoor Products
  • Battery Clerk
  • DCU - Digital Federal Credit Union
  • BJJ Sports
  • National Center for Missing and Exploited Children

[Subscribe to FusionBot - Site Search & Sitemap Solutions]
Copyright 1998 - 2016 LOGIKA Corporation. All rights reserved.
[about site search] [advertising] [build vertical search engine] [reseller program] [popular searches] [top clicks] [terms of use] [uptime]