Site Search
sitemap • search tips • faqs 
LOG-IN
SUBSCRIBE
PACKAGES
FEATURES
CUSTOMERS
ABOUT US
SUPPORT
Frequently Asked Question #34
Can I prevent the indexing of the META tags, ALT tags, and / or content within FORM or OPTION tags within EVERY page of my index?
By default, FusionBot includes the contents of your META Keywords, META Description, and ALT tags, as well as text between open and close OPTION tags within your searchable index. In addition, FusionBot will index text and attempt to parse anything that appears to be a link within FORM tags on your site, such as values within hidden fields, etc. There may be times, however, that the contents of these tags are pertinent for a general search engine, but not for your site search results, and as a result you would like FusionBot to omit them from the searchable index it creates for your site.

The following instructions will cause the FusionBot crawler to completely ignore all META, ALT, and OPTION tag content, as well as ignore content and apparent links / URLs within FORM tags. As an alternative, you can instruct FusionBot to ignore just specific tags on specific pages by referencing the following FAQ.

To omit your META and ALT, OPTION, and/or FORM tags from your index either add/modify your ROBOTS.TXT file to include the syntax below, OR login to your account, click on the 'Spider' tab, and select the 'Click to Setup' link under the 'Exclude Pages & Directories' option and include the syntax below in your Exclusion Form:

User-Agent: fusionbot
Disallow: /*.metakeys
Disallow: /*.metadesc
Disallow: /*.alt
Disallow: /*.option
Disallow: /*.form

The 'Disallow: /*.metakeys' line will prevent FusionBot from indexing then contents within your
<META NAME="keywords"> tag.

The 'Disallow: /*.metadesc' line will prevent FusionBot from indexing then contents within your
<META NAME="description"> tag.

The 'Disallow: /*.alt' line will prevent FusionBot from indexing then contents within your
<IMG SRC="/img.gif" ALT="Description"> tag.

The 'Disallow: /*.option' line will prevent FusionBot from indexing the contents within your
<OPTION VALUE="0">TEXT</OPTION> tag.

The 'Disallow: /*.form' line will prevent FusionBot from indexing the content between your
<FORM></FORM> tag, while in addition ignore / not follow any parseable links (URLs).

After making the changes, be sure to click the 'Request Spider' link, available via the 'Spider' tab. This will cause our spider to immediately re-crawl your site, with your new settings in place for excluding your specified content. You will receive a confirmation email when the spider has completed.


If you did not find an answer to your question, please return to the Support Page and fill out the form provided.


 Featured Customers:
  • Coca-Cola Enterprises
  • Rezidor Hotel Group
  • Covenant Health Products
  • American Electric Power
  • Everywherechair
  • Fleet Safety Equipment, Inc.
  • HandsOnTools
  • Chemglass Life Sciences
  • MerckSerono
  • R-JetTek Inkjet Technology Solutions
  • Madison Gas and Electric
  • TwitterGadget
  • True Beer
  • Galaxy.com
  • Capital Tractor
  • adamanta
  • BBQ Island Outdoor Products
  • Battery Clerk
  • DCU - Digital Federal Credit Union
  • BJJ Sports
  • National Center for Missing and Exploited Children

[Subscribe to FusionBot - Site Search & Sitemap Solutions]
Copyright 1998 - 2016 LOGIKA Corporation. All rights reserved.
[about site search] [advertising] [build vertical search engine] [reseller program] [popular searches] [top clicks] [terms of use] [uptime]