|How do I implement a mini-portal on my site?
Using the FusionBot mini-portal feature, you can provide the spider additional entrypoints into your site, and/or instruct the FusionBot spider to crawl, and make searchable, multiple URLs / domains for your account.
To implement a Mini-Search Portal, login to your account, under the Spider tab, click on the Mini-Portal link. Fill out the Site List form, click the 'Save' icon, and request a spider. At a minimum, each entry / line within your Site List form should include a single distinct URL constructed as follows:
Include on a separate line, up to your package's limit, each site you would like to be contained within your Mini-Search Portal.
Following are the variables that you my set for each entry within your mini-portal for more finite control over how the FusionBot crawler will index a particular site. Included in this table is a list of all valid variables you may include for each entry and a description of their function:
||Where URL is the Domain, Directory, and Start Page Name of the site to include in your mini-portal.
This is the only variable the MUST be present for each entry in your Site List form.
||Used to define how the FusionBot crawler should follow links within the Domain defined in the SITE="URL" variable:
BASE="0" - (Default) Only index links at or below the level of the directory included as part of the SITE="URL" variable.
BASE="1" - (Server Level) Index ALL pages within a single server specified in the SITE="URL" variable, regardless of the directory within which they are contained.
BASE="2" - (Domain Level) Index ALL pages for ANY server within the domain specified in the SITE="URL" variable. For example, this will cause BOTH http://marketing.mydomain.com AND
http://accounting.mydomain.com to be indexed for your entry.
||Where X equals the maximum number of pages to crawl for a particular entry.
If you do not specify a page count, FusionBot will attempt to index every page of each site included in your list, up to the maximum total page count available for your package.
When the only value within a URL that differentiates a site is that of a single variable within a page's querystring, use this function for specifying this value to the FusionBot crawler.
This scenario is common with many shopping-cart related sites that host multiple vendors under a single domain. In this case, the only differentiating factor may be a Customer Id value that is included
for every page within a site, such as CID=123456.
Setting the value of the Unique_String equal to CID=123456 will enable FusionBot to successfully crawl this type of site.
||By default, the FusionBot crawler does not accept cookies from a site when crawling its pages.
However, to successfully index sites that require the establishment of a unique session or the maintenance of session state during crawling, the FusionBot crawler must accept cookies.
In this scenario, setting the value of the COOKIES variable to A for "Accept" will ensure a successful crawl.
||For sites requiring login via HTTP Basic Authentication set the value of USERNAME equal
to a valid login name for the specified server.
||Used in conjunction with USER="" for accessing password protected sites via HTTP Basic Authentication. Set the value of PASSWORD equal
to a valid password for logging in to the specified server.
||Used to specify the METHOD TYPE the FusionBot crawler should use when requesting the page and its variables included in the URL defined in the SITE="URL" variable:
METHOD="GET" - (Default) Will cause the FusionBot spider to request the page specified in the SITE="URL" variable using the GET Method.
METHOD="POST" - Will cause the FusionBot spider to request the page specified in the SITE="URL" variable using the POST Method.
||Enables you to assign categories to your mini-portal sites (pages) in order to facilitate the building of a custom search box using FusionBot's search partitions feature.
Each category name MUST be delimited by a pipe (|), even when only a single category is assigned, REGION="|CATEGORY1|", category names must NOT contain spaces, and category names CANNOT be numeric only, i.e. they must be alphanumeric.
||Instructs FusionBot to append to the front of EVERY page crawled, as defined via the SITE attribute above, a tracking / redirect URL so that you can receive Affiliate / Commission Program credit for traffic sent via your search results.
For example, if you have specified the site http://www.widgets.com, to be crawled via the SITE attribute, and as part of this site FusionBot has indexed the page http://www.widgets.com/products.html, if you have specified a REDIRECT url of http://www.tracking.com/cgi-bin/aff.pl?, the clickable
link on your search results page to http://www.widgets.com/products.html would be constructed as http://www.tracking.com/cgi-bin/aff.pl?http://www.widgets.com/products.html, as would ALL other pages within this same domain.
Please Note: When including multiple variable entries for a single site be sure to include a space between each entry, otherwise your mini-portal will not be successfully created:
The FusionBot spider will always index your site first. If after we complete the indexing of your site, you have additional pages remaining, we will then check for your Site List and begin building your Mini-Search Portal around its content.
The same page limits (the maximum number of pages FusionBot will build its index around) apply whether or not you have implemented a Site List. A Mini-Search Portal simply offers another mechanism to fully utilize the number of pages available to you based on your package.
After you have built and saved your Site List, you must request a spider. This will initiate the building of your Mini-Search Portal. Please keep in mind that in consideration of the owners of the sites included in your list, the index speed will automatically be set to Medium.
If there are certain pages or directories that you wish to exclude from your index, for your mini-portal entries, add them to your robots.txt file or to your Online Exclusions Form, by entering the directory or page name you wish to exclude as an absolute URL.
You can also use your mini-portal to specify multiple entry points for a single site. For more information, please view the following FAQ.
If you did not find an answer to your question, please return to the Support Page and fill out the form provided.