FAST ESP 4.3.x : Delete Indexed Documents

QUESTION:

I have several collections that I would like to re-crawl from scratch, but I don't want to have to reconfigure all the settings for each. In FDS 3.x, is there a way to delete all crawled data without losing the collection configurations?


ANSWER:

Here are the steps required for deleting all crawled data and the index from a 3.2 installation without removing the crawler configuration:

IMPORTANT - This will cause complete loss of all indexed documents,
therefore, search will be unavailable for some time until the crawler has begun re-populating the collections. We strongly recommend initiating this procedure during a system maintenance window.

1. Stop FDS from the Admin GUI or using the command 'net stop FASTDSService'

2. Ensure all FAST processes have had time to stop completely and manually kill any remaining processes with the Task Manager

3. Delete all files and directories within the %FASTSEARCH%\data\directory, EXCEPT %FASTSEARCH%\data\crawler\run\domainspec (this file contains the crawler collection configurations)

4. Start FDS with the command 'net start FASTDSService'

5. Once all FDS processes are active in the System Management page, open up the collection configuration for each collection, verify that the settings are still correct and then click 'submit' on each to refresh the collection information.


NOTES:

-You may see temporary OSErrors for the PostProcessor trying to locate the collections directory (which will be in the process of being rebuilt).

- You may also see temporary errors from the QRServer, such as 'All partitions down', because the index is still being rebuilt.

- Some collections may start immediately crawling, while others may be idle for a short time before they start crawling.

FAST ESP : 'ConfigServerExceptions.CollectionError'

Question:
==========
I have a collection I am trying to delete through the Admin gui. When clicking on the trashcan it says the collection was fully deleted and gives me a success message. But when I go in and try to create a new collection with the same name I get the following:

FaultCode: 1.

Reason 'ConfigServerExceptions.CollectionError: The Collection
aehcatalog1 already exists (in d:\e\win2ksp3-i686\datasearch-3.1.0.10-
filter-flexlm-000
\common\datasearch\src\configserver\ConfigServerConfig.py:CreateCollec
tion line 794)'

What am I doing wrong?

Solution:
===========
The collection isn't actually deleted when initially performing the action of deleting. When you delete the collection is "scheduled for deletion", you see all the documents that are associated with the collection are blacklisted in the search index and will be removed as the deletes are pushed though the system (this happens automatically)

However if you try to add a collection back with the same name, you will not be able to because it wasn't fully deleted. In reality you will be able to add it back again, however it might take a few hours before the system is ready to accept a collection with the same name again.

A suggestion is to create a collection with a different name.If you want to add the collection back, you'll have to wait for the system to digest your request to delete it. That will allow you at least work with the collection and pipeline, until you have it set exactly the way you want.Then you can add the collection back as the
original name.

Introduction - FAST Taxonomy

The FAST Taxonomy Explorer contains Categorization, based on advanced Linguistic technologies that let you control the flow of information into your organization and order, access, and retrieve that data; as well as information created within your organization.

The Categorizer classifies documents and organizes information into a hierarchical or a flat set of categories.Categorization is the process of concisely defining the information within a particular doc-ument; in other words, the major topic or subject of the document. In the context of text-
based document searches, categorization is an automated process that classifies numerous text documents, placing them into a taxonomy. A taxonomy is an organized classification structure that facilitates information retrieval.The categorization process inserts category tags into the documents prior to indexing.

When the documents in an index have been categorized, end users can restrict a query to a specific category in that index. Categorizing documents increases the likelihood that your end users will obtain the meaningful results they seek for two reasons:

1. Metadata: Documents are organized and stored by category, according to
their metadata tags.

2. Filter: Queries can be filtered using the categories that you created as
part of your taxonomy.

You can also choose to categorize the end-user documents from among several languages.FAST have Taxonomy Explorer to create & test a catagory.

FAST - Search Engine

FAST is the leading global provider of enterprise search technologies and solutions that are behind the scenes at the world's best known companies. FAST's flexible and scalable enterprise search platform (FAST ESP) elevates the search capabilities of enterprise customers and connects people to the relevant information they seek regardless of medium. This drives revenues and reduces total cost of ownership by effectively leveraging IT infrastructure. FAST's solutions are used by more than 2,600 global customers and partners, including America Online (AOL), Cardinal Health, CareerBuilder.com, CIGNA, CNET, Dell, Factiva, Fidelity Investments, Findexa, IBM, Knight Ridder, LexisNexis, Overture, Rakuten, Reed Elsevier, Reuters, Sensis, Stellent, Tenet Healthcare, Thomas Industrial Networks, Thomson Scientific, T-Online, US Army, Virgilio (Telecom Italia), Vodafone, and Wanadoo.

FAST is headquartered in Norway and is publicly traded under the ticker symbol 'FAST' on the Oslo Stock Exchange. The FAST Group operates globally with presence in Europe, the United States, Asia Pacific, Australia, South America, and the Middle East.

In January, Microsoft made an accepted offer to acquire FAST for $1.2 billion.

FAST SEARCH - Home