StoreSiteStructure
 
Type

Boolean

Default

false

Allowed range

Minimum:  

Maximum:  

Recommended range

Minimum:  

Maximum:  

Required

no

Configuration section

[Default] and [SpiderJob]

Description

The connector stores site structures. This makes the spidering process faster when the same site is spidered again. Disable this parameter if you do not want the connector to store site structures.

If StoreSiteStructure is enabled and the parameter DeleteProcess is set to DREREMOVE, pages that are not found by the connector are removed from the site structure file and from IDOL server.

If StoreSiteStructure is enabled, the parameter DeleteProcess is set to DREREMOVE, and the parameter SiteStructureCompactCopy is disabled, pages that are not found by the connector are removed from IDOL server, but the link to the page is kept in the site structure file and the connector attempts to follow the link on subsequent spider cycles.

Use the parameter SiteStructureFlags to determine how the site structure is stored and checked.

Example

StoreSiteStructure=true

See also

DeleteProcess

SiteStructureCompactCopy

SiteStructureFlags