[Default] section

The [Default] section contains default parameters that apply to every spider job defined in the [Spider] section of the configuration file. The parameters configured in a spider job section override the parameters configured in the [Default] section.

For example:

[Default]
DREHost=127.10.9.2
ACIPort=20010
Database=News
LogFile=spider.log
nSockets=16
SpiderRepeatSecs=86400
SpiderCycles=-1
Depth=99
SpiderStartTime=03:00
SiteDuration=43200
MaxPages=2000
MaxPagesCheck=1
ImportStripLinks=true
FollowRedirect=true
StayOnSite=true
MinPageSize=4090
MaxPageSize=163840
MaxLinksPerPage=100
PageTimeout=100
CantHaveCheck=129
CantHaveCSVS=*archive*
AfterDate=-365
BeforeDate=7
SpiderAs=HTTP Connector Spider, contact spider@MyCompany.com
DateFormats=DDMMYYYY,YYMMDD,YYMMD,DDMONTHYYYY,MMDDYY,MMDD
BatchProcess=IMPORT
BatchSize=200
ImportMetaToFields=true
FollowRobotProtocol=true
PageDelay=30