Retrieve Data from a File System > Create a New Fetch Task

Create a New Fetch Task
To automatically retrieve files from a file system, create a new fetch task by following these steps. The connector runs each fetch task automatically, based on the schedule that is configured in the configuration file.
To create a new Fetch Task
1.
2.
3.
In the [FetchTasks] section of the configuration file, specify the number of fetch tasks using the Number parameter. If you are configuring the first fetch task, type Number=1. If one or more fetch tasks have already been configured, increase the value of the Number parameter by one (1). Below the Number parameter, specify the names of the fetch tasks, starting from zero (0). For example:
[FetchTasks]
Number=1
0=MyTask
 
NOTE The fetch tasks that you configure are run in sequence. If you want to run tasks in parallel, set the TaskThreads parameter.
4.
Below the [FetchTasks] section, create a new TaskName section. The name of the section must match the name of the new fetch task. For example:
[FetchTasks]
Number=1
0=MyTask
 
[MyTask]
5.
To specify the folders that the connector checks for files, set the DirectoryPathCSVs and DirectoryRecursive parameters.
 
[MyTask]
DirectoryPathCSVs=C:\MyFiles\,D:\MyFiles
DirectoryRecursive=True
6.
(Optional) To limit the folders that the connector checks for files, you can set the PathCrawlRegex and PathNoCrawlRegex parameters. These parameters can be useful when the location specified by DirectoryPathCSVs contains large folder structures that you do not want to index. For information about these parameters, refer to the File System Connector (CFS) Reference.
7.
(Optional) To limit the files that are retrieved, you can use the file selection parameters. For information about these parameters, refer to the File System Connector (CFS) Reference. A file is retrieved only if it matches all of the criteria that are set. For example:
[MyTask]
DirectoryPathCSVs=C:\MyFiles\,D:\MyFiles
DirectoryRecursive=True
DirectoryFileMatch=*.pdf,*.doc
DirectoryFileModifiedSince=2012-JUN-15 00:00:01
8.
 
NOTE The connector saves or updates a data file for each completed fetch task. If you make changes to the configuration and want to reset the connector so that it retrieves all of your data again, delete the data files (connector_[fetchtask_name]_datastore.db) in the connector’s installation folder.
Related Topics 
*
*