IDOL Server Operations > AgentBoolean Agents and Categories > Optimize AgentBoolean Matching > Customize Agent Content

Customize Agent Content
In large systems that experience heavy load, it is possible to optimize the performance of AgentBoolean queries by creating optimized agents or categories. This optimization is most useful when you use IDOL server to:
*
*
Return categories that match a non-indexed document. For example, to categorize a document and add the category information to a field before indexing.
When IDOL server matches query text against agents or categories, it uses the following matching order:
1.
It matches the query text against the Index fields in the agents or categories (for example TRAINING, or DRECONTENT fields).
2.
3.
For agents that match in step 2, it matches the query text against any FieldText restrictions in the agents or categories.
IDOL server checks the Boolean restriction only if the agent or category content matches the document.
To optimize performance, choose your category or agent content carefully to ensure that it matches only query text that is also likely to match the Boolean expression.
Use a Minimal List of Rare Terms
For many Boolean restrictions, you can select one or two terms that queries must contain to match the Boolean expression. If you select the rarest of these combinations, fewer documents match the agent content. This process reduces the number of Boolean expressions that IDOL server must match.
For example, if your Boolean restriction is:
“New Zealand” AND “French Champagne”
you can set the agent content to:
Zealand
In this Boolean expression, the query text must contain all of the four terms, so you can choose any of them as the agent content. Zealand is the rarest term, so fewer documents match this agent.
For other expressions, you can often similarly choose a minimal list of terms to reduce the number of documents that match against the Boolean restriction.
To find the minimal list of terms
*
Send the DocumentStats action with the QueryStats parameter set to true. For example:
http://localhost:9000/action=DocumentStats&Text=“New Zealand” AND “French Champagne”&QueryStats=true
This action returns a minimal list of terms that text must contain to match the Boolean expression. It provides the rarest terms where possible. The action returns the following information:
<autnresponse>
<action>DOCUMENTSTATS</action>
<response>SUCCESS</response>
<responsedata>
<autn:wildrequired>false</autn:wildrequired>
<autn:numberrequired>1</autn:numberrequired>
<autn:required>ZEALAND</autn:required>
</responsedata>
</autnresponse>
Using this method to choose the agent or category content can improve performance, especially in systems that receive a large number of queries.
Use AlwaysMatchType Fields
For some Boolean expressions that include wildcards and DREFUZZY or SOUNDEX expressions, the minimal list of terms must include a wildcard value.
For these expressions, the DocumentStats action with QueryStats set to true returns the following line:
<autn:wildrequired>true</autn:wildrequired>
The agent or category content cannot contain a wildcard value, so you must use a different approach to ensure that IDOL server checks the Boolean expression.
You can ensure that IDOL server checks any documents that contain one of these expressions by configuring an AlwaysMatchType field and adding it to the agents.
IDOL server always matches the content of an agent or a category that contains an AlwaysMatchType field. It then attempts to match the query text against the Boolean restriction, and returns any agents or categories that match at this stage.
To configure AlwaysMatchType fields
1.
2.
Decide which fields you want to always match. For a document to always match, the AlwaysMatchType field must have a non-empty string.
3.
4.
In the [FieldProcessing] section, add a new process to the list of processes. This process identifies the fields in the IDX or XML document that you want to always match. For example:
[FieldProcessing]
0=SetIndexFields
...
15=SetAlwaysMatchFields
5.
Create a new section for the process you added. In this section, create a property for the process (you define the property later by using configuration parameters). Set PropertyFieldCSVs to a list of fields to associate with the process. For example:
[SetAgentBooleanFields]
Property=AlwaysMatchFields
PropertyFieldCSVs=*/MARKERFIELD
6.
Create a new section for the property you added in which you set AlwaysMatchType to true. This property indicates that IDOL server must always match the associated fields in queries. For example:
[AlwaysMatchFields]
AlwaysMatchType=true
7.
After you configure the AlwaysMatchType field, you must manually create agents as IDX or XML documents.
For each agent, send the Boolean restriction in a DocumentStats action to IDOL server to return the minimal list of terms.
*
*
If the result contains a wildcard, add the AlwaysMatchType field to the agent IDX. For example:
#DREREFERENCE WineAgent
#DRETITLE My Wine Agent
#DREFIELD BOOLEANRESTRICTION=“Champ*” AND “Merlot”
#DREFIELD MARKERFIELD="1"
#DRECONTENT
#DREENDDOC
You can index your agent IDX or XML document into the Agentstore component agent database using a DREADD or DREADDDATA index action. For example:
http://localhost:9051/DREADD?C:\data\Agents.idx&DREDBName=Agent
Related Topics 
*