Boolean and Proximity Search

You can use the Query action to submit standard Boolean queries to IDOL Server, and to submit proximity queries that allow you to give words that appear close together in the search string a higher weighting.

IDOL Server uses APCM (Adaptive Probabilistic Concept Modeling) to rank the results.

NOTE:

You must specify all search operators in capital letters.

Boolean Operators

You can apply the operators in the following table to words, exact phrases, or other Boolean expressions in your query.

AND

Binary operator. Both terms must match in every document that returns. For example:

action=Query&Text=cat AND dog

This query returns only documents that contain both cat and dog.

NOT

Unary operator. Excludes the term that follows NOT from all the returned documents. For example:

action=Query&Text=cat NOT dog

This query returns only documents that contain cat but not dog.

NOTE:

To use NOT to exclude multiple terms, you must use brackets. Otherwise, NOT applies only to the term that immediately follows it. To use NOT to exclude a phrase, you must put the phrase in quotation marks. For example:

Document 1: I went to the city for the New Year

Document 2: I went to New York City for the New Year

The following query does not match either of the above documents:

action=Query&Text=city NOT (New York)

The following queries match the first document but not the second:

action=Query&Text=city NOT ("New York")
action=Query&Text=city NOT "New York"
OR

Binary operator. One or both terms must appear for the document to return. This option is the default behavior if you do not specify an operator between two terms. For example:

action=Query&Text=cat OR dog

This query returns documents that contain cat, dog, or both terms.

EOR or XOR

Binary operator. Logical exclusive OR. A document must contain only one of the terms for it to return. This operator is rarely used. For example:

action=Query&Text=cat XOR dog

This query returns only documents that contain either the term cat or the term dog. Documents that contain both cat and dog do not return.

()

Bracketed expressions. IDOL Server evaluates brackets from left to right. You can nest bracketed expressions. Brackets dictate the precedence and behavior of combined operator statements. For example:

action=Query&Text=(fish EOR pie) AND (chips EOR mash)

This query returns documents that contain one of the following combinations:

fish and chips

fish and mash

pie and chips

pie and mash

Proximity Search Operators

You can apply the operators in the following table to words, exact phrases, or other Boolean expressions to run a proximity search.

NOTE:

If the two specified words are adjacent to each other, their proximity is 1, if one word separates them, their distance is 2 and so on.

By default, the proximity ignores stop words (for example, because and is a stop word, the terms cat and dog have a proximity of 1 both in the text cat dog and cat and dog). However, if you set the StopWordIndex configuration parameter to index stop words, HPE Content Component includes stop words in the proximity calculation.

NOTE:

Proximity operators work recursively so that nested Boolean queries can have proximity operators apply to brackets or phrases. For example, in the expression

(term1) NEAR10 ((term2) DNEAR2 (term3))

the NEAR10 operator ensures that term1 is in proximity to an occurrence of term2 within two of term3.

NEARN

Returns only documents in which the second term is within N words of the first term; that is, the terms are N or fewer words apart. If you do not specify N, NEAR defaults to 5. For example:

action=Query&Text=red NEAR1 green

This query returns only documents in which the term red is adjacent to the term green. Documents that contain red green or green red return. Documents that contain red orange green do not return (the terms are not close enough to each other).

DNEARN

Directed NEAR. Returns only documents in which the second term is within N words of the first term, in the specified order. If you do not specify N, DNEAR defaults to 5. For example:

action=Query&Text=red DNEAR2 green

This query returns only documents in which the term green follows the term red, and is no more than two words away from the term red. Documents that contain red orange green return. Documents that contain green orange red or red orange blue green do not return.

WNEARN

Weighted NEAR (with OR operation). Proximity operator that returns documents that contain either of the two terms. It promotes relevance when term spacing is less than the specified N word distance (closer together implies higher relevance). If you do not specify N, WNEAR defaults to 5. For example:

action=Query&Text=red WNEAR7 green

This query returns documents that contain the term red or the term green. It gives extra relevance to documents in which red and green appear seven or fewer words apart in a piece of text. This weight increases as the terms get closer to each other. Documents in which the terms occur more than seven words apart, or in which only one term occurs, return with the normal relevance.

YNEARN

Weighted NEAR (with AND operation). Proximity operator that returns documents that contain both of the terms. It promotes relevance when term spacing is less than the specified N word distance (closer together implies higher relevance). If you do not specify N, YNEAR defaults to 5. For example:

action=Query&Text=red YNEAR7 green

This query returns only documents that contain both the term red and the term green. Extra relevance is given to documents in which red and green appear seven or fewer words apart in a piece of text. This weight increases as the terms get closer to each other. Documents in which the terms occur more than seven words apart return with the normal relevance.

BEFORE

Returns only documents in which the first term precedes the second one. For example:

action=Query&Text=red BEFORE green

This query returns only documents in which the term green appears later than the term red.

NOTE:

For a FieldText query with the BEFORE operator to successfully compare two occurrences of the same field, you must set the XMLFullStructure configuration parameter to True in the IDOL Server configuration file [Server] section.

AFTER

Returns only documents in which the first term appears later than the second one. For example:

action=Query&Text=red AFTER green

This query returns only documents in which the term red appears later than the term green.

NOTE:

For a FieldText query with the AFTER operator to successfully compare two occurrences of the same field, you must set the XMLFullStructure configuration parameter to True in the IDOL Server configuration file [Server] section.

XNEAR

Returns only documents in which the second term is exactly N words from the first term. For example:

action=Query&Text=cats+XNEAR2+dogs

This query returns only documents in which the term dogs follows the term cats and is exactly two words away from the term cats. Documents that contain, for example, cats scare dogs return, while documents that contain dogs eat cats or cats, dogs do not return.

This operator is available only for the Text action parameter.

SENTENCE

Returns only documents in which the second term is in the same sentence as the first term. For example:

action=Query&Text=cats+SENTENCE+dogs

This query returns only documents in which the term dogs occurs in the same sentence as the word cats.

HPE Content Component breaks the document into sentences by using a number of criteria. The most important criteria is the detection of an end of sentence marker, which includes a period (.), question mark (?), or exclamation point (!), as well as their multibyte variants. However, the presence of one of these characters is not always sufficient to mark the end of a sentence, because these characters are often used in abbreviations, names, and other items for purposes other than the end of a sentence. To locate a more accurate sentence boundary, HPE Content Component also uses characteristics such as capitalization and syntactic observations.

SENTENCENN

Returns only documents in which the second term is in the same sentence as the first term, and they are within NN words of each other. For example:

action=Query&Text=cats+SENTENCE10+dogs

This query returns only documents in which the term dogs occurs in the same sentence as, and within 10 words of, the word cats.

NOTE:

SENTENCE0 has the same behavior as SENTENCE.

DSENTENCE

Returns only documents in which the second term occurs later than the first term, in the same sentence. For example:

action=Query&Text=cats+DSENTENCE+dogs

This query returns only documents in which the term dogs occurs later in the same sentence than the word cats.

DSENTENCENN

Returns only documents in which the second term occurs later than the first term, and within NN words in the same sentence. For example:

action=Query&Text=cats+DSENTENCE10+dogs

This query returns only documents in which the term dogs occurs later in the same sentence as, and within 10 words of, the word cats.

NOTE:

DSENTENCE0 has the same behavior as DSENTENCE.

PARAGRAPH

Returns only documents in which the second term is in the same paragraph as the first term. For example:

action=Query&Text=red+PARAGRAPH+green

This query returns only documents in which the term green occurs in the same paragraph as the word red. The words do not have to occur in the same sentence in the paragraph.

PARAGRAPHNN

Returns only documents in which the second term is in the same paragraph as the first term, and they are within NN words of each other. For example:

action=Query&Text=cats+PARAGRAPH20+dogs

This query returns only documents in which the term dogs occurs in the same paragraph as, and within 20 words of, the word cats.

NOTE:

PARAGRAPH0 has the same behavior as PARAGRAPH.

Other Search Operators

If you set XMLFullStructure to True in the [Server] section of the IDOL Server configuration file, you can use the following operators in Text and FieldText queries to return XML documents in which fields or attributes occur together, or do not occur together.

WHEN Return only XML documents in which two fields that have the same parent field contain specified terms or phrases.
WHENN Return only XML documents in which two fields that have the same parent field at N levels from the root level contain specified terms or phrases.
NOTWHEN Return only XML documents in which the first field and value pair occurs, but the second field and value does not occur in the same parent field.

Examples

Consider the following three XML documents:

Document 1:

<DOC>
   <car>
      <make>audi</make>
      <color>red</color>
   </car>
   <car>
      <make>mercedes</make>
      <color>silver</color>
   </car>
</DOC>

Document 2:

<DOC>
   <car>
      <make>audi</make>
      <color>silver</color>
   </car>
   <car>
      <make>mercedes</make>
      <color>red</color>
   </car>
</DOC>

Document 3:

<DOC>
   <car>
      <make>audi</make>
      <body> 
         <color>red</color>
      </body>
   </car>
   <car>  
      <make>mercedes</make>
      <body> 
         <color>silver</color>
      </body>
   </car>
</DOC>

Nested Expressions

You can use complex nested expressions with the WHEN and NOTWHEN operators in query text. For example:

action=Query&Text=(London:CITY WHEN English:LANG) WHEN ("United Kingdom":COUNTRY)
action=Query&Text=(Lincoln:CITY NOTWHEN Nebraska:STATE) NOTWHEN ("United Kingdom":COUNTRY)

Field Attributes

You can use WHEN and NOTWHEN to return only XML documents in which two attributes that occur in the same field contain (or do not contain) specified terms or phrases.

For example:

You can also use WHEN and NOTWHEN to return only XML documents in which a field contains a specified term or phrase, and has an attribute that has a specific value.

For example:

action=Query&Text=Fr.html:_ATTR_HREF WHEN France:A

This query returns only XML documents in which the A field contains the value France, and has an HREF attribute with the value Fr.html.

The following document, for example, returns in this query:

<XML>
   <DOC>
      <A HREF="Fr.html">France</A>
   </DOC>
</XML>

The following document does not return:

<XML>
   <DOC>
      <A HREF="France.html">France</A> is next to <A HREF="Fr.html">Belgium</A>
   </DOC>
</XML>

_HP_HTML5_bannerTitle.htm