VgwDocSetReadFnc


Use the VgwDocSetReadFnc function to return a table of document fields. The Verity engine calls your VgwDocSetReadFnc function when performing a crawling operation.


Syntax

VdkError VgwDocSetReadFnc (
   VgwDocSet vgwDocSet,
   VdkUint4 startRow,
   VdkInt4 numRows,
   VgwDataTable* pvgwTable,
   VdkUint4* poutFlags)

Arguments

 


vgwDocSet

VgwDocSet A gateway document set handle.

startRow

VdkUint4 The starting zero based row number.

numRows

VdkInt4 The number of rows to read or -1 to specify all available rows.

pvgwTable

VgwDataTable* A pointer to a gateway VgwDataTable handle.

poutFlags

VdkUint4* A pointer to a flag with the following valid values OR'ed together:

VgwDocSetOut_Finish - no more folder or key items to read.


Member Descriptions

 

Table 8-2    VgwDataTableRec Members


session

VgwAppSession The session handle, which is passed in via the vgwSession argument.

numRows

VdkUint4 The number of documents in the values array.

numColumns

VdkUint2 The number of fields in the values array.

columnArray

VgwColumnInfo* A pointer to an array of VgwColumnInfo pointers that contain information about each field. (See VgwColumnInfoRec members below.)

values

VdkVoidp* A pointer to a table of values.

internal

VdkVoidp For internal use by the gateway.


 

Table 8-3    VgwColumnInfoRec Members


type

VdkFieldType The field data type flag. Valid values are:

VdkFieldType_Invalid for unsupported field names;

*_Text for text strings;

*_Signed for signed integers;

*_Unsigned for unsigned integers;

*_Date for VdkDate type data;

*_Float for double floats;

*_Stream for a binary blob of data.

name

VdkCString Name of the external field.

size

VdkUint4 The data size, in bytes, or 0 for ASCIIZ (null-terminated strings).

flags

VdkUint2 Behavior flags, which are ORed together to specify how to treat this field when it is retrieved from the repository. You can set the following values:

VgwColumn_ReadOnly for field values that are not to be changed.

*_Inline for fields that contain the actual value, not a pointer to the value.

*_PrimaryKey for fields whose value is the primary key for the gateway; for example, the VdkVgwKey field should have this flag set

*_RedirectFile for fields whose value is a the full pathname of a data file.

*_Redirect_URL for fields whose value is a URL to a data file.

*_Attachment for fields whose value is the full pathname to an attachment or subdocument.

internal

VdkVoidp For internal use by the gateway.

precedence

VdkInt2 Reserved for future use.



Returns

This function must return one of the following error codes:

VdkSuccess for success

 

VdkError_InvalidDoc if a folder or document key does not exist

 

VdkError_* for a standard Verity Developer Kit API error as described in the Verity Developer’s Kit Programming Reference

 


Discussion

Your VgwDocSetReadFnc function fills the table specified by the pvgwTable argument with documents that have been retrieved from the repository. Conceptually, each row in the table represents a document and each column represents a piece of metadata from the document. The most common piece of metadata is the primary VdkVgwKey key associated with the document.

When there are no more folders or documents to read, the VgwDocSetReadFnc function must set the pOutFlags argument to VgwDocSetOut_Finish; this flag is set on the last read in which documents or folders are returned in the table specified by the pvgwTable argument. For reads that occur after the last read that returned data, meaning that all documents or folders have been retrieved, the VgwDocSetReadFnc function must also set the numRows member of the VgwDataTableRec specified by the pvgwTable argument to 0.


Example

static VdkError
VDK_CALLBACK VgwDocSetRead(VgwDocSet pVDSet, VdkUint4 nStart,
VdkInt4 nRows, VgwDataTable* ppTable, VdkUint4* poutFlags)
{
VdkUint4 i, n;
VdkUint2 j;
VgwColumnInfo pInfos;
VgwDataTable pTable = NULL;

if (!ppTable)
return VdkError_InvalidArgs;
  *ppTable = NULL;
    if (poutFlags)
*poutFlags = 0;

if( !pVDSet || (nRows < -1) )
return VdkError_InvalidArgs;

assert(pVDSet->pSession);
assert(pVDSet->columnCount > 0);

/*****************************************************
* gateway allocates the table, values array, and the
* VgwColumnInfo array and structs
*****************************************************/

if( !(pTable = malloc(sizeof(VgwDataTableRec))) )
return VdkError_OutOfMemory;
VdkStructInit(pTable);

pTable->session = pVDSet->pSession;

/ *****************************************************************
 * initialize the data table for multiple rows and multiple columns *****************************************************************/

pTable->numRows = nRows;
pTable->numColumns = pVDSet->columnCount;

if( !(pTable->columnArray = (VgwColumnInfo*)
malloc(sizeof(VgwColumnInfo)*pTable->numColumns)) ){
free(pTable);
return VdkError_OutOfMemory;
}

if( !(pInfos = (VgwColumnInfo)
malloc(sizeof(VgwColumnInfoRec)*pTable->numColumns)) ){
free(pTable->columnArray);
free(pTable);
return VdkError_OutOfMemory;
}

for( j=0; j < pTable->numColumns; j++ ){
pTable->columnArray[j] = &pInfos[j];

/ **************************************************************
 * populate the field column info for each of the columns here
 * sample gateway reads field column info from static array
**************************************************************/

*(pTable->columnArray[j]) = *(pVDSet->columnArray[j]);
}

/****************************************************
* sample gateway does not put a limit per batch, so
* just always sets how many entries is in the queue
****************************************************/

if (nRows == -1)
pTable->numRows = pVDSet->keycount;

if( !(pTable->values = (VdkVoidp*)malloc
(sizeof(VdkVoidp)*(pTable->numRows*pTable->numColumns+1))) ){
free(pTable->columnArray[0]);
free(pTable->columnArray);
free(pTable);
return VdkError_OutOfMemory;
}

if (pTable->numRows == 0) {
*ppTable = pTable;
return VdkSuccess;
}
else if (pTable->numRows < 0) {
free(pTable->values);
free(pTable->columnArray[0]);
free(pTable->columnArray);
free(pTable);
return VdkError_InvalidArgs;
}
n = nStart + pTable->numRows;

/***************************************************************
* reads field column info from static array and populate value
   **************************************************************/

for( i = nStart; i < n; i++ ){

if (i >= pVDSet->keycount) {
pTable->numRows = i - nStart;

/*************************************
* no more keys or folders to read in
*************************************/

if (poutFlags)
*poutFlags |= VgwDocSetOut_Finish;

*ppTable = pTable;
return VdkSuccess;
}

    for( j = 0; j < pTable->numColumns; j++ ){
       /*********************************
* an invalid external field type
*********************************/
if (pTable->columnArray[j]->type == VdkFieldType_Invalid)
pTable->values[j] = NULL;

/***************************
* populate VdkVgwKey entry
***************************/

else if (!stricmp(VDK_VGWKEY,pTable->columnArray[j]->name)){

if( !(pTable->values[j] =
malloc(strlen((const char*)pVDSet->queue[i].vdkkey)+1))){
free(pTable->values);
free(pTable->columnArray[0]);
free(pTable->columnArray);
free(pTable);
return VdkError_OutOfMemory;
}
strcpy((char*)pTable->values[j],
(const char*)pVDSet->queue[i].vdkkey);

/*******************************
* decide if document is public
*******************************/

} else if (!stricmp(VGWF_SECFLAG,
        pTable->columnArray[j]->name)) {

/**********************************
* assume all documents are secure
**********************************/

pTable->values[j] = (VdkVoidp)VGW_SECURITY_CHECK_ACCESS;

/*****************************
* gather document acl labels
*****************************/

} else if (!stricmp(VGWF_ACLCACHE,
        pTable->columnArray[j]->name)) {

/*************************************************
* assume all documents have FIXEDACL access list
* which is equivalent to USERUID ('a'+'b'+'c')
*************************************************/

pTable->values[j] = (VdkVoidp)FIXEDACL;

/*******************************************************
* copy the key value to field values in sample gateway
*******************************************************/

} else if( !stricmp(VGWF_DOCKEY,
        pTable->columnArray[j]->name) ||
!stricmp(VGWF_VWURL, pTable->columnArray[j]->name) ){

if( !(pTable->values[j] =
malloc(strlen((const char*)pVDSet->queue[i].vdkkey) + 1)) ){
free(pTable->values);
free(pTable->columnArray[0]);
free(pTable->columnArray);
free(pTable);
return VdkError_OutOfMemory;
}
strcpy((char*)pTable->values[j],
(const char*)pVDSet->queue[i].vdkkey);

/*********************************
* an invalid external field name
*********************************/

} else {

pTable->columnArray[j]->type = VdkFieldType_Invalid;
pTable->values[j] = NULL;
}
}
}

*ppTable = pTable;
return VdkSuccess;
}