Fields and tokens need to be converted from the repository’s character set into the locale’s internal character set. The locale’s character set is specified in the charSetName member of the VgwSessionNewArgRec structure, which is available to your gateway driver’s VgwSessionNewFnc function.

Your gateway driver’s VgwStreamGetTokenFnc function can create a field token (VdkTokenType_Field) for the internal Charset field, which identifies the character set of the stream. Your gateway driver’s VgwStreamGetTokenFnc and VgwDocFieldRead functions must convert field and buffer values as well as the names of fields, zones, and other identifiers.

You can use the I18N library to perform the conversion. You typically call the Vi18nInit function from your gateway driver’s VgwSessionNew callback function to initialize the library and call the Vi18nClose function from your gateway driver’s VgwSessionFree callback function to release resources.

You call the Vi18nConvert function to perform the conversion. The Verity engine determines whether actual conversion is required; the Vi18nConvert function only performs conversion if required; otherwise, the function does not perform any work. You may need to call the Vi18nGetExpansionFactor function to calculate the size of the buffer when converting between character sets with different byte sizes; for example, from a single-byte character set to a double-byte character set or vice versa. For more information about the I18N library, including other functions that you can use, see Verity I18N Library.