Kourier Integrator Online Help

Kourier Import Processing Guide

This guide will help you understand how Kourier processes data that is being imported into a UniData/UniVerse database using either XML to U2 Import Specification File to U2 Import Specification, or Rest Import Specification.

Once the Kourier Import Processor has parsed a transaction into records, each record is processed in the following manner by the KMK.IMPORTRECORD subroutine. For more information, refer to the Kourier Import Services Guide topic.

Import Specification Event Handlers can be used to provide a method for adding specific logic, data manipulation, and invocation of application programming interfaces (APIs) while preparing to write incoming data to a U2 file. Each Event Handler has a specific timing and usage during an import, with access to the Kourier Import variables to manage the data.

Initialization

During this step in the processing of a record, all variables accessible to the UniBASIC programmer are set to their initial state as shown below. These variables are declared in the IMPORT.COMMON item stored in the KORE-INCLUDES file.

Variable	Description - Usage - Notes	Initialized Value
FILENAME$	The name of the file where records are to be written or deleted.	(e.g. ORDER-HEADER)
F.FILENAME$	UniBASIC file pointer to the name of the file where records are to be written/deleted.
IHTTPHDRS$	For REST based imports, the HTTP header fields passed in the request stored in standard header format.
IMPORTSUB$	The name of the import subroutine that implements event and validation logic. These subroutines are created automatically when an import specification is saved and are stored in the KT_IMPORTSUB file.	(e.g. KTIMPORTSUB$BBHR_SETLEAVE)
INTGID$	Determined from the name of the import specification being processed. For example, if the import specification was named BBHR_SETLEAVE the value of this variable would be BBHR.	(e.g. BBHR)
ISATEST$	If running in TEST mode from the GUI this variable is set to 1, otherwise it is set to 0.
IUD$	Indicates is the record should be inserted (I), updated (U), upserted (IU) or deleted (D). For imports that are executed via the REST interface the value of this variable is determine by the HTTP verb specified. When an import is process from a service this variable is set to IU.	(e.g. From REST - I; From Service - IU)
MSGHDRS$	Message headers, in standard header format, associated with the transaction or record being processed.
OHTTPHDRS$	For REST based imports, the HTTP header fields to return for the request stored in standard header format.
OLDRECORD$	A copy of the record read from disk before any changes are made to the current record	(empty string)
QUERYHDRS$	For REST based imports, the query parameters passed in the URL stored in standard header format.
RECISNEW$	Set to 1 if the record was read from disk, otherwise set to 0.	0
RECISLOCKED$	Set to 1 if the record is locked, otherwise set to 0.	0
RECORD$	The (current) record read from disk.	(empty string)
RID$	The item-id of the record to be read, written or deleted. This variable is populated from fields that reference a Position of 0.	(empty string)
RTNDATA$	For REST based imports, this variable can contain any information that should be returned to the HTTP request.
RTNEXPORTID$	For REST based imports, this variable contains the Response export item-id
SKIP$	This variable can be set to 1 to skip the next logical step in processing an import or a field.	0
TRANSID$
USER$(10)	This is a UniBASIC dimensioned array containing 10 storage locations that is reserved for user-written code. It can be used to pass information between event handlers and fields. It does NOT persist between the processing of each record to be imported.	(empty string)
XID$	The item-id of the remote record/row that is being processed. This variable is only assigned a value thru user-written UniBASIC code.	(empty string)
XMLRECORD$	For XML, the record to be processed. For flat-files the rows that make up a record. See the Import Service Guide for more details.
XMLTRANS$	For XML, the transaction to be processed including the transaction tag. For flat-files the rows that make up the transaction to be processed. See the Import Service Guide for more details.
Error Variables These variables are used to communicate errors between user-written code and the Kourier import engine. There are two types of errors: FATAL errors will stop processing by the import engine immediately. Examples of fatal errors are files not opening, parsing errors, etc. APPLICATION errors allow the import engine to continue processing so that all application errors can be displayed at one time. Examples of application errors are invalid codes, non-numeric data, etc. If any FATAL or APPLICATION error is returned, the transaction will be flagged as not loaded and will be treated as an exception.
APP.ERR$	This variable can be used to indicate an application error in either a formula or an event handler. It is a text string.	(empty string)
APP.ERR.CNT$	This is a READ ONLY variable. It contains the number of application errors encountered while processing to import.	0
APP.ERR.TXT$	This is a READ ONLY variable. It contains a list of application errors delimited by a value-mark.	(empty string)
FATAL.ERR$	Fatal error message number. The value of this variable must be an item-id to a record in the KORE-MESSAGES file.	(empty string)
FATAL.PARMS$	Fatal error message parameters. Each parameter is delimited by a value-mark.	(empty string)
Field Level Variables These variables have context when accessed from field level validation code.
ANO$	Attribute number.	{undefined}
EDITS$	Validation Edits.	{undefined}
FIELD$	Current value of a field.	{undefined}
MAX$	Maximum field length. A value of 0 indicates that the field length is unlimited.	{undefined}
MIN$	Minimum field length. A value of 0 means the field is NOT required. All other values indicate that the field is required and it must contain at least that number or characters.	{undefined}
NAME$	When processing a field, this variable contains the field name. When processing an event handler, this variable contains the name of the event enclosed by {}.	{Initialize}
SVNO$	Sub-value number.	{undefined}
VNO$	Value number.	{undefined}

Once all variables are initialized, the lines of the import specification are grouped as either record keys (those lines that have a Position referencing attribute 0) or fields (those lines that do not reference attribute zero). The lines are added to each grouping in the same order in which they are specified in the import specification.

Reading the Record

In this step, the fields that make up the record key (item-id) are processed thru the standard field loading and editing process (described below). If a FATAL error has not been raised processing continues.

The edited field value(s) are then used to set the value of the RID$ variable. This variable will be used to read, write or delete the record.

If the Before Read Handler has been specified, the NAME$ variable is set to {Before Read} and the user written UniBASIC code is executed. The Before Read Event can be used for:

Overriding the READ processing of the import engine
Assigning the value of the RID$ variable.
Initializing user-defined COMMON variables.

Once the Before Read Event logic has executed, and if a FATAL error has not been raised, processing continues.

Normally, the RID$ variable will either been set by Kourier or by the user-written code in the Before Read Event. If the SKIP$ variable has not be set, and the value of the RID$ variable is not empty, Kourier will attempt to read the record from the target file and update the following variables:

Variable	Value/Notes
OLDRECORD$	Assigned the value of RECORD$
RECISLOCKED$	Set to 1 if the record could be locked otherwise set to 0.
RECISNEW$	Set to 1 if the record was not read from disk otherwise set to 0.
RECORD$	Either the record read from disk or empty.

If the SKIP$ variable was set, which indicates that the user-written code performed the read operation, the variables shown above must be set appropriately by the user-written code.

If the value of RID$ was empty (because no field referenced 0 in the Position field), this indicates that the RID$ variable will be assigned a value in the Before Update Handler and that the read operation should not be attempted. In this case, the variables shown above will be set as follows:

Variable	Value/Notes
OLDRECORD$	Assigned the value of RECORD$
RECISLOCKED$	0
RECISNEW$	1
RECORD$	(empty string)

If the value of the IUD$ variable was set to “IU” (which supports the concept of an upsert) and the value of RECISNEW$ is set to 1 then the IUD$ variable will be set to “I” otherwise it will be set to “U”. Next the value of the IUD$ variable is compared to the configured Supported Options and if they are inconsistent a FATAL error is raised and processing is terminated. Additionally a check is made to make sure the value of the IUD$ variable is consistent with how the record was read. For example, if the value of IUD$ is “U” and the value of RECISNEW$ is set to 1 a FATAL error is returned because an update operation is not allowed for a new record.

If the After Read Handler has been specified, the NAME$ variable is set to {After Read} and the user written UniBASIC code is executed. The After Read Handler can be used for:

Assigning default values in the RECORD$ variable
Initializing user-defined COMMON variables.

Loading Fields

Once the After Read Handler logic has executed, and if a FATAL error has not been raised, and the SKIP$ variable has not been set, processing continues.

In this step, each line that makes up the fields of the record are processed thru the standard field loading and editing process (described below). If a line is updating a multi-valued field (that may or may not be part of an association) all values are loaded for that field before moving to the next field.

Once all fields have been loaded and a FATAL error has not been raised, processing continues with either Saving the Record if the value of the IUD$ variable is either "I" or "U" or Deleting the Record if the value of the IUD$ variable is "D".

Saving the Record

The value of the NAME$ is set to {BeforeUpdate}.

If the Before Update Handler has been specified the user-written UniBASIC code is executed. The Before Update Handler can be used for:

Overriding the WRITE processing of the Import engine
Assigning the value of the RID$ variable.
Assigning the value of the XID$ variable.
Calling applications specific routines that need to run when a record is written

When testing an import, the Before Update Handler is only called if the “Write Records to Disk” option has been enabled.

Once the Before Update logic has executed and if a FATAL error has not been raised processing continues.

If the SKIP$ variable is not set, Kourier verifies that the value of the RID$ variable is not empty and that the value of the APP.ERR.CNT$ is 0 before writing RECORD$ to the Target File and releasing the lock. If the RID$ variable is empty a FATAL error is raised and processing stops.

If the SKIP$ variable is set, this indicates that the user-written code performed the write operation and no additional processing is performed by Kourier.

Deleting the Record

The value of the NAME$ is set to {BeforeDelete}.

If the Before Delete Handler has been specified the user-written UniBASIC code is executed. The Before Delete Event can be used for:

Overriding the DELETE processing of the Import engine
Calling applications specific routines that need to run when a record is deleted

When testing an import, the Before Delete Handler is only called if the “Write Records to Disk” option has been enabled.

Once the Before Delete logic has executed and if a FATAL error has not been raised processing continues.

If the SKIP$ variable is not set, Kourier verifies that the value of the RID$ variable is not empty and that the value of the APP.ERR.CNT$ is 0 before deleting the RID$ from the Target File and releasing the lock. If the RID$ variable is empty a FATAL error is raised and processing stops.

If the SKIP$ variable is set, this indicates that the user-written code performed the delete operation and no additional processing is performed by Kourier.

After Saving/Deleting a Record

The value of the NAME$ variable is set to {WrapUp}.

If the Wrap Up Event has been specified, the user-written UniBASIC code is executed. The Wrap Up Event can be used for:

Generating return data that includes the full representation of a resource after it has been created.
Generating return data for REST-based imports when an action (e.g. customer specific pricing) is being processed.

When testing an import, the Wrap Up Event Handler is always called even if the “Write Records to Disk” option has NOT been enabled.

The Wrap Up Event Handler is called even if application errors (e.g. APP.ERR.CNT$ is greater than zero) have been detected. In general, you should test that variable before executing your code.

Once the Wrap Up logic has executed and if a FATAL error has not been raised, processing continues and the transaction log is updated (if enabled).

Transaction Logging and Exception Handling

Regardless of whether Kourier writes or deletes the record or user-written code writes or deletes the record the following variables should be assigned meaningful values since they are used by the transaction logging routines:

Variable	Value/Notes
FILENAME$	The name of the file updated
IUD$	Set to “I” or “U” or "D" to reflect the operation that was performed.
RID$	The record key (item-id) of the record updated.
XID$	If the integrating application provides a unique identifier for this record this variable should contain that string.

Finally, the KMK.IMPORTRECORD subroutine returns to the calling program.

Field Loading and Editing

Before processing each field the following variables are initialized to the values shown:

Variable	Value/Notes
ANO$	Attribute number determine from the Position expression.
APP.ERR$	(empty string)
EDITS$	Input edits.
FATAL.ERR$	(empty string)
FATAL.PARMS$	(empty string)
FIELD$	The value of the field.
MAX$	The maximum length for the field.
MIN$	The minimum length for the field.
NAME$	The field name.
SKIP$	0
SVNO$	Sub-value number determine from the Position expression.
VNO$	Value number determine from the Position expression.

If the field has Validation logic, the user written code is executed. Validation logic can be used for:

Conditionally making field be required.
Validating a field’s value and raising either application or FATAL errors.
Setting the default value for a field.

Once the Validation logic has executed and if a FATAL error has not been raised processing continues.

If APP.ERR$ is not empty, the application error is counted and appended to the APP.ERR.TXT$ variable.

If the SKIP$ variable is set no further processing is performed and the value of the FIELD$ variable is NOT updated in the RECORD$ variable.

If the SKIP$ variable is not set, the value of the FIELD$ variable will be processed as follows:

Apply all the edits specified for the field. If the value of the FIELD$ variable fails an edit an application error is counted and appended to the APP.ERR.TXT$ variable.
The length of the value in FIELD$ is compared to the minimum length specified for the field, and if the length is less than the value of the MIN$ variable an application error is counted and appended to the APP.ERR.TXT$ variable.
The length of the value in FIELD$ is compared to the maximum length specified for the field and if the length is greater than the value of the MAX$ variable an application error is counted and appended to the APP.ERR.TXT$ variable.