Share

Siebel Remote Transaction Gaps and Data Routing

 Background

During normal Siebel application operations, transaction gaps are created in the Siebel Remote transaction queue. If the gap lasts longer than the wait period, the transaction represented by the gap will be skipped over and transactions will not be routed to respective mobile clients on the application server. This technical note discusses transaction gaps, their causes, and recommendations on how to minimize impact on a Siebel Remote deployment.

Summary

The following topics are discussed in this document:

            • What is a Transaction Gap?

            • How are Transaction Gaps Created?

            • Detecting Transaction Gaps

            • Recommendations

 What is a Transaction Gap?

A transaction gap is a set of “missing” records in the Master Transaction Log (S_DOCK_TXN_LOG table), as determined by a break in the transaction ID list. The Transaction Processor subsequently processes these transactions. The following example depicts a transaction gap:

TXN_ID

998
999
1000
1002
1003

 There is a transaction gap between transaction 1000 and transaction 1002. The missing transaction has a transaction ID of 1001. A transaction gap can be temporary or permanent. Permanent gaps are created when a transaction is rolled back after transaction ID(s) have already  been obtained. Permanent gaps are harmless and can be ignored. Temporary gaps are those that if skipped, cause those transactions to be omitted during routing to mobile users. The result is that mobile users do not receive the omitted transaction data in their local databases.

How Are Transaction Gaps Created?

Transaction gaps are created because there are many different transactions in a system, each taking different resources and durations to complete. When a transaction is written to the transaction log, a transaction ID (an increasing sequential number) is obtained. These IDs are written, along with transaction data, into the Master Transaction Log (S_DOCK_TXN_LOG table) and are subsequently processed by the Transaction Processor. This is illustrated in the following diagram:

Thus, in a normal system operation, many processes (and their transactions) are vying for transaction IDs. Because the system operates on a “first-come first-serve” basis, a number of possibilities can occur, depending on the characteristics of the transaction involved. These scenarios are described below:

Scenario 1

The transaction updates only one table. It is short and the commit process occurs very quickly. This transaction would not create any gap. An example of this type of transaction is a connected user making updates to the database using a view or screen in the application.

Scenario 2

The transaction updates only one table. It is short but the commit process fails and the transaction is rolled back. A permanent gap with the size of one would be created. This is rare but can occur and is usually caused by a problem in the storage or database system.

Scenario 3

The transaction updates many tables and comprises multiple SQL transactions. Its processing time may take several seconds or minutes. If successful, this transaction creates a temporary gap that lasts for the duration of the transaction. The assumption is that there are other transactions similar to Scenario 1 occurring concurrently. An example of this type of transaction is an online user performing a merge of two accounts using a view or screen in the application.

998

999

1000

1002

1004

OM

EIM

AM

Others

Master Transaction Log

Transaction

Processor

 

Scenario 4

The transaction updates many tables and comprises multiple SQL transactions. Its processing duration may take several seconds or minutes. If it fails and the entire transaction is rolled back, this transaction would create a permanent gap with the size equal to that of the number of SQL transactions. This is rare but can occur and is usually caused by a problem in the storage or database system.

Scenario 5

The transaction is created by Batch Assignment Manager, which commits at a large batch size interval. Its processing operation may take several minutes or hours to complete. If successful, this transaction creates one or more temporary gaps that last for the duration of the transaction, or the time to process the batch size. The number of gaps and their sizes depend on the BatchSize parameter value specified and other activities in the system. Depending on the duration of transaction and the Transaction Processor Gap Pause parameter value, the gaps may or may not be skipped by the Transaction Processor.

Scenario 6

The transaction is created by Batch Assignment Manager, which commits at a large batch size. Its processing duration may take several minutes. If it fails, this transaction creates multiple permanent gaps.

Scenario 7

The transaction is created by Enterprise Integration Manager (EIM) jobs, which comprises multiple SQL transactions. Its processing time can take up to several minutes. If the transaction succeeds, it creates a temporary gap that will last the entire processing time. The gap size can vary from one to several transactions, depending on the type of EIM jobs, the commit size, the batch size (if commit size is not specified or honored), and how they are specified. The assumption is that there are other transactions similar to Scenario 1 or 3 occurring concurrently. Depending on the duration of the commit process and the Transaction Processor Gap Pause parameter value, the gap may or may not be skipped by the Transaction Processor.

Scenario 8

The transaction is created by EIM jobs, which comprise multiple SQL transactions. Its processing time can take up to several minutes to complete. If the transaction is rolled back by the system, it would create multiple permanent gaps.

Detecting Transaction Gaps

With Siebel version 7.7, or in Siebel environments that include fixes for Change Requests 12-ILOBGN or 12-ILPP0R, the following warning messages appear in the Transaction Processor log file when a transaction gap is detected and skipped:

GenericLog GenericError 1 0 2003-11-18 17:04:51

WARNING: A transaction gap has been detected after transaction 122.

Probable Cause: There maybe long running transactions in your system which are not committing transactions within the specified duration (600 sec)

 

Technical Note 499: Siebel Remote Transaction Gaps and Data Routing page 4 of 5

Recommendation: Reduce the batch size of your transactions. This will allow the transactions to be committed to the database within the wait time window.

GenericLog GenericError 1 0 2003-11-18 17:13:51

WARNING: The Gap Wait Time of 600 sec has been exceeded. 2 transactions have been skipped after transaction 122.

Probable Cause: There maybe long running transactions in your system which are not committing transactions within the specified duration (600 sec)

Recommendation: Reduce the batch size of your transactions. This will allow the transactions to be committed to the database within the wait time window.

In other versions, the only mechanism for detecting these events is by monitoring the transaction log (S_DOCK_TXN_LOG table) closely or from reports of missing data from mobile users.

Recommendations

Transaction gaps are unavoidable in the Siebel application given the nature of how the application is used. They are especially severe when OLTP transactions created by online users are intermingled with batch transactions created by Assignment Manager or EIM jobs. Monitor the Transaction Processor log files for warning messages of transactions being skipped because of transaction gap. If many of them appear, use the approaches describe below, either individually or in combination, to minimize impact of these transaction gaps.

            1. Reduce the size of data loads to ensure that the processing time is short.

 

Application component

Recommendation

Batch Assignment Manager              • Adjust the BatchSize parameter accordingly.            • Consider a low value, such as 100, for BatchSize to ensure that the processing duration is shorter than the Transaction Processor’s Gap Pause duration, which defaults at 10 minutes.

 

EIM              • Specify a small “Commit Operation” value of less than 100.            • Use a small batch size of 10 or less, especially for Merge operations. For other operations such as insert or updates, 100 is recommended so that the processing duration is shorter than the Transaction Processor’s Gap Pause duration which defaults to 10 minutes. Note that there is a trade-off in performance for this setting, however.

 

UI Import              • Include small numbers of records in each import file (100 or fewer). 
Custom scripts              • Commit frequently.            • Keep transaction size as small as possible.

 

 

            2. Increase the gap wait time to a period in which all long-running (batch) transactions can be completed. The parameter is called “Gap Pause” in Siebel version 7.7 and “_GAP_PAUSE_” (hidden parameter) in previous versions.

 

Technical Note 499: Siebel Remote Transaction Gaps and Data Routing page 5 of 5

For example, if the longest transaction in the deployment is 30 minutes, then setting the wait period to 35 minutes should prevent the majority of the temporary gaps from being skipped. The trade-off is that for every gap, both permanent and temporary, that is detected, the Transaction Processor can potentially wait for 35 minutes. The net effect is that system throughput of the Transaction Processor and Siebel Remote is reduced.

To set the Gap Pause parameter to 35 minutes or 2100 seconds for the Transaction Processor, use the following server manager (CLI) command before starting the component on any server.

            a. Login to the Enterprise server (without specifying the server) and issue the following command:

 

> change parameter _GAP_PAUSE_=2100 for component Txnproc

            b. Restart the Transaction Processor component.

 

To determine processing time for a batch Assignment Manager (BatchSize = 100) job, turn on the Performance event logs for the component using the following CLI server manager command:

> change evtloglvl Performance=4 for component AsgnBatch

Performance statistics, including the transaction processing duration, are written into the component log file. For an EIM merge, the total time for the job to complete is a good approximate for the transaction duration because EIM merge processes only commit at the end of the job.

            3. Turn off the Transaction Processor on all Siebel Remote servers while large batch jobs are being run. This will ensure that transactions created by these jobs are routed appropriately to mobile users once the Transaction Processors are turned back on after the data load.

 

Finally, if transaction gaps are detected on a particular application server at about the same time that batch jobs were being executed, mobile users associated with that application may experience missing data. To ensure that mobile users receive all appropriate data, consider re-extracting the mobile databases. If there were no batch jobs running during that period, these gaps may be permanent and are harmless. The probability that mobile users on the application server did not receive the appropriate transactions is very low. Thus, it is important that customers are aware of all activities in the system and assess the situation appropriately.

Leave A Reply

Your email address will not be published. Required fields are marked *

error: Content is protected !!