How Does the Message Grouping Option Affect File Transfer

Metastorm

The information in this article applies to:

Product: MIM and PM4Data
Version: 8.1.4 and above
Platform: all

Information

• The File Transfer Service splits data into data messages that are transferred over WebSphere MQ. If the data is too large to fit into a single data message, then multiple messages are utilized. Each message is assigned a unique Message Id that is found in the WebSphere MQ Message Header (MQMD). This allows a Receiving File Transfer Service to read selective messages from the queue specific to the transfer request being processed and in the correct order.

• Starting with version 8.1.4, a change was made to enable the use of WebSphere MQ Message Groups for grouping multiple messages and identifying them by the same Group ID rather than using individual messages with multiple Message IDs. This capability must be explicitly configured for the File Transfer Service or specified on a specific Transfer Request.

• The grouping attribute for an FTF command line File Transfer Request overrides the configuration for the File Transfer Service.

• By default, grouping is disabled for backwards compatibility; however, this may be modified in the future.

• The use of message grouping is beneficial for several reasons:

  1. The retrieval of messages by the File Transfer Service is more efficient when processing group messages, which results in better performance.
  2. The size of the Workflow messages is greatly reduced by using group messaging. If a transfer results in 1000 data messages, the Workflow will contain 1000 Message IDs. If message grouping is used, only 1 Group ID is sent.
  3. Less memory is consumed by the File Transfer Service.

• To configure the File Transfer Services to use Message Grouping by default, you must update XMOFTS.xml and XMOFTS\_Client.xml. These two files can be found in the registry under Services/Config. A new tag needs to be added to the configuration: true. See the following XMOFTS.xml configuration file. By default the grouping tag does not exist in the configuration file.

  true  $\{queueManager\} CQ.FTS CQ.FTS.DATA CQ.FTS.ICC CQ.FTS.ICC CQ.FTS 800 1 1 SYSTEM.DEFAULT.MODEL.QUEUE CQ.FTS.IMMED.\* 60 300 1440 IFS IFS XML true 25 true  80 FB PS 0 10 15 TRK SYSDA 0   92 1 1
  

• The grouping option can also be controlled through the FTF command line. This is provided for compatibility with releases prior to 8.1.4 by adding -grouping true to an FTF command line request. The following is an example of a request with grouping on:

  C:\\> ftf -sqm SUPGEMBAR -dqm SUPGEMABR -spath c:\\spath\\test.file -dpath c:\\temp\\test.file -type text -grouping true FTF Request Submitted (d9b7ee45-d51a-485a-8f1b-a8ead7fcc125)
  

• The grouping option can also be set through the Process Monitor.

• Note that immediate transfers are not affected by grouping. If your transfer is in an immediate transfer, no list of Message IDs is created, and grouping will not have an affect on the performance of the file transfer.

• Some transfers may automatically be changed into immediate transfers due to the Immediate Threshold feature, which converts normal transfers into immediate transfers based on file size.