Using a post receive exit script to copy or rename a file after the initial transfer

We're using MIM to transfer a file to a Linux file system- the receiving/consuming application is grabbing and consuming the file before MIM has finished writing the file, so they are receiving partial files. It was recommended to us that we set the dpath to a different directory or filename and use a post receive process exit to move/rename the file.

Has anyone used this solution, and if so, how did you get it to work? Has anyone encountered this problem and used an alternate solution? If so, what did you use?

Find more posts tagged with

BPM

Comments

Mike_Policarpio

You can use xmosys.xms as the post receive exit to call the unix mv command to rename the file specified as the destination file name (dpath) to the true desired file name. I have included a sample request on how to do this. In the sample, the dpath specified is /tmp/destfile.tmp. In the exitdata tag of the post receive exit, the unix mv command is called to rename /tmp/destfile.tmp to /tmp/destfile.out. The post receive exit will run only after the File Transfer Service is done writing the file /tmp/destfile.tmp.

      TEST_RENAME
      SUPVALHALLA
      SUPVALHALLA
      SUPVALHALLA
      SUPVALHALLA
      /tmp/sourcefile.txt
      /tmp/destfile.tmp
      5

         1



            8
            xmosys
            xmoSysCall
            SUPVALHALLA
            mv /tmp/destfile.tmp  /tmp/destfile.out


      admin

This is a solution that will work out of the box. If you wish, you can write a custom exit which may better suit your needs. The xmosys.xms should provide a good starting point/template on how to accomplish this.

Paul Roth

You likely know this already but just to cover off the options...

There are generally two approaches when sending data to a target application.

In the first scenario the target application is running and waiting for the file to arrive (or show up) and in the second the target application is typically not running and is triggered or directly initiated.

Frequently customers implementing MIM don't have the luxury to choose which option and are forced into the approach you are currently using because the focus is on a) replacing an existing file transfer based solution and b) minimizing change to the existing environment.

In scenario 1 and the example being discussed the target application is already watching (polling) for the arrival of a file. If the operating system does not provide the facilities to 'lock' the file then the most popular approach is to send the file into a temporary directory or use a temporary name that the target application is not watching for. Once the file transfer has completed successfully we can utilize a receiver post process to either move and or rename the file making it available to the target application.

When considering option #2 there may be advantages and disadvantages. In option #2 one of the advantages is that the target application is not running and not polling and therefore not consuming system resources. When the file arrives a post receive exit could trigger or initiate the target application. While generally this is a good approach in situations where there are many file transfers and the resource costs associated with frequently starting the target application are significant one would be better served to have the target application configured as a long running process that is either waiting for the arrival of files. Another alternative may be to 'signal' the long running application notifying it that a file is available for processing.

In most cases I would say that our MIM customers are using Option #1 where an existing application is waiting for the arrival of files. In these cases the transfer is configured to send to an alternative directory or use an alternative file name and then using the post receive exit point either move the file or rename it making it visible and available to the target application.

MARK HUBAND

We are indeed using option 1 where the consuming application is watching for the arrival of the file and we don't have the ability to lock the file while it is being written to.

Using the above solution, we've encountered a problem. For unit testing I'm trying the rename method in a Windows file system using the ren or rename command. Here's my request:

<?xml version="1.0" encoding="UTF-8"?>

MVH TESTING FOR POST-RECEIVER EXIT

EIDTHDQ01WT_QM

EIDTHDQ01WT_QM EIDTHDQ01WT_QM

\MVH\postreceiver.txt

\\cycihdq01wt\d$\MVH\TEMP\POSTRECEIVER.TXT

FALSE

TEXT

REPLACE

xmosys

xmoSysCall

EIDTHDQ01WT_QM

rename \\cycihdq01wt\d$\MVH\TEMP\POSTRECEIVER.TXT \\cycihdq01wt\d$\MVH\TEMP\POSTRECEIVERFILE.TXT

mqsid

Problem is we're receiving the following error from the exit service:

2010-05-28 12:58:57 1005** **:58:57 XMOE000IMQS00000 Received 'Workflow' on queue 'CQ.EXIT' and qmgr 'EIDTHDQ01WT_QM'
2010-05-28 12:58:57 LOG: Fri May 28 12:58:57 2010 ...command is: rename \\cycihdq01wt\d$\MVH\TEMP\POSTRECEIVER.TXT \\cycihdq01wt\d$\MVH\TEMP\POSTRECEIVERFILE.TXT
2010-05-28 12:58:57 LOG: Fri May 28 12:58:57 2010 Exception XMESYSTEMCALL:
2010-05-28 12:58:57 <?xml version='1.0'?>
2010-05-28 12:58:57
2010-05-28 12:58:57
2010-05-28 12:58:57    1
2010-05-28 12:58:57    0
2010-05-28 12:58:57    24
2010-05-28 12:58:57    2010
2010-05-28 12:58:57    XMUtilities is not a registered COM component. rc = -2147221005
2010-05-28 12:58:57    XMO Class
2010-05-28 12:58:57
2010-05-28 12:58:57

XMUtilities is found in the XMCOM.xml member in our registry.

---
Note: Some information in this post has been hidden to protect privacy.

Ethan Beisher

If possible, I recommend upgrading to 8.5.1. There are several new enhancements and bug fixes that are currently available in this release.

However, you should not see this error. I recommend you open a trouble ticket (see Support button at top of this page).

Ethan

Mike_Policarpio

The error you are encountering occurs when the XMUtilities tag is missing from XMCOM.xml. The entry is shown below:

xmoplat

The problem can also occur in mutiple registry environments where an older copy of XMCOM.xml which does not contain the XMUtilities tag is found. Check either your server.xml and/or client.xml files to see the search order for multiple registries. The registry location listed first under the RegistryList tag will be searched first. Verify the copy of XMCOM.xml that is found is valid and up to date.

MARK HUBAND

I made sure there is only one copy of XMCOM.xml running in the current registry, and inside XMCOM.xml, I verified the following is present.

xmoplat

On the last run I received the same error:

   1
   0
   24
   2010
   XMUtilities is not a registered COM component. rc = -2147221005
   XMO Class

We're really not in a position to upgrade to 8.5 for this immediate need. Should I open a trouble ticket?

Mike_Policarpio

I believe there is something in your environment that may be causing this issue. I suggest contacting customer support for further assistance.

MARK HUBAND

I think I have it working now. First I restarted MIM services on the queue manager, just to be sure. The test process then actually got into the post receive exit and gave a syntax error on the line. So I played with the actual DOS command and ended up with the following that worked (specify the full path only once):

rename \\cycihdq01wt\d$\MVH\TEMP\POSTRECEIVER.TXT POSTRECEIVERFILE.TXT

Thanks for everyone's help on this.

MARK HUBAND

Problem with the mv command. I finally got around to testing in the actual Linux file system and it appears that the Linux command set is not recognized in the post receive exit. The command being executed is:

mv TEMP_HOSTDN00000220100616025426.DAT /appl/softgrocer/data/aplhst/HOSTDN00000220100616025426.DAT

The error thrown by the FTS Exit service is 'mv' is not recognized as an internal or external command .

As a reminder, the coordinator for this process is running in a Windows queue manager and this file is being sent to a Linux client. The file is successfully written to the Linux file system, but the post-receive fails when executing the Linux move command.

I attempted the echo command in the post-receive exit which was recognized, but other Linux commands 'cp' and 'ls' were not recognized. And remember, this worked in the Windows environment. So it likes Windows command sets but not Linux. Is there some kind of pre-qualifier or notation required to invoke the Linux command set?

Mike_Policarpio

Try specifying the full path to the mv command. To find out where mv is located, login onto your linux box and type the command "type mv". Different flavors of Linux may have mv in a different location. An example is below:

supsuse10:~ # type mv
mv is /bin/mv

In your exitdata, specify /bin/mv instead of mv. The same will be true of the cp and the ls commands.

MARK HUBAND

We did find the mv command in the /bin/mv location. We changed the exit data to:

Which was interpreted as:

/bin/mv /appl/softgrocer/data/aplhst/TEMP_HOSTDN20100623000000000000TEST.DAT HOSTDN20100623000000000000TEST.DAT

The errors received were basically the same:

command is: /bin/mv /appl/softgrocer/data/aplhst/TEMP_HOSTDN20100623000000000000TEST.DAT HOSTDN20100623000000000000TEST.DAT
/bin/mv' is not recognized as an internal or external command operable program or batch file.

and

Post_Rec_Exit_Point
xmoSysCall():2
The system call failed

/bin/mv /appl/softgrocer/data/aplhst/TEMP_$XMDM_ORIGINAL_NAME$ $XMDM_ORIGINAL_NAME$

Mike_Policarpio

Are you sure the exit service being called is on the Linux node? The reason I ask is that the line

"/bin/mv' is not recognized as an internal or external command operable program or batch file"

occurs only on a Windows node. You can recreate the same exact error text by opening a windows command prompt

and entering a command that does not exist.

I was also able to produce the same error text in my Windows Exit Service log by specifying exitdata as the /bin/mv command but specfying the Exit QM as a Windows node instead of a Linux node.

Ethan Beisher

Add the mv command to a shell script.

mv %1 %2

and then call the shell script instead of the mv command. Does that work for you?