The problem of waiting time when launching a unix shell command from an application engine

The problem of waiting time when launching a unix shell command from an application engine

We made a unix shell script that would wait for a certain unknown amount of files to be downloaded, which takes an unpredictable time, and then immediately after the last file is downloaded, copies all files to another directory. The unix shell script is launched from a PeopleSoft Application Engine, and the amount of expected files is passed as a parameter.

One of our clients is using an application of Semantica for viewing pay slips and other documents in PDF, HTML and TIFF format. These documents are present in another database, that is written in java, with no access, and thus is a black box for us. From Peoplesoft you can hit a button online to view for example your pay slip of this month, and then your pay slip pops up in a new window. Behind the screens, when you hit the button, in peoplecode the following command is launched:

Imag is a PL/SQL package within the peoplesoft database, thats calls a routine in the Semantica database. The actual document, for instance a pay slip, is downloaded on the unix server, that is also the PeopleSoft application server. This document is published and with the URL you can view it. After a view minutes however, the document is automatically deleted from the unix server.

Our client asked us to get all documents of a certain population out of the Semantica database, and put them on a CD. Our first idea was to ask the firm Semantica to do that, since we have no access to the Semantica database. But because our budget was very limited, we decided to give it a go ourselves. Therefor we develloped an application engine that calls the imag package with an SQLEXEC as above, so that a certain document is downloaded on the unix server, and then launches a unix shell command from the application engine with the command EXEC(.., true); which copies the document to another directory. Then loop through the whole population and thus all documents can be put in a certain directory, can be zipped and put on a CD.

The difficulty was that a lot of the documents in TIFF format where actually multiple TIFFS. These are TIFFS with multiple pages. When calling the imag package, all pages of a TIFF are downloaded on the unix server one by one, which takes an unknown time. Some have for instance 80 pages, and it could take for instance 10 minutes to download them all. But this is not certain; the next downlaod of 80 pages can suddenly take 5 minutes or 2. Then after some time the documents are automatically deleted again. So the timing of the moment to copy all of the files that are related to one TIFF, needs to be very precise, and is at the same time not predictable.

So the unix shell script that copies the file to the other directory must work a bit like a deamon, which is waiting for the last document to be downloaded and then, when this task is completed, immidiately copies the whole bunch to another directory. Luckily we have information on how many pages each multiple TIFF has. This is a number in one of the related tables in the PeopleSoft database, and thus can be passed as a parameter to the unix shell script. In the unix shell script we used a while statement, to compare the expected amount of pages with the number of counted downloads in the download directory. When these two numbers would match, all files are copied. The waiting command in unix turned out to be quite simple:

The scripts waits as long as the while loop is running. After the command “done” the actual copying can begin. At first I was afraid that all this looping would ask a lot of resources from the unix server. It can very well be that the program keeps on looping for 10 minutes. But we let the application engine run for hours, and the unix server had no problems with it.