Transient Astronomy - Customizing the Pre-Patcher

For background, you should at least skim-read the paper Transient Astronomy with ACP Scheduler (PDF) and focus on the XML Pre-Patching section.

The role of the pre-patcher is to fix up any illegal XML that might be present in incoming VOEvent messages. If left in the message, illegal XML will cause validation to fail and will result in the message being discarded by the receiver.

Types of Programs

The pre-patcher can be any type of Windows command line program, written in any language. This includes Windows Script programs which run under the Windows command line script host cscript.exe, plus Perl and Python scripts which have their own execution engines. If the pre-patcher is an executable program, it can be written in any language which provides a command line interface and which supports reading from "standard in" and writing to "standard out" and "standard error".

Data Flow

The raw message XML is piped into the script or program’s standard input and the patched XML is expected to be written by the pre-patcher to its standard output. The receiver simply waits for the (external) pre-patcher program to exit before moving to the next phase of processing. The pre-patched XML is then fed to the receiver’s VOEvent parser/validator module, which contains its own pre-patching logic (see Transient Astronomy with ACP Scheduler).

Programming Considerations

Normally, when dealing with XML, one would feed the XML into an XML engine which parses the data and constructs a Document Object Model which can be manipulated. This is vastly preferable to picking through the raw XML text. However, in the pre-patcher it should be assumed that the incoming VOEvent XML is not valid and therefore cannot be parsed by an XML engine. Therefore, you are stuck with fixing up the raw XML data.

By far the most powerful tool for fixing up raw XML data is Regular Expressions. If you aren't familiar with regular expressions, you're at a serious disadvantage when writing your own pre-patcher. Virtually all programming languages and scripts include regular expression tools. It is beyond the scope of this document to describe regular expression programming.

Error Handling

If the pre-patcher program encounters an error and must exit unsuccessfully it is vital that it do both of the following:

1. Write all error messages to its standard error, and
2. Exit with error status. All programming languages provide a way for the program to specify an integer status value upon exiting. Typically, an error status is anything other than zero, which is the universal "success" status value.

You have control over your program's exit status value, and as you'll see below, the receiver will report the exit status value in its log and in a balloon popup if your program exits with any error status. Therefore, you can use the exit status value to help you isolate between sources of error within your program.

The Windows Script Host (cscript.exe) automatically does these two things. If your script fails to compile or if there is a fatal run-time error, cscript.exe puts all error text on its standard error and exits with non-zero status. The exit status value isn't too useful, but the error text is usually very useful.

Program Invocation

The receiver's configuration settings provide for the path to the executable and command line arguments to be passed to the program (see diagram on the right). The receiver will invoke the pre-patcher program as follows:

1. Look in the Scheduler's ReceiverData folder for the executable. The configured Executable path is appended to the path to the ReceiverData folder to form the entire path. Thus, a configured path of bin\foo.exe will result in the receiver looking in ReceiverData\bin\foo.exe. Normally, you would simply put your pre-patcher executable into the Receiver data and put the executable's file name into the configuration Executable path.
2. If it doesn't find the program in step 1, it then looks in your Windows\System32 for the executable. This is provided only for script pre-patchers which use the Windows Script host cscript.exe (which is located in Windows\system32). Do not put your pre-patcher executable in Windows\system32!
3. Construct a command line for the program. The configured Command line arguments form the command line. In addition, if the executable is cscript, a command line option //NoLogo is added. This prevents the Windows Script Host from writing its banner/logo to standard out (which would be seen as part of the patched XML by the receiver!).
4. Set the working directory for the pre-patcher to the ReceiverData folder.
5. The program is started as a separate process with the constructed command line, initially in the suspended state.
6. The received/raw XML is connected to the patcher program's standard input.
7. The patcher program's standard output and standard error are connected back to the receiver.
8. The receiver un-suspends (resumes) the patcher program, allowing it to run to completion.
9. The receiver waits for the patcher program to exit.
10. Upon exit, the patcher program's "exit status" is tested for success. If the patcher program exited with error status, the receiver logs the error and pops a balloon alert "**Pre-Patcher task exited with error status nnn: xxxxxxx", where xxxxx is any text that comes in from the receiver's standard error. In addition, any patcher output XML is discarded. The receiver acts as if nothing had ever been received.