Track 3 - Manual Upload Servlet w/ Polling Subscriber Connection and XSLT

This Demo Track illustrates 4 new Features.

In some cases the normal automated Publishing processes ( event-driven or scheduled client programs ) are not appropriate. They may be replaced by a totally Manual process for publishing. This allows you to call a business partner and to say "Look we have some hot data. Im gonna send you this secure data on Topic xyz right now". This is nothing more than the standard OReilly upload Servlet with its standard file chooser dialog for controlling uploads. The only prerequisite for this upload is that you first bind the browser session to a Topic as explained below. Only the Demos need this step.

The second feature is a Polling client that issues regular Web Service Requests to the Post Office on a Topic asking to be sent any updates on the topic received by the Post Office since the last request by this Client Subscriber on this specific Topic. Any Delta files with new data are returned in the Web Service Response as SOAP Messages with Attachments . The third feature is XSLT Processing that converts the raw XML used on the sender side of the connection to HTML according to a Translation Template meeting the markup needs of this individual subscriber. The last new thing in the demo is Logging Features of this framework. In the absence of actual DataBase transactions that cover movement of distributed data, logging is very important in maintaining the reliability of this framework.

Due to the constraints of the Demo and to restrictions on the scope of the software install, the topics and the subscriptions that you create with the demo Forms will not persist in a DataBase. Rather, each form submission binds the entities ( Topic, Subscription ) to your Session. For as long as your session lasts ( about 20 minutes ) you may run command line demos that trigger data interface activity on your new Topic and Subscription.

Create New Topic - bind the Topic to Browser Session

The First form binds a Topic to your session. One text field needs to be completed on the left of this form. Enter a name for your topic. When you have filled it in use the submit topic button. The URL for the real form is here. The webapp requires authentication, so on a challenge, use the following for credentials:

  • user=dogtown
  • password=zboys
Successful interaction with the topic creation form will result in this browser view.

Create New Subscription to the Topic

Note that successful creation of any subscription is mirrored by a specific ChannelData Command (CDC) and that it's the CDC that actually gets connections and moves data. Whenever a Publish Event occurs on your new topic, the Post Office dispatchs the Channel Data Command as an Event Handler to transfer data.

As in the Track 2 Demo, submission of the second form binds a Subscription to the session only this time you complete 5 fields on the right hand side of this form. Note that this demo and it's use of Polling as a Client Pull as opposed to a Push by the Post Office Server, mean that only the Polling field is really important. However, the demo requires that they all be filled in. Set Server IP Value to any valid IP Address where there exists an FTP daemon for which you posess valid Credentials. As an example, you may use the IP="" for server, and the values "anonymous" and "" for credentials. Anything can be supplied for File Name because this property is not used. Select the polling in the Protocol field.

A Calendar and Time chooser dialog operate along with the Begin Time Select Log Records in the lower left and also operate with the End Time Select Log Records in the lower right. For the purposes of the demo it is important that you select START values for a dateTime that is older than CURRENT_DateTime. Similarly, the END values that you select should define a date that is in the Future. Successful submission of the form creates a CDC and returns it to the browser in this view.

Get the Sample XML File - Download it

Download this file remembering to save it to the data/outbnd directory under the root where you did the Demo Install. Your save as dialog should preserve the name of the downloaded file as this will appear in a command line argument later on when you start up the Web Services Polling Client. This is the file that is uploaded later in the demo.

Run the Polling Console and then issue a Start Command at the Console

This is a Client program that is to be run by Subscribers wanting control over the timeing and processing which PULLS data to them rather than waiting for the normal Post Office Distribution threads to PUSH the data. Since this client app. is a Console, Running things is a 2-step process. First you have to Start the Console and it will come up with a prompt at which you must then enter START and STOP expressions. Get a command shell on your system and enter the command line below, adjust it to the path requirements of your system, remembering Windows conventions require all the information on a single line and with a "\" as the path separator. Refer to help with Shell commands.


On launch, demo debug settings dump the entire properties list before the prompt is listed. At the prompt line with the ">" below: type start and then press enter

-- listing properties --
Console deamon has started
Console deamon has started   - use < start > and  < stop > commands to proceed
> start
Console deamon has started
> start

> Clientcntrllr running callbacks on --TopicClient--
> TopicClient time is Wed Jan 07 16:52:52 PST 2004

The client starts cycling itself depending on the value you entered for the property util.timer.poll.period. The values listed above will issue a Web Service Call every 15 seconds that routes to the Post Office, requesting any updates on the test2 TopicName. Asynchronously, any authorized publisher of the test2 topic may be publishing to the Post Office. Then, for this specific subscriber, no more than 15 seconds will elapse prior to the Poller issuing the next Call request for updates to the Post Office. Any delta information will be returned in the Request Object.

Upload XML Data to Post Office Using File Upload Servlet

Go to the Upload Servlet using the file chooser to open the file prior to submit. Remember that in the demo, you must have completed the Bind Topic step mentioned above. The Submit Query button sends your XML data up to the Post Office where it gets Queued to the Distribution Thread responsible for the Topic on which you uploaded the data. In this specific case, because there are no active Subscriber Threads handling your topicname, the Post Office dispatches NO WORK to PUSH DATA to any Subscriber. However, other client initiated activity will quickly result in your recently uploaded data's distribution.

A Word about ASYNC State on the Post Office

Arrival of data on the Post Office, the result of the manual upload step, does not result in any Synchronous distribution process. The Post Office does not work this way. The Upload Unit of Work terminates with the upload files write copy on the File System of the Post Office. A TimeStamp attribute is associated with that write. Another important TimeStamp is the lastfile value in the Channel Data Command. A CDC exists for each subscription and, at the moment of the upload, the CDC for the Polling client reflects the fact that the file just uploaded has not been received yet as a result of a Web Service call requesting the new files on the topic. Specifically, lastfiles value in the CDC is less than the uploaded file's TimeStamp.

Polling Requests to the PostOffice and updated CDC state

As mentioned at the bottom of the Polling Topic above, no more than 15 seconds will elapse between the reciept by the Post Office of the uploaded file on this topic and the new Polling request on the topic. Each file that has not been received by the polling client is added as an Attachment Part to the SOAP Response Message and sent back to the client.

Activities on the local file system of the polling client are guided by some of the command line args used in the run command. Once received back on the client each Attachment Part from the SOAP Response is going to be written to the file system after constructing the proper name for the file.

From the command line, the 2 args above control serialized names for all the files written on the client. testXsl4.html will be the first file. Serialization will use the number 5 in the next file name and so on... If more than one uploaded file were to have been received by the Post Office in the interval since the Client's last polling request , then all such files would become AttachmentParts within the Response marshalled by the PostOffice. The CDC's lastfile value, updated on the server to reflect the state of the last set of transmitted data, equals the TimeStamp of the youngest file in the set sent to the client. Next time a polling request arrives on the topic, it will be this TimeStamp value that is compared to those belonging to the PostOffice's collection of files on this topic. If nothing new arrived, then the next Request will result in a Response with no attachments.

XSLT Processing - convert XML to alternate Markup preferred by Subscriber

In this case the file published to the Post Office via Manual Upload contained XML. However, the Subscriber preferred HTML Format. These 2 links comprise the input and output sides of a Transformation process occuring on the client after the Post Office returned the SOAP message Response. Details of the conversion are guided by the collections of expressions in the XSL template. The command line argument used to convey this information is actually the template.xslproperty whose full command line value in this demo is util.template.xsl=config/demo_poll.xsl. Go here for details on command line for starting the console.

B2BPO Message Logging and Reporting Features

Distributed messaging, a core feature responsible for reliable distribution of data, goes hand in hand with Logging Features of this framework. Since there are no DataBase transactions here, logging is very important to assure the proper completion of distributed work. Messages like SOAP messages are often the containers of data and files that are transiting amoung B2B partnering companies. In order to assure proper diagnostics and sufficient troubleshooting capabilities, extensive and easily tweaked logging is a component of every Java Class in the framework. In the section above, the HTML link gives an idea of some of the Message Archive Reporting Features available from the Post Office based Logging Portal.

The portal combines organized archives of message activity with archive viewing tools like Chainsaw. Archive Drilldowns provide detailed history's at the session level or on the Messaging Topic level of information A standard manifest at the message level allows drilldown on the content of individual messages, in this case SOAP Envelopes and their contents.

Due to the number of Protocols that are implemented, absolute consistency of logging across all implementations is not possible. Some Protocols like GridFTP offer less frequent logging due to the fact that 3rd party open source is being used and that none of the included classes were altered with respect to logging. Regardless of the protocol, when a CDC issues a command to move a file on the connection, there is always a Return Code and a Success/Failure notice. For example, the subscriber polling from this demo and the event driven Publisher from Track 2 demo use Web Services , SOAP , and due to the present Post Office architecture that relies on JAX-RPC calls to layer the messaging middleware onto a Services oriented API where there is no native SOAP code, the SOAP MEP is synchronous. You are guaranteed a Response or a Failure code. There is no wait to match up an ASYNC response. Details on the portion of the SOAP Spec invoked in collaboration with JAX-RPC are here.The list below gives a general idea of when logs are taken during processing:

  • at any failure notice from an underlying protocol doing data transport like SOAP, FTP, SSH
  • at data source when files are selected for publish on a topic
  • at the Post Office when files are first received on a topic
  • at the Post Office when files are delivered to a subscriber
  • depending on the Protocol, there may be logging at the subscriber on file delivery
Note: in the event that Publishers or Subscribers are using GridFTP then the logs taken at the get and put level are a function of the Globus Software package and the config settings used to launch the FTP daemon. The question of how to resolve these 3rd party logs with the reporting requirements of this framework are TBD.