Index Changes
This is version 41. It is not the current version, and thus it cannot be edited.
[Back to current version]   [Restore this version]

FTP BC Recovery

Wiki Main

FTP BC realizes recovery using "inbound message processing journaling" - a local file system or NFS based journal which records (persists) the inbound messaging context for each every inbound message consuming, it ensures that an inbound message consuming can survive the followings:

  • Uncorrectable Failure During Inbound Message Consuming (connection interruption, IO error, etc.)
  • Runtime Termination (Crash of Application, Components - BC or SE etc., Container, or Application Server)

And, when the application starts next time, the concerned message consuming can be resumed and the corresponding message routing completes based on the context information persisted in the recovery journal.

For details, please refer to FTP BC Transactional Messaging & Clustering Support & Message Sequence Numbering .

Recovery After Uncorrectable Error

Wiki Main

The followings demonstrate how an uncorrectable error interrupts inbound message consuming and gets recovered when the application gets fixed and restarted again.

Application Diagram:

This application uses FTPBC to poll the remote target directory <message_exchange_base_dir>/inbox for incoming message (files that match specified name pattern), when found, move to inbound staging area <message_exchange_base_dir>/inselect, then start reading the content of the file, when complete, parse the content and wrap it up in a JBI message (normalization), when done, send it into NMR, NMR routes the message to the downstream component - in this case - a BPEL SE executing some bpel script (a pass thru), the bpel script invokes local file write service which uses FILE BC outbound to: denormalize the message and write the payload to a local file - output%d.txt.

For details on how to create SOA BPEL module and adding new WSDL documents and BPEL processes, as well as adding BPEL modules to a SOA composite application, please refer to : Create Composite Application Using FTPBC

The following screen shots refelects the steps showing recovery in action:

Step 1: Create SOA BPEL Module, Add FTP Inbound One Way Messaging Service, Add FILE Outbound Service, Add BPEL process: GETFROM_FTP_2_LOCAL.bpel

  • Create a SOA BPEL module: DOAN_LOAD_FTP
  • Define a local file service to write a message to a local file, save the definition to FILE_WRITE.wsdl
  • Intentionally specify a local path with a non existing drive letter, so that the operation will fail - create a uncorrectable error

  • Defined a ftp service to poll a remote message, save the definition to FTP_GET.wsdl

  • Create a bpel process doing a copy through, save the bpel script to GETFROM_FTP_2_LOCAL.bpel
  • Configure the BP as atomic so that the file outbound error will bounce back to the ftp inbound

Step 2: Create SOA Composite Application

  • Create a composite application service assembly: DOWN_LOAD_FTP_COMP_APP, and adding the BPEL module: DOWN_LOAD_FTP.jar

  • Build the composite application and edit it in CASA editer

Step 3: Build and Deploy the composite application

  • Build and deploy

  • Exam the deployed project through JBI manager

Step 4: Start Messaging by feeding a inbound message

  • Manually feed a file at the ftp directory where the inbound processor is polling

  • Exam the local directory to see if the file is written, since the path d:\temp is invalid, the attempt to write to d:\temp\output0.txt will fail

  • The recovery log is stored at the persistence store, located at:

  • Location of the message which the message routing is not completed

  • Content of a recovery log entry : it records the context info for the message that failed delivery, which will be used by the recovery logic when next time the application starts

Step 5: Undeploy and Fix the WSDL

Step 6: Build and Deploy Again

Step 7: Exam recovery log and local directory

  • Recovery log entry is removed - processed
  • There is the output message at the local directory c:\temp\output0.txt

Recovery After System Crash

Wiki Main

In following shows how recovery works when an inbound consuming was interrupted by application server crash.

The same composite application used in previous section "Recovery After Uncorrectable Error" is used for the demo.

Step 1: Build and deploy the composite application

Step 2: Enabled debug for bpel se

Enabled debugging in BPEL SE

Step 3: Set break point in the bpel script

Set break point in netbeans IDE

Set break point in the bpel script source code

Step 4: Attach to the bpel se

Set the composite application as main project, get ready to attach

Set debug info on on the debugger side - netbeans IDE

Step 5: Feed inbound message

Feed inbound message, triggering the message consuming

Suspended after message polled but not ACKed or NACKed

Step 6: Kill the jvm for the application server to simulate a system crash

Locate and Kill the app server to simulate a crash

There is no output on the local destination

A recovery log entry persisted in the app server jbi persistence storage

Go to netbeans "Services" tab and refresh "Servers" node, the app server glassfish is not running

Step 7: Restart the application server which will also start the deployed composite application

When a service unit is started, the recovery will happen before the inbound processor goes into periodical polling:

Restarted the app server glassfish, and observing the server log

Server log shows that the particular recovery log entry has been processed successfully

Step 8: Exam recovery log and local destination directory

Check the local destination, the output has been written

Check the recovery log, it is cleaned up


Malformed Message Handling

Wiki Main

Malformed messages are caught during normalization, when an external message is obtained from external system, it is normalized (parsed using a XML schema or custom encoder) and wrapped in a JBI Message object, then the normalized message is set in a message exchange and sent into NMR (Normalized Message Router).

If a message is malformed, i.e., does not comply with its schema, it is moved out of the way (in inbound staging area), and marked with a suffix ".err" indicating such, a error log is persisted in a dedicated location at the application server which records the details of the error and location of the offending message.

This process applies to all the scenarios that a message is retrieved from remote and going through JBI Message normalization before being pumped into NMR, in another words, this process applies to:
  • Polling Side of: "Poll Request Message", "Put Request Message and Poll Response", "Poll Request Message and Put Response"
  • Receiving Side of: "Receive Request", "Receive Request and Send Response", "Send Request and Receive Response"
  • On Demand Get Message, On Demand Receive Transfer

For samples projects using complex XML message and custom encoded message please refer to:

Create Composite Application Using FTPBC - HelloDukeUsingComplexType

Create Composite Application Using FTPBC - HelloDukeUsingCustomEncoder

The following demo uses a composite application that poll a remote directory for inbound messages (e.g. Hello) and process the message and generate a response accordingly (e.g. Hello) and put the response to outbound directory.

The messages are delimited as is defined by Greeting.xsd with annotation specifying multi-level delimiters info:

A request message looks as follows:


A response message looks like follows:


Step 1: Create BPEL Module "POLL_AND_PUT_ENCODER" and Composite Application "POLL_AND_PUT_ENCODER_COMP_APP"

Step 1.1: Create "POLL_AND_PUT_ENCODER"

Create the FTPBC WSDL POLL_AND_PUT_ENCODER.wsdl as follows:

Set connection info

Set poll request message binding info : select "encoded data" as message type

Set poll request message binding info : select "root" element type for message part

Set poll request message binding info : specify the name of the encoder - 'customencoder-1.0' used by the runtime to lookup the corresponding encoder library

Set put response message binding info - through the same process as with request message binding info

The resulted WSDL for the definition of a service that poll a request (Hello) and process it and generate a personalized response bound to FTP transport

The resulted composite application rendered in CASA editor

BPEL script does the mapping between request message and the response - generating a personalized response

Step 2: Build and Deploy

Step 3:

Step 4:

Step 5:

Step 6:

Step 7:

Step 8:

JSPWiki v2.4.100
« Home Index Changes Prefs
This particular version was published on 27-Feb-09 17:27 PM, -0800 by jimfu