FTP BC Recovery
FTP BC realizes recovery using "inbound message processing journaling" - a local file system or NFS based journal which records (persists) the inbound messaging context for each every inbound message consuming, it ensures that an inbound message consuming can survive the followings:
- Uncorrectable Failure During Inbound Message Consuming (connection interruption, IO error, etc.)
- Runtime Termination (Crash of Application, Components - BC or SE etc., Container, or Application Server)
And, when the application starts next time, the concerned message consuming can be resumed and the corresponding message routing completes based on the context information persisted in the recovery journal.
For details, please refer to FTP BC Transactional Messaging & Clustering Support & Message Sequence Numbering .
Recovery After Uncorrectable Error
The followings demonstrate how an uncorrectable error interrupts inbound message consuming and gets recovered when the application gets fixed and restarted again.
This application uses FTPBC to poll the remote target directory <message_exchange_base_dir>/inbox for incoming message (files that match specified name pattern), when found, move to inbound staging area <message_exchange_base_dir>/inselect, then start reading the content of the file, when complete, parse the content and wrap it up in a JBI message (normalization), when done, send it into NMR, NMR routes the message to the downstream component - in this case - a BPEL SE executing some bpel script (a pass thru), the bpel script invokes local file write service which uses FILE BC outbound to: denormalize the message and write the payload to a local file - output%d.txt.
For details on how to create SOA BPEL module and adding new WSDL documents and BPEL processes, as well as adding BPEL modules to a SOA composite application, please refer to : Create Composite Application Using FTPBC
The following screen shots refelects the steps showing recovery in action:
Step 1: Create SOA BPEL Module, Add FTP Inbound One Way Messaging Service, Add FILE Outbound Service, Add BPEL process: GETFROM_FTP_2_LOCAL.bpel
- Create a SOA BPEL module: DOAN_LOAD_FTP
- Define a local file service to write a message to a local file, save the definition to FILE_WRITE.wsdl
- Intentionally specify a local path with a non existing drive letter, so that the operation will fail - create a uncorrectable error
- Defined a ftp service to poll a remote message, save the definition to FTP_GET.wsdl
- Create a bpel process doing a copy through, save the bpel script to GETFROM_FTP_2_LOCAL.bpel
- Configure the BP as atomic so that the file outbound error will bounce back to the ftp inbound
Step 2: Create SOA Composite Application
- Create a composite application service assembly: DOWN_LOAD_FTP_COMP_APP, and adding the BPEL module: DOWN_LOAD_FTP.jar
- Build the composite application and edit it in CASA editer
Step 3: Build and Deploy the composite application
- Exam the deployed project through JBI manager
Step 4: Start Messaging by feeding a inbound message
- Manually feed a file at the ftp directory where the inbound processor is polling
- Exam the local directory to see if the file is written, since the path d:\temp is invalid, the attempt to write to d:\temp\output0.txt will fail
- The recovery log is stored at the persistence store, located at:
- Location of the message which the message routing is not completed
- Content of a recovery log entry : it records the context info for the message that failed delivery, which will be used by the recovery logic when next time the application starts
Step 5: Undeploy and Fix the WSDL
Step 6: Build and Deploy Again
Step 7: Exam recovery log and local directory
- Recovery log entry is removed - processed
- There is the output message at the local directory c:\temp\output0.txt
Recovery After System Crash
In following shows how recovery works when an inbound consuming was interrupted by application server crash.
The same composite application used in previous section "Recovery After Uncorrectable Error" is used for the demo.
Step 1: Build and deploy the composite application
Step 2: Enabled debug for bpel se
Enabled debugging in BPEL SE
Step 3: Set break point in the bpel script
Set break point in netbeans IDE
Set break point in the bpel script source code
Step 4: Attach to the bpel se
Set the composite application as main project, get ready to attach
Set debug info on on the debugger side - netbeans IDE
Step 5: Feed inbound message
Feed inbound message, triggering the message consuming
Suspended after message polled but not ACKed or NACKed
Step 6: Kill the jvm for the application server to simulate a system crash
Locate and Kill the app server to simulate a crash
There is no output on the local destination
A recovery log entry persisted in the app server jbi persistence storage
Go to netbeans "Services" tab and refresh "Servers" node, the app server glassfish is not running
Step 7: Restart the application server which will also start the deployed composite application
When a service unit is started, the recovery will happen before the inbound processor goes into periodical polling:
Restarted the app server glassfish, and observing the server log
Server log shows that the particular recovery log entry has been processed successfully
Step 8: Exam recovery log and local destination directory
Check the local destination, the output has been written
Check the recovery log, it is cleaned up
Malformed Message Handling
Malformed messages are caught during normalization, when an external message is obtained from external system, it is normalized (parsed using a XML schema or custom encoder) and wrapped in a JBI Message object, then the normalized message is set in a message exchange and sent into NMR (Normalized Message Router).
If a message is malformed, i.e., does not comply with its schema, it is moved out of the way (in inbound staging area), and marked with a suffix ".err" indicating such, a error log is persisted in a dedicated location at the application server which records the details of the error and location of the offending message.
This process applies to all the scenarios that a message is retrieved from remote and going through JBI Message normalization before being pumped into NMR, in another words, this process applies to:
- Polling Side of: "Poll Request Message", "Put Request Message and Poll Response", "Poll Request Message and Put Response"
- Receiving Side of: "Receive Request", "Receive Request and Send Response", "Send Request and Receive Response"
- On Demand Get Message, On Demand Receive Transfer
For samples projects using complex XML message and custom encoded message please refer to:
Create Composite Application Using FTPBC - HelloDukeUsingComplexType
Create Composite Application Using FTPBC - HelloDukeUsingCustomEncoder
The following demo uses a composite application that poll a remote directory for inbound messages (e.g. Hello) and process the message and generate a response accordingly (e.g. Hello) and put the response to outbound directory.
The messages are delimited as is defined by Greeting.xsd with annotation specifying multi-level delimiters info:
A request message looks as follows:
A response message looks like follows:
Step 1: Create BPEL Module "POLL_AND_PUT_ENCODER" and Composite Application "POLL_AND_PUT_ENCODER_COMP_APP"
Step 1.1: Create "POLL_AND_PUT_ENCODER"
Create the FTPBC WSDL POLL_AND_PUT_ENCODER.wsdl as follows:
Set connection info
Set poll request message binding info : select "encoded data" as message type
Set poll request message binding info : select "root" element type for message part
Set poll request message binding info : specify the name of the encoder - 'customencoder-1.0' used by the runtime to lookup the corresponding encoder library
Set put response message binding info - through the same process as with request message binding info
The resulted WSDL for the definition of a service that poll a request (Hello) and process it and generate a personalized response bound to FTP transport
The resulted composite application rendered in CASA editor
BPEL script does the mapping between request message and the response - generating a personalized response
Step 2: Build and Deploy