Index Changes

Difference between version and version     

Back to Fuji EIP Split, or Fuji EIP Split Info


At line 1 added 1 line.
[{Image src='fuji-logo8.gif' link='https://fuji.dev.java.net/' alt='Fuji Home' align='left'}]
At line 7 added 2 lines.
The purpose of the split pattern is to take a single input message and output one or more output messages based on a boundary condition. The boundary condition can be specified in a wide variety of languages and is generally tied to the type of data being split (e.g. XML -> XPath). Some split expressions are more powerful than others, with the option to mangle or enrich the data as a side effect of being split.
\\
At line 12 added 1 line.
The split pattern is very common in the integration space, particularly in batch-oriented environments. Some of the more common use cases follow (feel free to add your own!).
At line 10 changed 1 line.
! Enveloped Data
Take a large input message and split it into more digestible pieces, also known as chunking. This allows for more efficient memory utilization, as chunked pieces can be reclaimed by the garbage collector individually. Chunking also allows for a divide and conquer approach, where parallel processing results in faster overall throughput for a given message size.
At line 16 added 5 lines.
A message needs to be unwrapped into individual pieces so that each piece can be processed by the appropriate message stream. Commonly paired with content-based routing.
! Enveloped Data
Refinement of dispatch centered around an envelope structure in the input message. A classic example of this use case is receipt of an ANSI X12 EDI message which contains multiple transaction types inside a single input message. Split can be used to break out each transaction type and forward on to the appropriate back-end processing system.
! Keep the Baby, Lose the Bath Water
Avoids situations where a large message is completed rejected due to one or two bad records.
At line 18 changed 1 line.
type := splitter implementation (e.g. xpath, regex)
type := splitter implementation (e.g. xpath, regex, xsl)
At line 32 added 10 lines.
! Examples
Split with inline configuration
\\
{{{
route do
from "batch-inbound"
split xpath ("/PurchaseOrder")
to "po-process"
end
}}}
At line 43 added 1 line.
__Note:__ ''I'm wondering if it would be more appropriate for the in-line configuration to be contained within 'do/end' blocks, similar to how broadcast is handled today. This may be a good general-purpose construct for scoping all IFL language features. Thoughts?''
At line 45 added 19 lines.
Split with external configuration
\\
{{{
route do
from "batch-inbound"
split xpath "po-split"
to "po-process"
}}}
{{{
> cat po-split.xml
/PurchaseOrder
}}}
! Misc.
* The IFL parser should not place any constraints on the type identifier in a split definition. This means that the value for type can only be verified at the time of service artifact generation.
* Known types to the parser/editor should provide contextual help in the form of syntax highlighting, code completion, etc. For the time being, this will be limited to xpath, regex, and xsl configuration.
* Inline configuration for unknown split types present a problem for the IFL parser. There is really no other choice than allowing for late-binding validation in the plugin for the particular service type (''see Configuration'').
At line 65 added 1 line.
Each type implementation will have its own specific configuration. There however is a configuration which can be defined for all types in general:
At line 67 added 17 lines.
|| Attribute || Description || Default
|''batchSize'' | Limits the of output messages that are processed in parallel. | Default is 1000. If the inbound message exchange is transactional (I.E. MessageExchange.isTransacted() returns true) the batchSize is forced to 1
The service artifact layer in Fuji will have a reasonable default behavior for all split types. Individual split types can override the default behavior by implementing the following contracts:
* Generation - the configuration generated from IFL for a given split type. A facility must exist for discovering template service artifacts for a given split type. This could be an SPI or a convention for resource location inside a bundle/jar file. For external configuration, a default implementation should be provided that results in an empty file with the split configuration name used as the file name. Generation is not relevant for inline configuration.
* Validation - during the packaging phase of the application build, split types will have an opportunity to validate configuration. This should be exposed as an SPI that a split type provider can implement.
\\
The directory layout for generated configuration should be:
{{{
app-root/
split/
{name}/
[configuration]
}}}
Where '{name}' is the named configuration provided in IFL.
At line 85 added 29 lines.
The XPath filter type provides a simple facility for breaking up a single XML input message into multiple XML output messages. This filter is useful for messages which have a recurring element that can be processed as is (i.e. without enrichment or other processing).
\\ \\
Input Message
{{{
<orders>
<order>
<item>bang001</item>
<desc>guns</desc>
<amount>50</amount>
</order>
<order>
<item>parkayXYZ</item>
<desc>butter</desc>
<amount>100</amount>
</order>
</orders>
}}}
XPath Configuration
{{{
/orders/order
}}}
Output Messages
{{{
<order>
<item>bang001</item>
<desc>guns</desc>
<amount>50</amount>
</order>
}}}
At line 115 added 57 lines.
{{{
<order>
<item>parkayXYZ</item>
<desc>butter</desc>
<amount>100</amount>
</order>
}}}
! XSL
Like the XPath filter, the XSL filter allows for data from an input message to be extracted and forwarded as one or more output messages. The additional capability that XSL provides is the ability to mangle the data as part of the split. A common mangling use case is the inclusion of correlation data from a parent element in a child element that is split into a separate message.
Input Message
{{{
<orders>
<orderNum>PO123</orderNum>
<order>
<item>bang001</item>
<desc>guns</desc>
<amount>50</amount>
</order>
<order>
<item>parkayXYZ</item>
<desc>butter</desc>
<amount>100</amount>
</order>
</orders>
}}}
XSL Configuration
{{{
}}}
Output Messages
{{{
<orders>
<orderNum>PO123</orderNum>
<order>
<item>bang001</item>
<desc>guns</desc>
<amount>50</amount>
</order>
</orders>
}}}
{{{
<orders>
<orderNum>PO123</orderNum>
<order>
<item>parkayXYZ</item>
<desc>butter</desc>
<amount>100</amount>
</order>
</orders>
}}}
Content enrichment is another common use case, where data is inserted into a split message that is not defined in the original message content model.
At line 173 added 1 line.
The regex (regular expression) split type is useful for non-XML data. The user provides a regular expression as the configuration for the split and each match of the expression marks a message boundary for the split.
At line 31 changed 1 line.
!! Runtime
Input Message
{{{
ORDER|PO123
ITEM|bang001|guns|50
ITEM|parkayXYZ|butter|200
ORDER|PO456
ITEM|oopsABC|huggies|300
}}}
Regex Configuration
{{{
"ORDER"
}}}
Output Messages
{{{
ORDER|PO123
ITEM|bang001|guns|50
ITEM|parkayXYZ|butter|200
}}}
At line 33 changed 1 line.
- split properties
{{{
ORDER|PO456
ITEM|oopsABC|huggies|300
}}}
At line 199 added 4 lines.
! Other
* X12 - ANSI X12 EDI data is encapsulated within interchange, group, and document envelopes. This split can be configured to extract messages at any of these levels. In addition, information from parent envelopes can be included in the split data if required.
* EDIFACT - Another EDI data format. Pretty much the same story (with different syntax) as X12.
* HL7
At line 204 added 14 lines.
!! Packaging
At application build time, all split configurations should be packaged as part of the application. Since the ultimate consumer of this configuration information is the runtime split implementation, and not a component, it is not appropriate to package the individual configuration instances into a service unit. Rather, the following convention should be followed:
{{{
app-root /
META-INF /
flow/
split/
[configuration]
}}}
Where '{name}' is the named configuration provided in IFL. This layout mimics the configuration layout, but includes META-INF/ and flow/ as parent directories.
\\ \\
All split configuration should be included in this directory, including inline configuration. Since inline configuration is not named, a name must be generated. Using a pattern of 'split$n' should work, where 'n > 0' and increments for each inline split definition in the IFL.
\\ \\
Individual split types may have specific requirements around how the service artifacts are packaged, so an SPI should exist to allow these types to process the split configuration output before it is bundled inside an application.
At line 219 added 1 line.
!! Runtime

JSPWiki v2.4.100
[RSS]
« Home Index Changes Prefs
This page (revision-14) was last changed on 21-Jul-09 09:42 AM, -0700 by MarkWhite