Enterprise Integration Patterns - Aggregate

Overview

The purpose of the aggregate pattern is to take a set of one or more input messages and create a single output message.

Use Cases

Aggregate can be used to satisfy a wide range of use cases depending on the message type and message flow requirements. The following use cases represent the most common categories for using aggregate.

Reassembly

Take a set of related messages and cram them into a single message. Most commonly used in conjunction with split and broadcast, where the number of messages in the input set is known.

Batching

Hold a bunch of messages and release them as a single batch message. The determination of when to release the batch message can be based on a time window, number of messages, size of messages, etc.

Enveloping

A refinement of batching, an enveloping aggregate can be used to create a group of messages with common characteristics (same type, same destination, same source, etc.). These messages are wrapped in one or more envelopes before they are forwarded as a single message.

Gating

An aggregate which can be used to make sure that messages from different flows are held until all messages arrive at the aggregate.

Selection

The aggregate can choose to only forward a subset of the aggregated messages.

IFL

Syntax

aggregate [type] [name | expression]
type := aggregate implementation (e.g. set, select)
name := named configuration which contains the aggregate details
expression := inline aggregate configuration

Examples

Aggregate with inline configuration

route do
   from "multiple-in"
   aggregate set {count=5}
   to "single-out"
end

Aggregate with external configuration

route do
   from "multiple-in"
   aggregate set "message-batch"
   to "single-out"
end

> cat message-batch/aggregate.properties
count=5

Configuration

Each aggregate type will have its own unique configuration requirements. The service artifact layer in Fuji will have a reasonable default behavior for all aggregate types. Individual aggregate types can override the default behavior by implementing the following contracts:

• Generation - the configuration generated from IFL for a given aggregate type. A facility must exist for discovering template service artifacts for a given aggregate type. This could be an SPI or a convention for resource location inside a bundle/jar file. For external configuration, a default implementation should be provided that results in an empty file with the aggregate configuration name used as the file name. Generation is not relevant for inline configuration.
• Validation - during the packaging phase of the application build, aggregate types will have an opportunity to validate configuration. This should be exposed as an SPI that a aggregate type provider can implement.

The directory layout for generated configuration should be:

app-root/
   aggregate/
      {name}/
           [configuration]

Where '{name}' is the named configuration provided in IFL.

Set

The set type provides for aggregation of input messages as a set. The configuration of set allows the user to define the conditions under which a set is complete and messages in the set are forwarded on as a single, aggregated message. The following configuration options are supported:

• size : the size threshold for aggregated messages. Once the threshold is reached, all messages in the set are aggregated. Valid units are 'mb', 'kb', and 'b' (e.g. 10mb = 10 megabytes).
• count : number of messages that must be in the set before they are aggregated into a single message.
• wait : amount of time to wait before the set is aggregated. Valid units are ms, s, m, h (e.g. 1h = 1 hour).
• order : specifies the order in which input messages are added to the aggregate message. Valid values are 'created' and 'received'.

Input Messages

<record>
   <num>001</item>
   <data>Veni</desc>
</record>

<record>
   <num>002</item>
   <data>Vidi</desc>
</record>

<record>
   <num>003</item>
   <data>Vici</desc>
</record>

Set Configuration

count=3

Output Message

<records>
  <record>
     <num>001</item>
     <data>Veni</desc>
  </record>
  <record>
     <num>002</item>
     <data>Vidi</desc>
  </record>
  <record>
     <num>003</item>
     <data>Vici</desc>
  </record>
</records>

<order>
   <item>parkayXYZ</item>
   <desc>butter</desc>
   <amount>100</amount>
</order>

Packaging

At application build time, all aggregate configurations should be packaged as part of the application. Since the ultimate consumer of this configuration information is the runtime aggregate implementation, and not a component, it is not appropriate to package the individual configuration instances into a service unit. Rather, the following convention should be followed:

app-root /
      META-INF /
            flow/
                 aggregate/
                    [configuration]

Where '{name}' is the named configuration provided in IFL. This layout mimics the configuration layout, but includes META-INF/ and flow/ as parent directories.

All aggregate configuration should be included in this directory, including inline configuration. Since inline configuration is not named, a name must be generated. Using a pattern of 'aggregate$n' should work, where 'n > 0' and increments for each inline aggregate definition in the IFL.

Runtime

This page (revision-10) was last changed on 22-Sep-08 07:57 AM, -0700 by KeithBabo.
This page was created on 20-Aug-08 12:38 PM, -0700 by KeithBabo.

More info...