Index Changes

FTP Binding Component Functional Specification

Update History:

09/19/2006 created

09/26/2006 added use cases etc.

10/26/2006 added screen shots

12/05/2006 added description for clustering & persisted sequence numbering

12/12/2006 added validation rules for extensibility elements

12/20/2006 added additional attributes messageName,

messageNamePrefixIB, messageNamePrefixOB, and stage for ftp:message

and ftp:messageActivePassive, and additional attributes

cmdChannelTimeout, dataChannelTimeout for ftp:address, also added

additional validation rules

12/23/2006 added more validation rules

01/26/2007 added SA migration support

08/12/2008 update the spec to cover new features and QoS added for Open ESB pack

08/26/2008 added "Password Handling" description for FTPBC

1. Introduction


The FTP Binding Component (refer to as FTP BC herein) is a binding
component implementation in compliance with JBI Specification 1.0.
As specified in JBI Specification 1.0 (JSR 208):
BCs are used to send and receive messages via particular protocols and transports.
They serve to isolate the JBI environment from the particular protocol by
providing normalization and denormalization from and to the protocol-specific
format, allowing the JBI environment to deal only with normalized messages. (Note
that protocol-specific metadata can be attached to a normalized message, or the
message exchange, in their metadata, allowing protocol-specific information to be
conveyed to a SE or BC in a fashion that is opaque to other JBI environment
components.)
FTP BC provides a message transportation via FTP protocol so that
services (which comprise operations) can be defined using WSDL and
bound to FTP as its underlying message transportation protocol,
other components in a JBI environment, e.g., Service Engines (SE)
can further orchestrate the services consumption & provision.
The FTP BC implements all required BC interfaces in JBI
specification so that it can be deployed and run in any JBI
compliant target environment.
To facilitate the process of service definition and binding, the
implementation also comes with a design time component –
a Netbeansmodule which makes WSDL authoring and FTP binding a convenient
process in Netbeans IDE.
With FTP BC, Service Engines can exchange messages with each other
through services defined via WSDLs that have FTP BC extensibility
elements as operation definition and binding mechanism, by “messages
exchange over FTP” or “message transportation over FTP”, we mean the
following:




Figure 1. message transportation via FTP

As illustrated, FTP BC uses directories on FTP server as message
persistence where service consumers and service providers exchange
messages, in the implementation, some assumptions are made so that
message transportation can be done in a FTP context, see section 3
“Message Transportation Via FTP” for more details.

2. Requirements & Features


This section outlines features that will be supported in FTP BC.

External Protocols

  • FTP (RFC 959)
  • FTP/TLS (RFC 2228 + RFC 4217)

Directory Listing Styles
Regarding the directory listing results, FTP servers on different OS platforms are diversified, built-in parsing logic has been introduced to parse the directory listing for the following directory listing style:
  • UNIX
  • AS400
  • AS400-UNIX
  • HCLFTPD 6.0.1.3
  • HCLFTPD 5.1
  • HP NonStop/Tandem
  • MPE
  • MSFTPD 2.0
  • MSP PDS (Fujitsu)
  • MSP PS (Fujitsu)
  • MVS GDG
  • MVS PDS
  • MVS Sequential
  • Netware 4.11
  • NT 3.5
  • NT 4.0
  • UNIX
  • UNIX (EUC-JP
  • UNIX (SJIS)
  • User Defined
  • VM/ESA
  • VMS
  • VOS3 PDS (Hitachi)
  • VOS3 PS (Hitachi)
  • VOSK (Hitachi)

Furthermore, FTP servers whose directory listings are not covered by
the above out-of-box styles, user defined FTP directory list parsing
mechanism is also available, by leveraging a configuration file
containing directory listing parsing information, user can specify
the rules (heuristics) how directory listing from a specific FTP
server should be parsed, refer to ftp:address for details.

Pre/Post Transfer Operations

It is common practice that some pre and/or post operations are
performed after a FTP PUT or GET, e.g., archive a file that is
downloaded successfully so it won't be downloaded again, or move
away a file that is in the way of a PUT so that it won't get
overwritten, etc.
As a value added convenience, the following pre/post transfer
operations are supported:

Pre Operations:

  • COPY – make a copy of the target file to a location specified
by other pre operation parameters before the transfer is carried out
  • RENAME – move the target file to a location specified by other
pre operation parameters before the transfer is carried out Post Operations:
  • DELETE – delete the target file after the transfer is carried
out
  • RENAME – move the target file to a location specified by other
post operation parameters after the transfer is carried out

Proxy Protocols

Proxy protocols supported : SOCKS4, SOCKS5

Note that proxy configuration is in the scope of the FTP BC deployed
on a host, not per service information, hence, its configuration is
a FTP BC Runtime configuration parameter.

Connection Pooling

Connections to FTP servers should be pooled


FTP BC Extensibility Elements

In SOA business orchestration, FTP BC can be bound to either a
service consumer or service provider, the interfaces exposed are
defined by a WSDL, FTP BC implements a set of FTP BC binding
specific extensibility elements so that a service can be defined and
bound to FTP protocol.
The followings are FTP BC WSDL extensibility elements implemented

  • Address - FTP connectivity element, specifies the FTP
connectivity information such as, FTP URL (host, port, login, password), directory listing style, user defined heuristics for directory listing parsing.
  • Binding - FTP binding element, a marker element indicating a
FTP binding, this element does not have attributes.
  • Operation – FTP operation element, a marker element indicating
a FTP operation, this element does not have attributes.
  • Transfer – FTP transfer element, specifies a message transfer
from a sender and receiver perspective, e.g., to specify a message transfer for a service request, there can be a sender and a receiver involved, the WSDL author can specify: (1) the target sender sends to – represented by attribute ftp:sendTo, the target receiver receives from – represented by attribute ftp:receiveFrom, also, the additional operations performed before a message is sent (PUT) to target or after a message is received (GET) from the target – called Pre/Post operations. (2) messageCorrelate – if enabled, a UUID tagging based message correlation scheme will be used to correlate requestresponse of a synchronous service invoking, etc.
  • Message – FTP message element, specifies a message transfer
from a service consumer and/or service provider perspective, the WSDL author can specify: (1) the message repository – represented as attribute ftp:messageRepository – a base directory where all the working directories for a message transfer are created, such as “inbox” for posting request (by consumer) and polling request (by provider), “instage” for staging request, “outbox” for posting response (by provider) and polling response (by consumer), “outstage” for staging response, etc., (2) messageCorrelate – if enabled, a UUID tagging based message correlation scheme will be used to correlate request-response of a synchronous service invoking, etc.
  • MessageActivePassive (deprecated) – FTP messageActivePassive element, same
as FTP Message element, but allows consumer and provider set their FTP usePassive flag to different values. Please refer to section 5 “FTP BC Extensibility Elements” for detailed annotations for each elements' attributes, and refer to ftp_ext.xsd from main FTPBC wiki page for full schema definition of the extensibility elements. Patterns For Message Transfer Target Specification Name pattern is a string including special character that needs to be expanded, special characters are escaped by percentage sign %.
As can be seen from section 3 “Message Transportation Via FTP”, FTP directory & file name leveraging is the corner stone for message exchange via FTP, the following name patterns are provided to facilitate service providers and service consumers to specify message file names that have agreed semantics between them. The characters indicate the escaped character %. For example, abcmeans abc%d, and %d is not expanded again. date/time format is used, e.g., abc%y%y%y%y means abc2001.
  • Working file name replacement (%f), usually used in pre/post
operation's destination directory / file, e.g., when working file is invoice.dat, then %f.bak will be invoice.dat.bak after expansion.
  • UUID - %u, a UUID compliant with JDK 1.5 java.util.UUID, when
used in a name pattern, a unique file name can be generated and used to put a message to a directory without the problem of overwriting other files, also UUID names are used for message correlation.
  • Sequence Numbering - %{seq_name}, a reference to a persisted
sequence numbering, refer to “FTP BC Clustering Support Design” for more details

Application Variable

A service assembly (SA) configured for one particular context are often migrated to a different context, one of the context specific information is the binding information in extensibility elements' attributes, for example, a service bound to FTP could have the following segment in its WSDL:

...
<ftp:address url=”ftp://usr1:pass1@localhost:21” ...>
...

for different migration destination context, the url could be different, and needs be re-configured, note, such attributes can be large in number in a real life WSDL, considering there can be multiple WSDLs in one SA, the re-configuration can be time consuming and error-prone.

To facilitate the migration, FTP BC supports a mechanism - SA migration environment variables, environment variables ('tokens') are defined in the runtime configuration property - “Application Variables” for a FTP BC installation, “Application Variable” is a tabular data consisting of one or more name value pairs, on the other hand, WSDL authors are allowed to include references of these 'tokens' in the attributes values of FTP BC extensibility elements in their WSDL, the references are resolved at deployment time.

The following is an example of element with attributes containing variable references:
<ftp:address
url=”ftp://${my_ftp_acct}:${my_ftp_acct_password}@${my_ftp_host}:${my
_ftp_port}” ...>

3. Message Transportation Via FTP

As illustrated in Figure 1, FTP BC uses directories on FTP server as
message persistence where service consumers and service providers
exchange messages, in the implementation, some assumptions are made
so that message transportation can be done in a FTP context in a more
controllable manner, especially when a directory is accessed
concurrently by multiple components (refer to “FTP BC Clustering
Support Design” for more details).
Here are the assumptions:

(1) For a specific service operation with FTP binding, the endpoint
information, i.e., host name, port, account login, password, plus a
base directory, specifies a message exchange area on the remote FTP
server where consumer and provider communicate, take a requestresponse
type operation for example, the followings are performed in
the area when carrying out the message transportation from consumer
to provider (request – IN-Route) and from provider to consumer
(response – OUT-Route):

Consumer: Staging (Put) request
Consumer: Expose (Rename to target location) request to provider when
staging completes
Provider: Poll (Get periodically) request
Provider: Archive/Remove request after received
Provider: Staging (Put) response
Provider: Expose (Rename to target location) response to consumer
when staging completes
Consumer: Poll (Get periodically) response
Consumer: Archive/Remove response received
The message exchange area – a dedicated directory on remote FTP
server and all its sub-directories (serving as archiving area,
staging area, exchange area, etc) can be created by either FTP BC
runtime or administratively, just as queues and topics in JMS
paradigm.

(2) For a given service, both the service consumer and service
provider either login using the same FTP account (so they share the
same FTP context – e.g., login home where all relative paths refer
to), or if different accounts are used, administrative configuration
need to be made to ensure the login home is mapped to the same
location on FTP server and appropriate access rights granted
accordingly

(3) Different service operations use dedicated message exchange areas
on the remote FTP server, no overlapping between these base
directories

(4) FTP file names are leveraged so that the consumers and providers
can push and poll messages between them, e.g., consumers post request
messages into FTP file with names that match the following pattern:
msg.., while the corresponding providers retrieve
these request messages with the same pattern from the agreed
location, similar process occurs for the response routing from
providers to consumers, numerous patterns are available for
configuration of a particular message transfer (implemented as FTP BC
extensibility elements: ftp:message, ftp:messageActivePassive (deprectaed),
ftp:transfer) so that messages posted got polled according to their
name pattern; note, however, it does not provide request-response
messages correlate across the FTP

(5) Request-response correlation is supported by leveraging names of
the FTP files that serve as intermediary message persistence on FTP
server, the contract is described as follows:

On the consumer side:

the message routing starts with consumer who invokes a service
(INVOKE in BPEL script), on the other side of the NMR, FTP BC
OutboundProcessor accepts the request message and de-normalized it
and put the message body – the payload to a FTP file with name as
req. (see Figure 1 for examples), then the consumer
thread (FTP BC OutboundProcessor) will spawn off a ResponsePoller
thread which starts polling a response with name
resp., note that the is the same
UUID for request and response, when resp. becomes
available at the agreed on location between the consumer and
provider, it is fetched, and the request-response correlation
completes, the fetched message is wrapped up in a normalized message
and send back into NMR using the same Message Exchange ID so that the
response will be available as the OUT variable of the INVOKE activity
in the BPEL script.


Figure 2. UUID tagged request files
On the provider side:

the provider polls (RECEIVE activity in BPEL) the message receiving
area that is agreed on between the consumer and provider for a
request with FTP file name pattern like req.%u, where suffix %u
denotes any string match the pattern of a typical UUID in string form
(refer to JDK java.util.UUID – since 1.5), see Figure 1 for examples.
Upon receive of the request message (polled from remote FTP
directory), the UUID tag is extracted from the file name, and
attached as meta data for the message exchange, the payload is
wrapped in a normalized message and send into the NMR.
The request message participates in any business logic in the service
unit and a response is figured out and send back to NMR with the same
message exchange (ME – attached with meta information including the
UUID tagged with the request), the OutboundProcessor will fulfill the
contract by putting the message to a FTP file named as
resp., see Figure 3 for exampes of UUID tagged
response messages.


Figure 3. UUID tagged response files

This completes the request-response correlation.

Note that message correlation scheme can be applied to multi-hop
service invoking, i.e., during the processing of the first service
invoking, e.g., invoke service1.operation1, the provider might in
turn invoke other services, such as service2.operation2 as shown
below:



Figure 4. Two hop synchronous services invoking with message
correlation

Note also, though the UUID tagging mechanism described above is
provided by FTP BC implementation, it is not the only means to
achieve message correlation, the service unit doing the message
transportation can use the information embedded inside the message
payload to do the correlation at application level, but that is
beyond the scope of this document, see scenarios on FTP BC wiki page
for samples.

4. FTP BC Runtime Configuration


The following is FTP BC runtime configuration property sheet:


Figure 5. Runtime configuration for FTP BC

4.1 FTP BC Runtime Parameters


  • Number of Outbound Threads – the number of outbound threads, default is 10.
  • InvokeTimeout – Time out in milliseconds for a synchronous service invoking, after a consumer sends a request, it will poll the response periodically (the polling interval is specified by ftp:pollIntervalMillis), InvokeTimeout is the elapsed milliseconds when the response poller thread will exit and report a timeout, default is 1000,000.
  • Use Passive FTP - indicate if passive ftp is enabled.
  • UseProxy – if proxy is used for FTP transfer, default is “false”.
  • ProxyURL – the URL for the proxy, default is “socks5://localhost:1080”.
  • Proxy User ID - if it is socks5, the user login id to the socks5 server, if specified, overwrite the one in ProxyURL
  • Proxy ser Password - if it is socks5, the password for the specified user id, if specified, overwrite the one in ProxyURL
  • Connection Pool Minimum Size - default = 2, the minimum number of connections that should be kept in the pool
  • Connection Pool Maximum Size - default = 32, the maximum number of connections that can be kept in the pool, when a connection is returned to the pool, the value is checked, if the return of the connection will cause the pool size greater than this value, the connection is thrown away
  • Max Idle Timeout for Pooled Connections - default = 60000, a timeout in milli-seconds, if a connection stays in the pool longer than the value, the connection will be evicted from the pool.
  • Application Variables – Application variables are tokens defined for specific JBI components on the application server, application variables can be referenced in the attribute values of the extensibility elements in the specific BC's WSDLs, the references are resolved at the Service Unit deployment time.
  • Application Configuration - Application configuration is a mechanism that allows the definition of named group of configuration parameters specific to a JBI component, and allow the group of parameters be referenced by an end point so that the binding can be completed with the parameters (e.g. host, port, user, password, etc.) provided in the named group.

4.2 FTP BC Application Variables


Application variables are tokens defined for specific JBI components on the application server,
application variables can be referenced in the attribute values of the extensibility elements in
the specific BC's WSDLs, the references are resolved at the Service Unit deployment time, application
variables are implemented as tabular data of name value pairs, each pair represents an environment
variable and its current value, see the following screen shot for example:

The major usage of the application variable is to be referenced by attributes in the WSDL so that the WSDL is parameterized such that those attribute value which are dependent on the target environment can be resolved at the deployment time.
For example, A WSDL is created to define a web service getStockQuote and is bound to FTP transporation, but at the time when the WSDL is created, the author might not know the actual FTP server info such as host name, port, user ID, password, dir list style, etc., using Application Variable, the WSDL can be defined and the composite application can be created, build, and ready to be deployed (see figure 8).
At the deploy time, the administrator can have the variables defined (as shown below) at the target deploy destination (e.g. application server).
Fail to resolve any referenced variable will result in deployment failure.

As shown in figure 5, Runtime configuration parameters "Application Variables" and "Application Configuration" can be edited further using application variable editor and application configuration editor, just click the button on the right side of the parameter value field will bring up the respective editor.

Figure 6. Adding new application variable
Application variables can be categorized into the following types:

STRING
BOOLEAN
NUMBER
PASWORD

here are some examples:


Figure 6. Application variables of different types
Click OK of the editor, the newly added variable will be persisted, Click Cancel, the newly added entries will be lost.

Figure 7. Added application variables persisted
As can be seen, the variables are serialized and the result is placed in the value field.
Application Variables referenced in WSDL:


Figure 8. Reference application variables in WSDL


4.3 FTP BC Application Configurations


Application configuration is a mechanism that allows the definition of named group of configuration parameters specific to a JBI component, and allow the group of parameters be referenced by an end point so that the binding can be completed with the parameters provided in the named group.
Application configuration is defined through using runtime configuration propertie sheet, and its reference can be specified through CASA editor - composite application editor as shown below:


Figure 9. Open up FTPBC application configuration editor for application configuration object named myAppCfgObj001

Figure 10. Reference of FTPBC application configuration object named myAppCfgObj001 in a JBI endpoint through CASA editor
Reference to application configuration objects from a composite application will be resolved at deployment time.
Note, parameters in an application configuration object can further reference application variables.
Password Handling in FTPBC:
There are runtime configuration parameters and WSDL extensibility elements' attributes that serve as 'password', they should be masked on the GUI and encrypted when in storage, for FTPBC, for example, "ProxyUserPassword" in FTPBC runtime configuration, "KeyStorePassword" from FTPBC Application Configuration.

Attributes in WSDL extensibility elements such as ftp:message are xsd:string type, and since there is no way for a generic editor to distinguish a xsd:string value used as 'password' or plain text, the GUI could not mask it, as an alternative, APP VAR can be used as the value of a 'password', and in turn the corresponding APP VAR can be type PASSWORD, and be encrypted when persisted out at the target app server.

Here are some examples showing the password handling:
Password handling covers the following:

Runtime Configuration Params (Top Level):

persisted in the file: /domains/domain1(e.g.)/jbi/components/sun-ftp-binding/install_root/workspace/config.properties

#Tue Aug 26 13:21:26 PDT 2008
ProxyUserID=proxy_user_001
ConnectionPoolMaxSize=32
ProxyURL=socks5\://localhost\:1080
ConnectionMaxIdleTimeout=60000
ConnectionPoolMinSize=2
UseProxy=false
ProxyUserPassword=xuVNoOTI6ShZYj2zMy8YQw\=\=
UsePassiveFTP=true
InvokeTimeout=1000000
OutboundThreads=10

Runtime Configuration Params (App Var):

persisted in the file: /domains/domain1(e.g.)/jbi/components/sun-ftp-binding/install_root/workspace/ApplicationVariables.properties

my_ftp_host=nova.stc.com{STRING}
my_ftp_port=21{NUMBER}
my_ftp_password=xuVNoOTI6ShZYj2zMy8YQw=={PASSWORD}

Runtime Configuration Params (App Configurations):

persisted in the file: /domains/domain1(e.g.)/jbi/components/sun-ftp-binding/install_root/workspace/ApplicationConfigurations.properties

myAppCfgObj001.SECURE_TYPE=ExplicitSSL
myAppCfgObj001.KEYSTORE=c:\key_and_trust\keystore.jks
myAppCfgObj001.USERDEFINEDDIRLISTSTYLECFG=
myAppCfgObj001.USEUSERDEFINEDDIRLISTSTYLE=false
myAppCfgObj001.PASSWORD=xuVNoOTI6ShZYj2zMy8YQw==
myAppCfgObj001.DIRLISTSTYLE=UNIX
myAppCfgObj001.configurationName=myAppCfgObj001
myAppCfgObj001.PORT=21
myAppCfgObj001.USERDEFINEDDIRLISTSTYLE=
myAppCfgObj001.TRUSTSTOREPASSWORD=xuVNoOTI6ShZYj2zMy8YQw==
myAppCfgObj001.KEYSTOREPASSWORD=xuVNoOTI6ShZYj2zMy8YQw==
myAppCfgObj001.USER=user1
myAppCfgObj001.KEYALIAS=ftp_client_001
myAppCfgObj001.KEYPASSWORD=VWiPa0YG6Ob+dOcTW4F6iQ==
myAppCfgObj001.HOST=nova.stc.com
myAppCfgObj001.TRUSTSTORE=c:\key_and_trust\keystore.jks
myAppCfgObj001.ENABLE_CCC=false
myAppCfgObj002.SECURE_TYPE=None
myAppCfgObj002.KEYSTORE=
myAppCfgObj002.USERDEFINEDDIRLISTSTYLECFG=C:\MY_FTP_HEURISTICS.CFG
myAppCfgObj002.USEUSERDEFINEDDIRLISTSTYLE=true
myAppCfgObj002.PASSWORD=+SCWwXgGwzOR3PsSvaCc/w==
myAppCfgObj002.DIRLISTSTYLE=AS400
myAppCfgObj002.configurationName=myAppCfgObj002
myAppCfgObj002.PORT=21
myAppCfgObj002.USERDEFINEDDIRLISTSTYLE=MY_AS_400
myAppCfgObj002.TRUSTSTOREPASSWORD=GA324/Om18roWSXF30Lx+g==
myAppCfgObj002.KEYSTOREPASSWORD=GA324/Om18roWSXF30Lx+g==
myAppCfgObj002.USER=anonymous
myAppCfgObj002.KEYALIAS=
myAppCfgObj002.KEYPASSWORD=GA324/Om18roWSXF30Lx+g==
myAppCfgObj002.HOST=localhost
myAppCfgObj002.TRUSTSTORE=
myAppCfgObj002.ENABLE_CCC=false

5. FTP BC Extensibility Elements


Annotation of Extensibility Elements

FTP BC supports the following extensibility elements:
  • <ftp:binding> - required as indication of FTP based binding, no attributes.
  • <ftp:operation> - required as indication of FTP based operation binding, no attributes.
  • <ftp:address> - specify end-point for FTP based operation binding, see annotations for attribute descriptions.
  • <ftp:message> - specify message transfer for FTP based operation binding, this element takes same attribute values when used by either service consumer and service provider, making it suitable to be used in a symmetrical WSDL, see annotations for attribute descriptions.
  • <ftp:transfer> - specify message transfer details for FTP based operation binding, the difference between this element and <ftp:message> are that the attributes are represented from perspective of sender and receiver involved in the message transfer, see annotations for attribute descriptions.

The following are annotations for each of the FTP BC extensibility
elements, name space = http://schemas.sun.com/jbi/wsdlextension/
ftp/, prefix = ftp.
Note, for current implementation, all attributes, except for those
attributes of type boolean, enumeration, can include application
variable references in its value, for example:

In the following element,

<ftp:address
url=”ftp://${my_ftp_acct}:${my_ftp_acct_password}@${my_ftp_host}:${my
_ftp_port}” ...>

Attribute “url” will be resolved to
ftp://usr1:abc\@yahoo.com@eagle.sun.com:1021 if all the variables are
defined as shown in the screen shot in previous section.
Attribute with variable references in its value is not subject to
validations concerning the format of the content.

* ftp:address

#Element NameAttribute NameDescriptionMandatoryDefaultExample
1ftp:addressftp:urlURL pointing to FTP serverYesN/Aftp://<user>:<password>@<host>:<port>
2ftp:addressftp:userFTP login user ID - if specified, overwrite the user ID in ftp:urlNo
3ftp:addressftp:passwordFTP login password, if specified, verwrite the password in ftp:urlNo
4ftp:addressftp:dirListStyleout of box listing stylesNoUNIX
5ftp:addressftp:userDefDirListStyleuser defined listing style nameNoN/A
6ftp:addressftp:useUserDefinedHeuristicsindicate if user defined listing style will be usedNoN/A
7ftp:addressftp:userDefDirListHeuristicslocation of user defined heuristics fileNoN/A
8ftp:addressftp:scuredFTPsecured FTP type - None, ExplicitSSL, ImplicitSSL NoneExplicit
9ftp:addressftp:enableCCCWhen ExplicitSSL selected, enableCCC indicates if Control Channel will be cleared after handshakefalsefalse
10ftp:addressftp:keyStoreWhen secured FTP is ExplicitSSL or ImplicitSSL, keyStore is the key store locationNoc:/key_stores/my_kstore.jks
11ftp:addressftp:keyStorePasswordPassword of the key storeNo
12ftp:addressftp:keyAliasAlias for the key pair entry in key store
13ftp:addressftp:keyPasswordPassword protecting the key pair entry
14ftp:addressftp:trustStoreWhen secured FTP is ExplicitSSL or ImplicitSSL, trustStore is the trust store locationNoc:/trust_stores/my_tstore.jks
15ftp:addressftp:trustStorePasswordPassword of the trust storeNo
16ftp:addressftp:modeFTP transfer mode - ASCII or BINARYBINARYASCII
17ftp:addressftp:controlChannelEncodingthe charset name for FTP control channelISO-8859-1ISO-8859-1
18ftp:addressftp:cmdChannelTimeoutFTP command channel timeout in millis4500045000
19ftp:addressftp:dataChannelTimeoutFTP data channel timeout in millis4500045000


(1) url (String): FTP URL in the form of

ftp://<user>:<password>@<host>:<port>

where user is the ftp
account login, password is the password for the account, host is
the host where FTP server is running, port is the FTP port
(default is 21).

(2) user (no default) - the FTP login user id, if specified, will overwrite the one in url
(3) password (no default) - the FTP user password, if specified, will overwrite the one in url
(4) dirListStyle (Enumeration – default is UNIX): built-in directory listing styles, see section 2 “Requirements & Features” for a full list of directory listing styles.

(5) userDefDirListStyle (String): name of the user defined style, it should match an entry in a heuristics file specified by “userDefDirListHeuristics”.

(6) useUserDefinedHeuristics (true/false – default is false): indicate if a user defined directory listing style is used instead of built-in styles.

(7) userDefDirListHeuristics (String): a path pointing to a user provided heuristics file, this file should be accessible to the FTP BC runtime in the deployed environment, note, this is not applicable to clustered environment, the content of the file are entries for each user defined styles, see APPENDIX A for an example, the entry key is the name of the user defined style, and the entry value is the regular expression used to parse directory listing result.

(8) securedFTP (default : "None") - the secured FTP type, Enumeration of {"None", "ExplcicitSSL", "ImplicitSSL"}
(9) enableCCC (default : false) - enable Clear Command Channel after SSL handshake completed for FTP control channel, applicable only to "ExplicitSSL"
(10) keyStore (no default) - when secureFTP is either "ExplicitSSL" or "ImplicitSSL", keyStore is the location of the key store on the target system
, when not specified, can default to trust store
(11) keyStorePassword (no default) - the password protecting the key store
(12) keyAlias (no default) - Alias for the key pair entry in key store, associated with the client cert used in client AUTH
(13) keyPassword (no default) - the password protecting the keyAlias entry
(14) trustStore (no default) - when secureFTP is either "ExplicitSSL" or "ImplicitSSL", trustStore is the location of the trust store on the target system
, when not specified, can default to key store
(15) trustStorePassword (no default) - the password protecting the trust store
(16) mode (Enumeration): BINARY/ASCII, default is BINARY, FTP transfer mode.

(17) controlChannelEncoding (default ISO-8859-1) - FTP Control Channel encoding, when left blank - ISO-8859-1 assumed
(18) cmdChannelTimeout (Integer >= 0): socket read time out in millisecond for FTP command channel, 0 indicates infinite.

(19) dataChannelTimeout (Integer >= 0): socket read time out in millisecond for FTP data channel, 0 indicates infinite.


Attributes from ftp:address are grouped into "Connection Settings", "Directory Listing Style", "Security Settings", "Transport Settings" (QoS : Usability)
as shown in figure 11.


Figure 11. Attribute Grouping for ftp:address

* ftp:transfer

#Element NameAttribute NameDescriptionMandatoryDefaultExample
1ftp:transferftp:sendTotarget spec for sender - could be a patternNoN/A
2ftp:transferftp:sendToHasPatternsif sendTo contains patternsNofalse
3ftp:transferftp:appendif message is appended to the target fileNofalse
4ftp:transferftp:receiveFromtarget spec for receiverNoN/A
5ftp:transferftp:receiveFromHasRegexsif the target spec has regexsNofalse
6ftp:transferftp:preSendCommandoperation performed before send (put) message to targetNoNONE,COPY,RENAME
7ftp:transferftp:preSendLocationdestination of preSendCommandNoN/A
8ftp:transferftp:preSendLocationHasPatternsif the destination contains patternsNofalse
9ftp:transferftp:postSendCommandoperation performed after send (put) message to targetNoNONE,RENAME
10ftp:transferftp:postSendLocationdestination of postSendCommandNoN/A
11ftp:transferftp:postSendLocationHasPatternsif the destination contains patternsNofalse
12ftp:transferftp:preReceiveCommandoperation performed before receive (get) a message from the targetNoNONE,COPY,RENAME
13ftp:transferftp:preReceiveLocationdestination of preReceiveCommandNoN/A
14ftp:transferftp:preReceiveLocationhasPatternsif the destination contains patternsNofalse
15ftp:transferftp:postReceiveCommandoperation performed after message is received (get) from the targetNoNONE,DELETE,RENAME
16ftp:transferftp:postReceiveLocationdestination of postReceiveCommandNoN/A
17ftp:transferftp:postReceiveLocationHasPatternsif the destination contains patternsNofalse
18ftp:transferftp:senderUsePassiveif FTP passive mode is enabled for the senderNotrue
19ftp:transferftp:receiverUsePassiveif FTP passive mode is enabled for the receiverNotrue
20ftp:transferftp:useweather the message is used as a literal or encoded by an encoder - encoded / literal NoN/Aliteral
21ftp:transferftp:partreference to the part name of the abstract messageNoN/A
22ftp:transferftp:encodingStyleencoding style used to identify an associated encoderNoN/Aud1encoder-1.0
23ftp:transferftp:pollIntervalMillisthe polling interval in milli seconds when ftp:transfer is used by a polling partyNoN/A

(1) sendTo (String): the target where the sender sends the message to, i.e., the target of a FTP put, it takes the form of relative //.../ or absolute ///.../, where , , ..., can be literal or pattern, depending on “sendToHasPatterns”.

(2) append (true/false – default is false): a flag indicating if the message is appended to the target file, when true, the message is appended to the target file (specified as “sendTo”), otherwise, it is an overwrite.

(3) sendToHasPatterns (true/false – default is false): when true, the “sendTo” contains patterns in its path components and/or file name, otherwise, the path and file name are literals.

(4) receiveFrom (String): the target where the receiver polls the message from, i.e., the target of a FTP get, it takes the form of relative //.../ or absolute ///.../, where , , ..., can be literal or regular expression, depending on “ReceiveFromHasRegexs”.

(5) receiveFromHasRegexs (true/false – default is false): when true, the “receiveFrom” contains regular expressions in its path components and/or file name, otherwise, the path and file name are literals.

(6) pollIntervalMillis (Integer – default is 5000): interval for an inbound component (receiver) polling the target in milliseconds.

(7) preSendCommand (Enumeration – default is NONE): operation performed before sending the message, can be NONE/COPY/RENAME.

(8) preSendLocation (String): a destination file for “preSendCommand”, it takes the form of relative //.../ or absolute ///.../, where , , ..., can be literal or pattern, depending on “preSendLocationHasPatterns”.

(9) preSendLocationHasPatterns (true/false – default is false): a flag indicating if “preSendLocation” contains patterns.

(10) postSendCommand (Enumeration – default is NONE): operation performed after sending the message, can be NONE/RENAME.

(11) postSendLocation (String): a destination file for “postSendCommand”, it takes the form of relative //.../ or absolute ///.../, where , , ..., can be literal or pattern, depending on “postSendLocationHasPatterns”.

(12) postSendLocationHasPatterns (true/false – default is false): a flag indicating if “postSendLocation” contains patterns.

(13) preReceiveCommand (Enumeration – default is NONE): operation performed before receiving the message, can be NONE/COPY/RENAME.

(14) preReceiveLocation (String): a destination file for “preReceiveCommand”, it takes the form of relative //.../ or absolute ///.../, where , , ..., can be literal or pattern, depending on “preReceiveLocationHasPatterns”.

(15) preReceiveLocationHasPatterns (true/false – default is false): a flag indicating if “preReceiveLocation” contains patterns.

(16) postReceiveCommand (Enumeration – default is NONE): operation performed after receiving the message, can be NONE/DELETE/RENAME.

(17) postReceiveLocation (String): a destination file for “postReceiveCommand”, it takes the form of relative //.../ or absolute ///.../, where , , ..., can be literal or pattern, depending on “postReceiveLocationHasPatterns”.

(18) postReceiveLocationHasPatterns (true/false – default is false): a flag indicating if “postReceiveLocation” contains patterns.

(19) senderUsePassive (true/false – default is true): use passive FTP on the sender side.

(20) ReceiverUsePassive (true/false – default is true): use passive FTP on the receiver side.

(21) use (Enumeration): “literal”/”encoded”, indicating if the message is a literal or has an encoder associated with it, when use=”encoded”, encodingStyle must be specified.

(22) encodingStyle (String): URI identifying a encoder for the message body.

(23) part (String): reference to a part name in the abstract message defined in the WSDL for the operation concerned.

(24) messageCorrelate (true/false – default is true): a flag indicating if the UUID tagging based message correlation is enabled, when true, the message correlation is enabled, otherwise, message correlation is not enabled.

Attributes from ftp:transfer are grouped into "Sender", "Receiver", "Message Encoding", "Message Part", "Message Correlate" (QoS : Usability)
as shown in figure 12.


Figure 12. Attribute Grouping for ftp:transfer

* ftp:message


#Element NameAttribute NameDescriptionMandatoryDefaultExample
1ftp:messageftp:messageRepositorydirectory on FTP server where messages are stored, staged, retrived and archivedYesN/A
2ftp:messageftp:messageNamemessage name, usually is a name pattern expressionNo%u
3ftp:messageftp:messageNamePrefixIBmessage name prefix for inbound messageNoreq.
4ftp:messageftp:messageNamePrefixOBmessage name prefix for outbound messageNoresp.
5ftp:messageftp:pollIntervalMillisthe polling interval in milli seconds when ftp:message is used as inboundNoN/A
6ftp:messageftp:archiveif message is moved to a archive directory after retrievedNotrue
7ftp:messageftp:protectif message is moved to a protect directory before overwritten by next messageNotrue
8ftp:messageftp:stageif staging is enabled for message transportationNotrue
9ftp:messageftp:useweather the message is used as a literal or encoded by an encoder - encoded/literalNoN/Aliteral
10ftp:messageftp:partreference to the part name of the abstract messageNoN/A
11ftp:messageftp:encodingStyleencoding style used to identify an associated encoderNoN/Aud1encoder-1.0
12ftp:messageftp:messageCorrelateif message name GUID tagging is enabled to correlate request to a response during a synchronous service invokingYestrue


Among other message transfer information, implicitly assumes the working directories layout shown below for per service message exchange.

Figure 5. working directories layout for per service message transfer specified by ftp:message


(1) messageRepository (String): a base directory on FTP server, where sub-directories are created as working directories for conducting message transfer, see Figure 5 for an example, where messageRepository = MSG_REPO – a path relative to login account home directory, and working directories are created using default values such as “inbox”, “instage”, “inarchive”, “outbox”, “outstage”, and etc.

(2) messageName (String – default “%u”): message name, i.e., the name of the FTP file created for each message transported via FTP, usually it is a name pattern expression containing special symbols (escaped by percentage sign) such as %u (a UUID value), %h%h (hour of current time), %S%S%S%S (millisecond of current time), etc.

(3) messageNamePrefixIB (String – default is “req.”): a prefix added in front of the messageName to form the actual message name for messages transported in the INBOUND direction (consumer to provider), it should not contain any pattern symbols nor percentage sign.

(4) messageNamePrefixOB (String – default is “resp.”): a prefix added in front of the messageName to form the actual message name for messages transported in the OUTBOUND direction (provider to consumer), it should not contain any pattern symbols nor percentage sign.

(5) pollIntervalMillis (Integer): interval for an inbound component polling the target, default is 5000 milliseconds.

(6) archive (true/false - default is true): a flag indicating if message will be archived after it is fetched by a component (either consumer or provider), when true, the message will be archived (moved to a dedicated directory such as “inarchive” or “outarchive”), when false, the message will be deleted.

(7) protect (true/false - default is true): a flag indicating if in the way message will be moved to a dedicated directory such as “inprotect” or “outprotect” before incoming message overwrite it, when true, in the way message will be moved (protected), when false, overwritten.

(8) stage (true/false – default is true): a flag indicating if staging is enabled or not during message transfer.

(9) use (Enumeration): “literal”/”encoded”, indicating if the message is a literal or has an encoder associated with it, when use=”encoded”, encodingStyle must be specified.

(10) encodingStyle (String): URI identifying a encoder for the message body.

(11) part (String): reference to a part name in the abstract message defined in the WSDL for the operation concerned.

(12) messageCorrelate (true/false - default is true): a flag indicating if the UUID tagging based message correlation is enabled, when true, the message correlation is enabled, otherwise, message correlation is not enabled.
Attributes from ftp:message are grouped into "Messaging Configuration", "Message Encoding", "Message Part", "Message Correlate" (QoS : Usability)
as shown in figure 13.


Figure 13. Attribute Grouping for ftp:message

Extensibility Elements Validation Rules

A set of validation rules are introduced and can be checked at design time and enforced at runtime as well, such that errors in extensibility element's settings can be detected as early as possible.

The following table list all the validation rules and their corresponding error messages are shown in “comment” column, see APPENDIX B for a list of all validation error messages.

Name spaces:

xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:ftp="http://schemas.sun.com/jbi/wsdl-extensions/ftp/"

Design Time Validation Rules:

element attribute rule comment
ftp:binding for one operation, there can be at most 1 ftp:binding FTPBindingValidation.ONLY_ONE_FTP_BINDING_ALLOWED
ftp:address if ftp:binding presents for the operation, ftp:address must present FTPExtValidation.MISSING_FTP_ADDRESS
ftp:address url when ftp:address presents, attribute url is mandatory,
and must be a valid FTP
URL defined as follows:
ftp://<user>:<password>@<host>:<port>
FTPAddress.MISSING_FTP_URL
ftp:address url url must start with ftp:// FTPAddress.INVALID_FTP_URL_PREFIX
ftp:address url must be a valid URL defined as follows:
ftp://<user>:<password>@<host>:<port>
FTPAddress.MALFORMED_FTP_URL
ftp:address url url should not include a path FTPAddress.INVALID_FTP_URL_PATH_NOT_ALLOWED
ftp:address url url must include FTPAddress.MALFORMED_FTP_URL_HOST_REQUIRED
ftp:address url url still have a place holder value – not set with a real URL FTPAddress.REPLACE_FTP_URL_PLACEHOLDER_WITH_REAL_URL
ftp:address url in URL must be positive integer FTPAddress.INVALID_PORT_IN_URL
ftp:address useUserDefinedHeuristics, userDefDirListStyleif useUserDefinedHeuristics = true,
userDefDirListStyle must be specified
FTPAddress.MISSING_UD_DIRLSTSTYLE_NAME
ftp:address useUserDefinedHeuristics, userDefDirListHeuristics if useUserDefinedHeuristics = true,
userDefDirListHeuristics must be specified
FTPAddress.MISSING_UD_HEURISTICS_CFG_LOC
ftp:address cmdChannelTimeout,dataChannelTimeout Must be >= 0 FTPAddress.INVALID_FTP_CH_TIMEOUT
ftp:transfer sendTo, receiveFrom both sendTo and receiveFrom not specified is
a sure invalid ftp:transfer
FTPTransfer.BOTH_SENDTO_AND_RECEIVEFROM_ARE_NOT_SPECIFIED
ftp:address preSendCommand,
preSendLocation,
postSendCommand,
postSendLocation,
preReceiveCommand,
preReceiveLocation,
postReceiveCommand,
postReceiveLocation
if pre/post operation presents, i.e., not “NONE”,
corresponding location must be specified.
FTPTransfer.PRE_POST_OPERATION_WO_LOCATION
ftp:transfer sendTo, receiveFrom,
preSendLocation,
postSendLocation,
preReceiveLocation,
postReceiveLocation
It is invalid to specify a
directory (i.e., path ending
with '/') as value for these
attributes.
FTPTransfer.A_PATH_POINTING_TO_FILE_REQUIRED
ftp:transfer use, encodingStyle If use=”encoded”,
encodingStyle must be
specified with a URI
identifying a encoder.
FTPTransfer.MISSING_STYLE_WHEN_USE_ENCODED
ftp:message use, encodingStyle If use=”encoded”,
encodingStyle must be
specified with a URI
identifying a encoder.
FTPMessage.MISSING_STYLE_WHEN_USE_ENCODED
ftp:message messageRepository messageRepository must be specified FTPMessage.A_PATH_POINTING_TO_MESSAGE_EXCHANGE_AREA_REQUIRED
ftp:message,
ftp:transfer
ftp:binding presents, but there is no ftp:transfer or
ftp:message in wsdl:input
FTPBindingValidation.NO_EXTENSIBILITY_ELEMENT_FOUND_IN_INPUT
ftp:message,
ftp:transfer
ftp:binding presents, but there is no ftp:transfer or
ftp:message in wsdl:output
FTPBindingValidation.NO_EXTENSIBILITY_ELEMENT_FOUND_IN_OUTPUT
ftp:message,
ftp:transfer
part when part is specified in
ftp:message or ftp:transfer,
the referenced part must
exist in the corresponding
abstract message definition
FTPExt.PART_REFERENCE_INVALID
ftp:message,
ftp:transfer
pollIntervalMillis must be > 0 FTPExt.INVALID_POLL_INTERVAL
ftp:message,
ftp:transfer
actually this rule concerns operation binding element
wsdl:operation, the input and output binding
element(s) or wsdl:operation element
should match the corresponsing abstract
operation's input and output child element(s), e.g., if the
abstract operation has one
input and one output, then the operation binding
should also have one input and one output, if the
abstract operation only
requires one input, then the operation binding should
also have only one input accordingly.
FTPBindingValidation.OP_SIG_MISMATCH_BINDING_ABSTRACT
ftp:message,
ftp:transfer
for a FTP BC operation
binding, if it requires input
and output, then its input
and output binding should
have the same type of FTP
BC extensibility elements.
FTPBindingValidation.INPUT_OUTPUT_REQUIRE_SAME_TYPE_FTPBC_EXT
ftp:transfer when ftp:transfer presents in wsdl:output, it presents at
most once.
FTPBindingValidation.ATMOST_ONE_TRANSFER_IN_OUTPUT
ftp:transfer when ftp:transfer presents in wsdl:input, it presents at
most once.
FTPBindingValidation.ATMOST_ONE_TRANSFER_IN_INPUT
ftp:message when ftp:message presents in wsdl:output, it presents at
most once.
FTPBindingValidation.ATMOST_ONE_TRANSFER_IN_OUTPUT
ftp:message when ftp:message presents in wsdl:input, it presents at
most once.
FTPBindingValidation.ATMOST_ONE_TRANSFER_IN_INPUT
ftp:message messageNamePrefixIB When presents, should not contain percentage sign % FTPMessage.IB_MSG_PREFIX_HAS_PATTERN
ftp:message messageNamePrefixOB When presents, should not contain percentage sign % FTPMessage.OB_MSG_PREFIX_HAS_PATTERN
ftp:operation ftp:operation must present
when ftp:binding presents
FTPBindingValidation.MISSING_FTP_OPERATION
ftp:operation ftp:operation presents but ftp:binding is not FTPBindingValidation.FTP_OPERATION_WO_FTP_BINDING
ftp:operation ftp:operation can present at most once as child element of wsdl:operation FTPBindingValidation.AT_MOST_ONE_FTP_OPERATION
ftp:operation when presents, must reference an abstract operation FTPBindingValidation.OP_ABSTRACT_NOT_FOUND


Runtime Time Validation Rules:
All the design time rules will be checked at runtime also and in addition to that, the proxy information is validated as follows: If useProxy = true, proxyURL must be specified and must be a valid proxy URL in the form of:
{socks4|socks5}://[<user>[:<password>]@]<host>[:<port>]

Runtime validation error messages are similar to those of design time
and they are not listed here for brevity.

6. Usage of FTP BC Extensibility Elements


Please refer to FTP BC wiki page, for video captures, showing how to
use FTP BC extensibility elements in a WSDL, and further implement
and/or invoking the services defined by the WSDL in business
processes (as BPEL scripts).

7. Message Types And Encoding


As described in the extensibility elements, the following attributes
are used for message encoding:

  • ftp:message->use, ftp:transfer->use – the value can be “literal” meaning the
message part is used as is (no encoder associated to encode it)
or “encoded” meaning the part has to be encoded by an encoder
specified by another attribute - “encodingStyle”.
  • ftp:message->part, ftp:transfer->part – a reference (by name) to the message-part
in the abstract message, when not specified the first part is
picked as the payload.
  • ftp:message->encodingStyle, ftp:transfer->encodingStyle – a name associated with an encoder
which will parse the part into a XML model.

8. References


  • JBI Specification 1.0 (JSR 208)
  • FTP BC Clustering Support Design

9. APPENDIX


APPENDIX A: Sample User Heuristic Definition

  1. --------------------------------------------------------------------------
  2. Section: UNIX
  3. --------------------------------------------------------------------------


UNIX!Commands Supported By FTP
Server!value=APPE%CWD%DELE%LIST%MKD%NOOP%PASS%QUIT%RETR%RNFR%RNTO%SITE%STOR%TYPE%USER!set=APPE%CWD%DELE%LIST%MKD%NOOP%PASS%QUIT%RETR%RNFR%
RNTO%SITE%STOR%TYPE%USER
UNIX!Header Lines To Skip!value=1!set=1
UNIX!Header Indication Regex Expression!value=^ *total!set=^ *total
UNIX!Trailer Lines To Skip!value=0!set=0
UNIX!Trailer Indication Regex Expression!value=!set=
UNIX!Directory Indication Regex Expression!value=^ *d!set=^ *d
UNIX!File Link Real Data Available!value=Yes!set=No%Yes
UNIX!File Link Indication Regex Expression!value=^ *l!set=^ *l
UNIX!File Link Symbol Regex Expression!value=[ ]->[ ]!set=[ ]->[ ]
UNIX!List Line Format!value=Blank Delimited!set=Blank Delimited%Fixed
UNIX!Valid File Line Minimum Position!value=9!set=9
UNIX!File Name Is Last Entity!value=Yes!set=No%Yes
UNIX!File Name Position!value=9!set=9
UNIX!File Name Length!value=0!set=0
UNIX!File Extension Position!value=0!set=0
UNIX!File Extension Length!value=0!set=0
UNIX!File Size Verifiable!value=Yes!set=No%Yes
UNIX!File Size Position!value=5!set=5
UNIX!File Size Length!value=0!set=0
UNIX!Special Envelope For Absolute Pathname!value=!set=
UNIX!Listing Directory Yields Absolute Pathnames!value=No!set=No%Yes
UNIX!Absolute Pathname Delimiter Set!value=///!set=///
UNIX!Change Directory Before Listing!value=No!set=No%Yes
UNIX!Directory Name Requires Terminator!value=No!set=No%Yes

APPENDIX B: Validation Error Code

Omitted for brevity, also the validation error messages grows, for the current, check out the Bundle.properties from CVS

JSPWiki v2.4.100
[RSS]
« Home Index Changes Prefs
This page (revision-61) was last changed on 08-Jun-09 11:24 AM, -0700 by jimfu