Introduction

Dynamic Services Composition is a prototyping/research project aiming at proving a light-weight visual web-based tool to support Project Fuji and specifically the Integration Flow Language.

• Prototype demo movie: http://ksorokin.ru/fuji-dsc/dsc-e2e-demo.swf
• Prototype live version: http://ksorokin.ru/fuji-dsc/r16/samples/d.s.c./index.html (NB: use Firefox only)

Terminology

• Service
  an entity deployed to a service engine
• Adapter
  an entity deployed to a binding component
• Component
  a common name for services and adapters
• Link
  a connection or route between two components
• Message
  an XML document (hence a tree-like structure), which is used as the container for information being trasferred between components; a message, while being XML, can contain non-XML native payloads

Functionality Requirements

Authoring

DSC should allow users to visually author message flows between various components available in the underlying target Fuji runtime. Ideally, for the simplest use-cases, this should not require any manual configuration of the components or writing code. At the same time, while keeping simple things simple, the tool should not forbid to dwell deeper and manually correct even the smallest configuration details of each component.

The very process of authoring should consist of dragging and dropping components on the canvas, connecting them with links and, optionally, configuring (or writing code for business logic) both the components and the connecting links.

There should be no "deploy" step in the flow creation process. At the same time as the component is dropped on the canvas, it should be deployed and go live, as the configuration data is updated in the tool, the live version of the component should be updated as well.

If a component cannot be deployed due to misconfiguration, the user should be properly notified and the component should be marked with a warning badge, indicating that it is not live. Once the configuration is corrected, the component should be auto-deployed.

Creating New Components and Sharing

DSC should be able to obtain information about the existing service engines and binding components from the runtime and present them to the user as options to create a new component. Properties of this "template", when dropped to the canvas, will be populated with default values.

By default the newly created components (and those "copied" from existing ones) are available to the user who created them. The tool, however, should provide an option to easily "share" the component, so that it becomes available (read-only) to other users of the tool.

At the same time the user should be able to "share" the whole message flow, or any connected fragment ("snippet") of it, provided that it is capable of receiving or sending messages or both. A "self-contained" message flow snippet, which reads from an out-only component and ends up in an in-only one cannot be shared as it does not provide any value to other users (cannot be re-used).

Reusing Existing Components and "Flow Snippets"

DSC should be able to obtain information about the already-deployed components from the runtime and present them to the user in an accessible manner. The user should be able to search through the components and drag and drop them to the canvas in order to reuse in his current message flow. In this case the properties of the re-used components should be read-only. The user also should be able to easily create an editable copy from the re-used component in order to be able to easily modify its properties so that they suit the needs of a particular message flow.

In addition to being able to reuse individual components, the user should be able to re-use the "flow snippets" which are shared by the other users.

The search function should be provided in two flavors -- a quick inline (probably keyword/tag-based) search and a more robust variant, which will be separately accessible from a modal window.

Tracing, Debugging and Profiling

The tool should provide some simple functions to trace and debug the message flow being edited. Namely there should be a way to see all the messages which are in queue to be processed by a component, which are in-transport, i.e. are being transferred from a component to a component.

If a given component type supports debugging the user should be able to see the message that is currently processed by the component and to monitor how it changes.

The user should be able to pause the execution of the message flow at any time and examine the flow configuration (message states, component states). He then should be able to select a component and:

• if it supports debugging be able to step through the processing of the current message;
• resume the message flow execution till the next incoming message starts being processed;
• resume the message flow execution till the component finishes processing a message (that is equivalent to an out-only component generating a message);

The user should be able to monitor the time it takes for a message to be processed by a certain component, the time a message is in-transit and if the component supports profiling, the time/memory measurements appropriate for a given component type.

Monitoring and Dashboard

The tool should provide the user with basic analytical information, such as:

• the number of messages processed by a given component
• average time per message
• average time a message needs to wait in queue to be processed by a component
• average time it takes a message to pass a selected fragment of the flow

This information should be available in the form of customizable dashboard and/or as a table.

Testing

The user should be able to create test messages which would be supplied to the components in order to test their functionality. The approach here should be similar to that of the composite application testing approach. The user should be able to select a message flow snippet (or the whole message flow) and define the input message that will be sent to the incoming component (or components, if there is more than one entry-point to the snippet). It also should be possible to define the golden values for the message value at any "control point" in the flow: "before" a component and right "after" the component.

The user should be able to execute an individual test or any subset of the whole test suite, monitor their execution and see the results: number of passes, number of failures, failures details.

Versioning

As the user edits the work flow, each significant (yet TBD) change should be stored in a version control system, namely Subversion. The actual stored artifacts should represent a fully generated maven-based project, which can afterwards be checked out from a third-party tool or an IDE and be edited.

The possible conflicts should be resolved via user interaction. If the tool tries to save (commit) a change and there is a conflict, an appropriate dialog should be shown to the user, asking him to resolve the conflicts.

If the stored version was updated, but there are no conflicts the user should be notified of the change and the flow should be updated from the source control repository, or, if the user wishes to do that, the version be reverted to that of the user's working copy.

The behavior in case, when a user edits the maven project from another application, then deploys without committing, while another user is still editing the flow in the web-interface is undefined. Whoever manages to deploy last will get the prize.

Additionally at some point in time the user, might want to "fix" the current version of the message flow, this will result in a branch being created in the version control system.

Integration with Other Tools

Since the "sources" of the message flow are stored in a version control system (which, at the same time is publicly available (that should be optional)) and, more importantly in a standard maven project-based format, it is absolutely possible to let other tools to check out the sources, and work with them.

NetBeans

This is pretty straight-forward. The user running NetBeans should be able to check-out the sources for a message flow (based on its identifier or codename) make the necessary changes and the commit back to the repository. Note that there is no automatic deployment of the components in this case. The user needs to manually deploy, or open the project in the web-based tool in order to deploy the project.

Miscellaneous

Access Control

Since the tool is planned as a multi-user environment, some basic access control and authentication mechanism should be provided. This will be used for filtering out the "available" components lists, and, optionally for sending out alerts and notifications, using the user-supplied contact information.

Social Aspects

Since the available components form a kind of "functionality cloud" it might make a lot of sense to allow some social network stuff, such as commenting on others' message flow snippets and rating them.

Auditing

Since, in general, the tool will allow any user create any type of service (e.g. a JRuby script) the engine (and the tool) should provide capabilities to track the resources usage quotas, e.g. processor time, memory footprint, disk usage footprint. Appropriate actions should be taken in case the user exceeds the quotas, such as disabling the user-owned components, sending out notifications, interrupting the too-long execution of a component.

User Interface

Technology

As the user interface is intended to be run within a browser container, the technology that is to be used is mostly obvious: (X)HTML + Javascript. There will be some deviations, however, as the <canvas> tag will be used to draw the nodes of the message flow diagram.

With regard to Javascript frameworks used, the UI will be based on Prototype and Scriptaculous frameworks.

AJAX calls will be used to keep the client-side and server-side models in sync.

UI Elements

The UI elements hierarchy of the message flow editor:

• Canvas (diagram)
  • Node
    • "Main graphic"
    • Toolbar
      • Icons
    • Icon
    • Badges
  • Link
    • "Main graphic"
    • Toolbar (???)
      • Icons
    • Badges (???)
• Toolbar
  • Icons
  • Login fields (replaced by Welcome message and Logout link upon logging in)
  • Inline settings
    • Auto-save
    • Auto-deploy
• Floating dockable windows
  • Palette
    • Title (drag handle)
    • Sections
      • Title
      • Search bar
      • Components
        • Icon
        • Title
    • Resize panel
  • Tracer/Debugger
    • Title (drag handle)
    • Tabs
      • Messages Queue
        • Messages list
          • Message tree
            • Tree handle
            • Icon
            • Title (name)
            • Value
      • Statistics
        • ???
      • ???
    • Resize panel
  • Pop-up windows
    • Component Properties
      • ???
    • Message Value Editor
      • ???
    • General Preferences
      • ???
    • Registration Wizard
      • ???

Dependencies

Runtime

The number of dependencies on the Fuji runtime is quite heavy. Below is the list of functionality, that the runtime should provide (at least to some extent).

• Ability to store additional meta-data about deployed components (display name, description, keywords, type). Possibly this should be external to the runtime itself, we need to elaborate on how to keep that in sync.
• API to access the list of deployed components with their states and meta-data
• API to (re-; un-)deploy a component, preferably a given list of components. This could be done via the tool's creating a maven project on the fly, building it and deploying with standard means, but a more dynamic thing would be very welcome. Ideally the tool would just supply the type of the component, its meta-data and properties.
• Debugger-related APIs:
  • Pause the message flow (+ see open questions)
  • Get the list of messages at each control point (before/after a component)
  • Change the value of a given message (based on an ID?)

Other teams

• Documentation -- apparently we would ned someone to review the labels and to write some guide.
• HIE -- someone needs to _professionally_ review the UI from the usability standpoint, also we need some support from the (HIE-controlled? ;) graphics designer

Links

• Top Down Approach Input - proposals and feedback that are being taken into account in driving the dynamic service composition.

Open Questions

• How to present external entities, e.g. Google Maps. People definitely know them as services, but from the Fuji point of view, they are definitely HTTP/SOAP or HTTP/REST adapters?
• What should we do with "enterprise-grade" functionality, such as transactions, fault handling, etc?
• How do we implement a message flow, which, e.g. starts off from an incoming SOAP message and then produces a response to the caller?
• Should we differentiate between a shared component, i.e. a stand-alone service or an adapter, and a shared "flow snippet"?
• How do we handle the situation when the same service is used in two different message flows, and one of them is being debugged (e.g. is set on pause) by a user? What should happen to the second flow, apparently it should stay functional, but how?

This page (revision-28) was last changed on 06-Jun-08 06:42 AM, -0700 by KirillSorokin.
This page was created on 16-May-08 03:48 AM, -0700 by KirillSorokin.

More info...