Index Changes

Throttling feature in HTTP/SOAP BC

What Is Throttling?

Throttling was added to the HTTP/SAOP BC, as part of the QoS systemic quality initiative, in order to provide on-the-wire quality of message delivery. Increased load and in particular, large message payloads, can cause memory usage spikes which results in significant peformance degradation. Throttling is used to set the maximum number of concurrent messages that can be processed by a particular endpoint in order to limit resource consumption so that adequate, constant performance can be achieved.

Extending Grizzly to Handle Request Throttling

The HTTP/SOAP BC uses the Grizzly/JAX-WS stack. Grizzly handles the HTTP protocol and thus the only way to achieve a viable throttling solution was to extend Grizzly to handle how requests are read off the wire and possibly suspend requests when deemed necessary. Requests that are suspended are re-evaulated at a later time to determine whether the target endpoints are ready to handle the requests; if so, then the requests are resumed for processing as usual. To extend Grizzly, its Pipeline implementation, com.sun.enterprise.web.connector.grizzly.LinkedListPipeline, was extended and the addTask(Task) method was overriden. The extended class is called LinkedListThrottlePipeline, in the package com.sun.jbi.httpsoapbc.embedded. In the overriden addTask(Task) method, the following strategy is taken:
  1. Read only the first line of the HTTP request and parse the line for the request URI - the key here is to read as little as possible to prevent memory consumption as much as possible
  2. Use the request URI to locate the HTTP/SOAP BC's internal representation of an enpoint (resource)
  3. Determine whether the endpoint can service the request
    1. If the endpoint's maximum concurrency limit is reached, cache the SelectionKey along with the ByteBuffer use to read the request
    2. If the endpoint can service the request, continue processing the request as normal

The throttling implementation, used by the HTTP/SOAP BC, relies on the HTTP/SOAP BC to notify the LinkedListThrottlePipeline to resume any suspended requests. When an endpoint completes the processing of any request (successfully sending back an HTTP response for the HTTP request), the endpoint will call LinkedListThrottlePipeline.resumeSuspendedRequests(..) to fire off a thread which will resume as many suspended requests as the endpoint could currently handle. The whole source file for LinkedListThrottlePipeline can be viewed here - LinkedListThrottlePipeline.

Work is underway to incorporate LinkedListThrottlePipeline into Grizzly such that it can be extensible in allowing custom throttling policies to be implemented. The goal is to have this functionality ready in the next Grizzly's release (or hopefully a patch) so that the HTTP/SOAP BC make use of it in the future release. Sources for this can be viewed here -

Configuring HTTP/SOAP BC Endpoint for Throttling

Throttling is configured using the CASA editor.
Open up the CASA editor.
Click on a connection to configure QoS.
In the Properties sheet for the connection, you can set the number for the maximumConcurrencyLimit under ThrottlingExtension,

What the Composite Applciation project system generates
The Composite Application project system will generate the appropriate throttling configuration for the connection in the jbi.xml for the service assembly when it is built. The following is the snippet of the jbi.xml with throttling configured. Notice that the connections element containing the throttling grandchild element is in the "" namespace as opposed to the default JBI namespace, ""),
      <consumer endpoint-name="PassThruPort" service-name="ns1:PassThruService"/>
      <provider endpoint-name="PassThruPortTypeRole_myRole" service-name="ns2:PartnerLink1"/>
<connections xmlns="">
      <consumer endpoint-name="PassThruPort" service-name="ns1:PassThruService"/>
      <provider endpoint-name="PassThruPortTypeRole_myRole" service-name="ns2:PartnerLink1"/>
      <throttling xmlns="" maximumConcurrencyLimit="2"/>

The attribute, maximumConcurrencyLimit, of the throttling element is used to configure the maximum number of concurrent messages that can be processed on this connection. From the HTTP/SOAP BC's perspective, this number will used to set up the maximum number of concurrent messages that the PassThruPort endpoint (its own internal endpoint) will send to the the "PassThruPortTypeRole_myRole" endpoint (the provider endpoint). The link between the "PassThruPort" endpoint to the request URI is in the matching WSDL port,

<service name="PassThruService">
   <port name="PassThruPort" binding="tns:PassThruBinding">
       <soap:address location="http://localhost:18181/PassThruService/PassThruPort"/>

In this example, the request URI is "/PassThruService/PassThruPort". The HTTP/SOAP BC pieces all these information together and use it when applying throttling policy on each of the endpoints.

JSPWiki v2.4.100
« Home Index Changes Prefs
This page (revision-9) was last changed on 07-Mar-08 14:18 PM, -0800 by jamestran