SNMPBC Design Notes

Modules

The SNMP BC consists of two parts: the SNMP engine (called SNMPRA) and the BC itself. The BC acts like a JBI wrapper around the engine. The engine can be tested separately from the BC. The engine defines a few interfaces that the BC needs to implement. The engine communicates with the BC through these interfaces. As such, the BC depends on the engine, but the engine does not depend on the BC.

Fault management in SNMPRA (the engine)

The engine is started by the BC. The BC sets up the engine with the port number the engine will be listening on, and a callback handler for the metadata SE and processing SE-s. Traps can be received by the engine at a rate of 10000-20000 traps per second, a rate that greatly exceeds the processing speed of the processing engines. Hence, the engine needs to be able to buffer a great number of traps before passing them on to the processing engines so that bursts of traps can be handled properly. Passing on the traps is done more efficiently if the traps are grouped in batches.

When a trap is received, it needs to be determined to which processing engine it needs to be sent. In case of v2 traps, the traps may also need to be decrypted and authenticated. To get this meta data, an asynchronous call is made to the Metadata SE. Next to batching and buffering, caching is important to increase performance. Hence, the engine keeps a cache of all metadata that it has used. TBD: cache flushing strategy.

Key to buffering and batching is the BatchQueue class. Newly received traps are kept in a batch queue. A processing thread will go over the traps in a batch. This thread will check each trap to see if its metadata is in the cache. If it is not, a request is placed in another batch queue. The batchid of that batch, is the requestid. The trap is placed in a holding area. The holding area is associated with the requestid so that when the response comes in from the metadata SE, the traps in the assoiated holding area can be processed.

For a trap for which there is metadata, the trap is placed in one or more batchqueues: one for each processing engine that has expressed interest in the trap through the metadata.

The BC will send the batch of traps to a processing engine. It's important that the batches are sent at the same rate that the processing engine can handle them to avoid accumulation of traps in the NMR. Hence, the sending of traps to processing engines needs to be throttled.

The amount of traps that can be buffered is limited by memory. Care must be taken to conserve memory. Key to this is the realization that a trap is received as a byte. This byte takes the minimum amount of memory required to store a trap. Converting the trap in a Java object, e.g. the JAXB representation, multiplies the memory required to store a trap by an order of magnitude.

Because of the nature of the UDP protocol, there's no guarantee that traps are received in exactly the same order as they were sent by the agent. However, it's important to roughly maintain the order of the traps throughout processing as it is easy for a newly received trap to bypass older traps that have been waiting due to a cache miss. Implementation guarantees TBD.

SNMPBC WSDL Extension

Fault Management

SNMP BC keeps an internal mapping of adaptationIds to Adaptation ServiceEndpoint.  This mapping is specified in the WSDL file of the Adaptation CompositeApp, by the "adaptationId" attribute in the WSDL.  Whenever an Adaptation CompositeApp is deployed to SNMP BC, BC will add this adaptionId to its internal mapping.

The steps for below diagram are:

1. SNMP BC got a trap
2. SNMP BC queries MetaData Service Engine to find out adaptationId of Adaptation Service that can process the trap, based on incoming trap's properties, such as host, port, etc.
   
3. MetaData Service Engine returns the appropriate adaptationId to SNMP BC, SNMP BC will cache this result, and uses the adaptationId to figure out the ServiceEndpoint of Adaptation Service
4. SNMP BC creates normalized messsage and send to this ServiceEndpoint using NMR
   

Performance Management

Processing Engine can also poll data from SNMP Agent through SNMP BC.

The steps for below diagram are:

1.  Processing Engine sends poll request msg to SNMP BC
2. SNMP BC uses the appropriate SNMP manager instance to invoke operation on the SNMP Agent OID asynchroneously
   
3. SNMP Agent returns result to SNMP BC
4. SNMP BC forwards result to Processing Engine
   

Class Diagram

Class Diagram of classes of interest for both SNMP engine and jbi adaptor. 

This page (revision-6) was last changed on 27-Jun-07 22:27 PM, -0700 by FrankKieviet.
This page was created on 27-Jun-07 18:25 PM, -0700 by Edward Chou.

More info...