• General Design Requirements

Object Methods

• All APIs that require a SAFplus::Handle ClusterUniqueId shall also accept a string name. This name shall be looked up in the Name service and resolved to a SAFplus::Handle.

• All APIs that modify the object or global state shall accept an optional Transaction parameter. The call shall validate and reserve the state change but not execute until the Transaction is committed.

• All APIs that block shall provide accept a Wakeable object to offer both threaded and callback semantics as described in detail in the subsequent section.

Synchronous, asynchronous and transactional programming

Most APIs will be synchronous. APIs that would block for significant time should be (such as those that send a message and wait for a reply) offered as both synchronous and asynchronous flavors. This shall be implemented as follows:

• APIs shall accept an optional parameter as the last in the parameter list. This parameter is an object that can be used in a variety of ways to implement callback, blocking or transactional semantics.
• If this parameter is not supplied or is NULL, the API is being called synchronously.

Example:

Version HelloExchange(int node, Version myVersion, Wakeable& wake); // Returns other node's version, raises exception on error

Application code can call this function in the following ways:

• Synchronous

   1 HelloExchange(1,"1.0.0");

• In the above case, since "wake" is not specified, the HelloExchange function internally creates a semaphore an blocks. It is essentially shorthand for this equally valid method:

   1 ThreadSemaphore sem;
   2 HelloExchange(1,"1.0.0",sem);
   3 sem.take();  // Blocks until hello exchange "gives" the semaphore
   4 

• Asynchronous

  • Simultaneous Synchronous
    This technique preserves the readability of synchronous code but is actually asynchronous.

       1   ThreadSemaphore sem;
       2   for (int node = 0; node < maxNodes; node++)
       3     {
       4     HelloExchange(1,"1.0.0",sem);
       5     }
       6   sem.take(maxNodes);  // Blocks until the sem is "given" maxNodes times
       7 
    

• In this case, the HelloExchange function is run "maxNodes" times asynchronously and then blocks on the sem.take() function until every HelloExchange call completes.

• Returning values
  Return values can be handled in several ways. First, they can be specified as members of a Wakeable derivation:

     1   class HelloExchangeResult
     2     {
     3       Version version;
     4       ThreadSemaphore& sem;
     5       void give(int amt) { sem.give(amt); }
     6     };
     7 
     8   ThreadSemaphore sem;
     9   HelloExchangeResult result[maxNodes](sem);
    10   for (int node = 0; node < maxNodes; node++)
    11     {
    12     HelloExchange(1,"1.0.0",result[node]);
    13     }
    14   sem.take(maxNodes);  // Blocks until the sem is "given" maxNodes times
    15   for (int node = 0; node < maxNodes; node++)
    16     {
    17     printf("Node %d version %s", node, result[node].version);
    18     }
  

• In this case, a class is defined that contains both the return value and a reference to the "wakeable" object. The HelloExchange function is run "maxNodes" times asynchronously and this thread blocks on the sem.take(). When the response is received, the return value is placed into the passed HelloExchangeResult object and the semaphore is given. When the semaphore is given "maxNodes" times, this thread wakes up and the results are processed.

• Note that this solution does not process the results when they are received. It is possible to do so using a "ThreadCondition" object instead of a ThreadSemaphore since that object can wake the thread each time the semaphore is given.

• Classic Asynchronous
  The classic asynchronous technique uses function callbacks to continue processing. This technique is STRONGLY DISCOURAGED because it is:

  • bug prone
  • difficult to debug because execution context is not carried in a thread but by an object on a list
  • difficult for others to understand because conceptual operations are broken across multiple functions instead of being encapsulated.
  • difficult to maintain.
  However, it may be effective for simple uses or in combination with the Simultaneous Synchronous technique. With this technique, the Wakeable is not a semaphore but a unique class whose "give" member implements the next step in the algorithm.

     1   class HelloExchangeResult:public Wakeable
     2     {
     3       int node;
     4       Version version;
     5       void give(int amt) { printf("Node %d version %s", node, version); }
     6     };
     7 
     8   ...
     9   HelloExchangeResult* result = new [maxNodes];
    10   for (int node = 0; node < maxNodes; node++)
    11     {
    12     result[node].node = node;
    13     HelloExchange(1,"1.0.0",result[node]);
    14     }
    15   return;
    16   
  

• Recipes

  To block with a timeout, ThreadConditions can be used instead of ThreadSemaphores. Another timeout solution is to place the semaphore on a timer queue before calling the API. For multi-process, use ProcessSemaphores.

Configuration

All SAFplus components manage their configuration via the SAFplus::Mgt objects and backing database. SAFplus will convert XML configuration files from and to the database (automatically on startup if the DB does not exist) so no direct use of XML is necessary.

Statistics

Statistics are reported through the same SAFplus::Mgt objects (but statistics are not stored to the Database)

Fault

If a component takes a fault, it may assert. If a component detects a fault, timeout or other error in another component, it must report that error to the Fault Manager component and then retry. It may not assume that the other component is faulted, failed or dead until it receives a notification from the Fault Manager announcing the component's failure.

The Fault Manager is authoritative:

If a component is working fine and receives an announcement reporting its own failure, it must quit. If a component thinks that another component is working but receives a fault manager notification of its death, it must behave as though the component has failed.