• General Design Requirements

Object Methods

• All APIs that require a ClusterUniqueId shall also accept a string name. This name shall be looked up in the Name service and resolved to a ClusterUniqueId

• All APIs that modify the object or global state shall accept an optional Transaction parameter. The call shall validate and reserve the state change but not execute until the Transaction is committed.

Synchronous, asynchronous and transactional programming

Most APIs will be synchronous. APIs would block for significant time should be (such as those that send a message and wait for a reply) should be offered as both synchronous and asynchronous flavors. This shall be implemented as follows:

• APIs shall accept an optional parameter as the last in the parameter list. This parameter is an object that can be used in a variety of ways to implement callback, blocking or transactional semantics.
• If this parameter is not supplied, the API is being called synchronously.

Example:

Version HelloExchange(int node, Version myVersion, Wakeable& wake); // Returns other node's version, raises exception on error

Application code can call this function in the following ways:

• Synchronous

   1 HelloExchange(1,"1.0.0");

• In the above case, since "wake" is not specified, the HelloExchange function internally creates a semaphore an blocks. It is essentially shorthand for this equally valid method:

   1 ThreadSemaphore sem;
   2 HelloExchange(1,"1.0.0",sem);
   3 sem.take();  // Blocks until hello exchange "gives" the semaphore
   4 

• Asynchronous

  • Simultaneous Synchronous
    This technique preserves the readability of synchronous code but is actually asynchronous.

       1   ThreadSemaphore sem;
       2   for (int node = 0; node < maxNodes; node++)
       3     {
       4     HelloExchange(1,"1.0.0",sem);
       5     }
       6   sem.take(maxNodes);  // Blocks until the sem is "given" maxNodes times
       7 
    

• In this case, the HelloExchange function is run "maxNodes" times asynchronously and then blocks on the sem.take() function until every HelloExchange call completes.

• Returning values
  Return values can be handled in several ways. First, they can be specified as members of a Wakeable derivation:

     1   class HelloExchangeResult
     2     {
     3       Version version;
     4       ThreadSemaphore& sem;
     5       void give(int amt) { sem.give(amt); }
     6     };
     7 
     8   ThreadSemaphore sem;
     9   HelloExchangeResult result[maxNodes](sem);
    10   for (int node = 0; node < maxNodes; node++)
    11     {
    12     HelloExchange(1,"1.0.0",result[node]);
    13     }
    14   sem.take(maxNodes);  // Blocks until the sem is "given" maxNodes times
    15   for (int node = 0; node < maxNodes; node++)
    16     {
    17     printf("Node %d version %s", node, result[node].version);
    18     }
  

• In this case, a class is defined that contains both the return value and a reference to the "wakeable" object. The HelloExchange function is run "maxNodes" times asynchronously and this thread blocks on the sem.take(). When the response is received, the return value is placed into the passed HelloExchangeResult object and the semaphore is given. When the semaphore is given "maxNodes" times, this thread wakes up and the results are processed.

• Note that this solution does not process the results when they are received. It is possible to do so using a "ThreadCondition" object instead of a ThreadSemaphore since that object can wake the thread each time the semaphore is given.

• Classic Asynchronous
  The classic asynchronous technique uses function callbacks to continue processing. This technique is STRONGLY DISCOURAGED because it is:

  • bug prone
  • difficult to debug because execution context is not carried in a thread but by an object on a list
  • difficult for others to understand because conceptual operations are broken across multiple functions instead of being encapsulated.
  • difficult to maintain.
  However, it may be effective for simple uses or in combination with the Simultaneous Synchronous technique. With this technique, the Wakeable is not a semaphore but a unique class whose "give" member implements the next step in the algorithm.

     1   class HelloExchangeResult:public Wakeable
     2     {
     3       int node;
     4       Version version;
     5       void give(int amt) { printf("Node %d version %s", node, version); }
     6     };
     7 
     8   ...
     9   HelloExchangeResult* result = new [maxNodes];
    10   for (int node = 0; node < maxNodes; node++)
    11     {
    12     result[node].node = node;
    13     HelloExchange(1,"1.0.0",result[node]);
    14     }
    15   return;
    16   
  

• Recipes

  To block with a timeout, ThreadConditions can be used instead of ThreadSemaphores. Another timeout solution is to place the semaphore on a timer queue before calling the API. For multi-process, use ProcessSemaphores.