Dictionary (AKA Checkpoint)
The dictionary entity forms the backbone of the coordination of information between nodes in the cluster. This entity is fully replicated to all nodes that are interested in it -- it is not a "distributed" dictionary where every node has partial data.
Major Features
Most major features can be selected at object creation time to optimize for speed or for utility
Replicated: Replicated efficiently to multiple nodes
Nested: Dictionary values can be unique identifiers that resolve to another Dictionary
Persistent: Dictionaries can automatically store themselves to disk (Persistence)
Notifiable: You can subscribe to get notified of changes to dictionaries (Event)
Shared memory: For efficiency, a single service per node can maintain coherency (with the rest of the cluster) of a dictionary. To do this the dictionary is stored in shared memory
Transactional: Dictionary operations can be wrapped in a Transaction
Design
In this document the term "Dictionary" will be used to refer to the entire replicated Dictionary abstraction. The term "local replica" will be used to refer to a particular copy of the dictionary.
Process Access
A Dictionary can be located in process private memory or in shared memory based on an option when the Dictionary is created.
A Dictionary that is used by multiple processes on the same node should be located in shared memory for efficiency.
Dictionary Creation
A dictionary is always identified by a Handle. At the API layer, a string name can be used. If the latter, this name will be registered with the Name service using the dictionary's Handle as the value.
Internode Replication and Communication
Discovery
The implementation of the dictionary will register a new group with the Group service. It is identified by a well-known Handle or by the ClusterUniqueId returned by the Group registration. It may also be identified by a string entry in the Name service and all APIs that need a Dictionary will accept either a ClusterUniqueId or a string name.
The group service shall be used to identify the process responsible for updating the local replica on the node -- the "node primary replica" -- and the process that can write to the dictionary -- the "master replica".