Dictionary (AKA Checkpoint)

The dictionary entity forms the backbone of the coordination of information between nodes in the cluster. This entity is fully replicated to all nodes that are interested in it -- it is not a "distributed" dictionary where every node has partial data.

Major Features

Most major features can be selected at object creation time to optimize for speed or for utility

• Replicated: Replicated efficiently to multiple nodes

• Nested: Dictionary values can be unique identifiers that resolve to another Dictionary

• Persistent: Dictionaries can automatically store themselves to disk (Persistence)

• Notifiable: You can subscribe to get notified of changes to dictionaries (Event)

• Shared memory: For efficiency, a single service per node can maintain coherency (with the rest of the cluster) of a dictionary. To do this the dictionary is stored in shared memory

• Transactional: Dictionary operations can be wrapped in a Transaction

Design

In this document the term "Dictionary" will be used to refer to the entire replicated Dictionary abstraction. The term "local replica" will be used to refer to a particular copy of the dictionary.

Process Access

A Dictionary can be located in process private memory or in shared memory based on an option when the Dictionary is created.

A Dictionary that is cluster or node-wide should be located in shared memory for efficiency.

Internode Replication and Communication

Discovery

A dictionary will be registered with the Group service. It is identified by a well-known ClusterUniqueId or by the ClusterUniqueId returned by the Group registration. It may also be identified by a string entry in the Name service and all APIs that need a Dictionary will accept either a ClusterUniqueId or a string name.

The group service shall also be used to identify the process responsible for updating the local replica on the node -- the "node primary replica" -- and the process that can write to the dictionary -- the "master replica".

Replication