SAFplus

Configuration

SAFplus component and application configuration can be used independently or together with the SAFplus Management add-on product. With the Management product, NETCONF and SNMP and the SAFplus CLI (Command Line Interface) "northbound" interfaces are available to access and modify configuration data.

Overview

Component and application configuration is accomplished by defining a schema that defines the required configuration data. This schema is defined in the YANG file format described in RFC6020. YANG is a simple data definition language with syntax familiar to C or Java programmers yet can be translated without loss of information into an XML format (called YIN). Definitions are organized into "modules" which can contain notifications (asynchronous messages from the server to the client), remote procedure calls, and a complex structured data feature-similar to SNMP MIB files, XML, or any other data definition language.

Status, and statistics use this same infrastructure; the only difference is that these data are not stored persistently (to disk).

Using the SAFplus Management add-on product, YANG files can be compiled into a C++ class structure. These classes represent the configuration, status, statistics, notifications and functions that were defined by the YANG information model. Application programmers can extend these classes by adding additional objects or creating derived classes to capture other related data.

These C++ objects are automatically connected to and underlying persistent database and to the northbound protocols when the application "binds" the root object of the configuration tree to a location within the YANG defined information tree. During this "bind" initialization, child C++ objects are automatically created if they exist in the persistent database.

Configuration can be defined by XML files is loaded and saved to the persistent database by OpenClovis tools. Offline, configuration can be manipulated by Python (or C) scripts and then exported as XML files or pushed into the persistent database.

This architecture allows applications to access configuration via an intuitive object-oriented interface and avoids the need for every application to handle details of northbound interfaces, configuration/XML file parsing, CLI interfaces, and configuration database access.

SAFplus Cluster Modelling Tool Integration

OpenClovis YANG file extensions can be used to specify configuration information to be associated with instances of application components in the SAFplus IDE. The SAFplus IDE will automatically create data entry dialogs for the specified information and output the proper XML configuration files during code generation.

A "leaf" (data item) is IDE enabled with the "safplus:ide" extension for example: 

   1       leaf timeout
   2       {
   3         type uint64;
   4         description "The maximum time this operation should take before completion. Specified in milliseconds.";
   5         safplus:ide "Maximum Time: ";
   6       }

This leaf would create a text entry box in the configuration dialog labelled "Maximum Time: ". Mouse-over help would display the description.

YANG Schema Extensions

SAFplus defines certain extensions to the basic YANG modelling language that help translate YANG to C++ and display configuration in the IDE and CLI. They are defined in the SAFplusTypes.yang file and can be included in your yang file like this: 

   1   import SAFplusTypes { prefix "safplus"; }

Overrides the generated type. For example: 

   1      leaf-list serviceUnits
   2      {
   3        safplus:c-type "SAFplusAmf::ServiceUnit*";
   4        type instance-identifier;
   5        description "This component is the proxy for the components listed here.";
   6      }

A YANG instance-identifier is normally modelled as a string which contains the name-and-path of the instance. In the example above, the generated type will be overridden to a "SAFplusAmf::ServiceUnit*" pointer: 

   1 ClMgtProvList<SAFplusAmf::ServiceUnit*> serviceUnits;

Indicates that this type already exists "natively" in c++ so nothing should be generated. The argument is the name of the type. Example: 

   1     typedef SaTimeT
   2     {
   3       type uint64;
   4       c-existing "SaTimeT";
   5       description "A representation of a time interval, specified in milliseconds";
   6     }

Supplies the prompt for the GUI or CLI when requesting this field. Example: 

   1       leaf duration
   2       {
   3         type safplus:SaTimeT;
   4         description "The time period (in milliseconds) involved.";
   5         safplus:ui-prompt "within time: ";
   6       }

The prompt will be: within time: rather than: duration:

Places this object in a visual or logical grouping for better organization. 

   1      container serviceUnitFailureEscalationPolicy
   2       {
   3         uses EscalationPolicy;
   4         description "The maximum Service Unit failure rate allowed before this node is faulted";
   5         safplus:ui-group "Service Unit Failure Escalation Policy";
   6       }

This command triggers a sub-window (not necessarily a modal dialog box) that is created for these options. If the value is 'class <name>' or 'object <name>' this indicates that this subwindow should be instantiated as the custom code denoted by <name>. All other values are the title of the sub-window.

This indicates the minimum serialization granularity which has an effect on database reads/writes. The effect of the 'atomic' field is to cause all siblings and children to be written to the database as a single unit. This can dramatically reduce database size and improve database access times for those fields that are generally loaded or saved together. Database size is especially reduced for leaf-list objects. For example: 

   1   container AllTogether
   2   {
   3     safplus:atomic;
   4     leaf stringTest
   5       {
   6       type string;
   7       }
   8 
   9     leaf-list int64Test
  10     {
  11       type int64;
  12     }
  13 }

All the objects in "AllTogether" will be serialized as a single unit, including the entire int64Test array. 

   1   list AList
   2   {
   3     key "name";
   4     atomic;
   5 
   6     leaf name
   7     {
   8       type string;
   9     }
  10     leaf data
  11     {
  12       type int64;
  13     }
  14   }

Each element in the AList array is serialized separately, but fields within the element (name and data) are stored together. 

   1 container atomicList
   2 {
   3   atomic;
   4   list AList
   5   {
   6     key "name";
   7     leaf name
   8     {
   9       type string;
  10     }
  11     leaf data
  12     {
  13       type int64;
  14     }
  15   }
  16 }

This entire list is serialized atomically.

XML And Python

Configuration can be loaded and saved in XML format through a Python script.

Convert XML to database:

python dbalpy.py -x <fileNoExt>

This command converts <fileNoExt>.xml into the configuration database format.

Object Serialization

There are two object serialization formats, text (XML) and binary (database and RPC calls).

Internals

Object serialization/deserialization must handle references (pointers). Therefore it must occur within the context of an object namespace.

Binary serialization shall be called marshallization (marshal, demarshal), Text serialization shall be called XMLizaton (XMLize, deXMLize).

All marshal operations occur via 4 overloaded APIs: 

   1   inline void demarshal(const std::string& input,ClMgtObject* context, <type>& result) {...}
   2 
   3   inline marshal(const <type>& input,ClMgtObject* context, const MarshalBuffer& output) {...}
   4 
   5   inline void deXMLize(const std::string& input,ClMgtObject* context, <type>& result) {...}
   6 
   7   inline XMLize(const <type>& input,ClMgtObject* context, const MarshalBuffer& output) {...}

Note, we need to find or create an efficient extensible buffer type "MarshalBuffer". This type should allow you to specify its size at creation time, for cases where the marshaled size can be pre-calculated. And it should be automatically and efficiently extensible for cases where its size cannot be pre-calculated. The act of "flattening" is converting a complex buffer (a linked list of small buffers, for example) into a single char* allocation of contiguous memory.

Ideally, MarshalBuffer could translate directly into a scatter-gather list in IOC and DB so the buffer never needs to be flattened.

"Freed" MarshalBuffers should be stored in linked lists of different sizes rather than actually calling "free()". This is both much more efficient and will make memory leak obvious.

Individual buffer chunks should have beginning and end guard bytes to detect applications overwriting the buffer.

SAFplus: Configuration (last edited 2014-05-02 21:20:04 by AndrewStone)