Overview
The SAFplus Management system provides an integrated NETCONF, SNMP, CLI (local and remote), local API, and web access platform whose data model presents on these northbound interfaces as a single management data configuration and statistics tree. Yet data can be stored and requests serviced by many separate applications running on many nodes in a cluster. Incoming requests from multiple protocol handlers are automatically translated into a standard high performance message request format and routed to the application that is currently servicing that portion of the data tree. Data routing is dynamic. Applications can "bind" to portions of the tree at any time, allowing data servicing to be handed from one application to another and for crashing applications to be seamlessly restarted or failed over.
This guide describes how to add SAFplus Management into a cluster. It assumes that cluster applications are already management-enabled. To learn how to enable your applications for management access, please refer to the SAFplus Management Developer's Guide.
This document was prepared using the Ubuntu 14.04 Linux distribution. Your distribution may be different.
Installation
This section describes how to install SAFplus Management from packages provided by OpenClovis.
You can also "install" from source code by acquiring the source code via tarball or our repository and then building it. In that case, the binaries will be located in a different place ...[source directory]/target/[machine architecture]/, so all paths specified in this document must be converted to this location.
First add our key to your package manager
wget -q -O - http://download.openclovis.com/files/debian/OpenClovis.asc | sudo apt-key add -
Next, add our repository
sudo add-apt-repository "deb http://download.openclovis.com/files/debian $(lsb_release -s -c) contrib"
Update the list of software packages in the repository
sudo apt-get update
Finally, install SAFplus Management
sudo apt-get install safplus-mgt
This installation has placed SAFplus Management binaries, libraries and helper files into the directory /opt/safplus/<version>/mgt.
Although the SAFplus CLI can be installed on any machine, and it typically not installed on the network element, it is useful to have a copy on the local development machine for easy access. To install it, run:
sudo apt-get install safplus-cli
Running
This section describes how to run SAFplus Management either on the development or on a target machine.
Configure SSH Access
If you have installed safplus_mgt from a .deb or .rpm package, it will be installed to /opt/safplus/<version>/mgt. If you are building SAFplus Mgt from source, on the development machine binaries and other files are generated in the "target base directory" (.../target/[architecture]/) subdirectory. They can be run directly from this location. Or to run on another machine, first move the target base directory and all subdirectories to any directory in the target machine using any technique (scp, archive, etc). But for the purposes of this document, we will assume that it has been installed to the package location (/opt/safplus/<version>/mgt).
The first step is to enable NETCONF access through ssh. We have created a simple script to accomplish this in Ubuntu 14.04:
sudo /opt/safplus/<version>/mgt/mgt_install
If this script will not work for you, you simply need to change your /etc/ssh/sshd_config configuration file and add:
Port 830
Subsystem netconf /opt/safplus/<version>/mgt/bin/netconf-subsystemThen restart ssh via:
service ssh restart
We also find that it is sometimes necessary to explicitly kill the sshd process to get it to access the changed configuration file.
To verify that this worked, let's see what process id is listening to port 830 and then make sure that process is "sshd":
# sudo netstat -tulpn | grep :830 tcp 0 0 0.0.0.0:830 0.0.0.0:* LISTEN 10931/sshd tcp6 0 0 :::830 :::* LISTEN 10931/sshd # ps 10931 PID TTY STAT TIME COMMAND 10931 ? Ss 0:00 /usr/sbin/sshd -D
For CentOS, you have disabled the firewalld by command:
# iptables -F
Or configured SELinux to accept listening on port 830:
# service ip6tables stop # service iptables stop # iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 830 -j ACCEPT # service iptables save # service iptables start # service ip6tables start
Then restart sshd:
# service sshd restart
Configuration Files
SAFplus7 installs with reasonable configuration for testing and development so you can skip this section if desired.
Configuration for the NETCONF protocol occurs just like in the OpenYuma project (via the netconfd.conf file). Please see the OpenYuma documentation for more information and look at .../examples/eval/websvc/netconfd.conf for an example. By default, the netconfd.conf file should be located at /opt/safplus/<version>/mgt/share/netconf/netconfd.conf, although this location can be overridden via command line arguments.
Initial values for "native" (not using the SAFplus back-end) OpenYuma YANG modules are set via the startup-cfg.xml file. This is simply an XML file that specifies data formatted as defined by YANG modules. In particular, NETCONF access control happens here, since it is controlled by the "nacm" module (ietf-netconf-acm.yang). For more information, see the OpenYuma project and documentation relating to the particular YANG module that specifies the data structure. For use as a template, a simple startup-cfg.xml file that turns on access control is included here:
<?xml version="1.0" encoding="UTF-8"?>
<config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<nacm xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-acm">
<enable-nacm>true</enable-nacm>
</nacm>
</config>But this example is only useful as a template because access control rules must be defined. An full example of setting up the nacm (access control) module is located in .../examples/eval/websvc/startup-cfg.xml.
This file should be located in /opt/safplus/<version>/mgt/share/netconf/, or in a location specified by the netconfd.conf file.
Environment Variables
An Up-to-date script to set up the management runtime environment can be found in the examples/eval/websvc/setup directory. This section provides an overview of the process.
First, it is important to set up your LD_LIBRARY_PATH and PYTHONPATH variables to point to the location of binaries and libraries:
cd /opt/safplus/7.0/mgt # Or your installation director export LD_LIBRARY_PATH=`pwd`/lib:`pwd`/plugin:$LD_LIBRARY_PATH export PYTHONPATH=`pwd`/lib:`pwd`/bin:`pwd`/test:$PYTHONPATH
Next, set up the SAFplus logging and communications variables. Of course you will need to change your interface to one appropriate for your machine:
# Logging controls appropriate for development export CL_LOG_SEVERITY=DEBUG export CL_LOG_CODE_LOCATION_ENABLE=1 # Pick a location for the log files mkdir -p ~/safplusLog export SAFPLUS_LOG_DIR=~/safplusLog # Pick the interface that you are using to communicate between nodes in your cluster. export SAFPLUS_BACKPLANE_INTERFACE=eth0
If you are just communicating to local processes, you might use the loopback interface instead of eth0 as shown above. But to do so with the UDP transport you must also enable broadcasts over the loopback:
export SAFPLUS_BACKPLANE_INTERFACE=lo # To use UDP broadcasts on the loopback interface I need to enable a broadcast address sudo ifconfig lo broadcast 127.255.255.255
Finally, if you were running a SAFplus NETCONF enabled application, you would need to point the server to your .yang definition files:
export SAFPLUS_MODULES_PATH=<directory1>:<directory2>
However we are not running any applications so just point this to nothing, so the server does not complain:
export SAFPLUS_MODULES_PATH=
If you are running this management application within the SAFplus HA Platform, you will need to configure SAFplus Platform appropriately (setting ASP_NODENAME to a node specified in your HA database). See the SAFplus Platform documentation for more details.
For network intensive applications, you might also want to configure Linux to provide more resources to networking:
sysctl -w net.core.wmem_max=20485760 sysctl -w net.core.rmem_max=20485760 sysctl -w net.core.rmem_default=20485760 sysctl -w net.core.wmem_default=20485760
Execution
SAFplus Management can be run within the SAFplus Availability/Scalability Platform or in a standalone configuration. To run within the SAFplus Platform, the "safplus_mgt" and "netsnmpd" binaries should be added to your cluster's model inside a 1+1 redundancy group. For more information about this see the SAFplus Platform documentation. Like any SAFplus application, note that it is also possible to run the application within the SAFplus cluster but started manually simply by running the applications on the command line or in the debugger when safplus_amf is already running.
To run SAFplus Management without high availability, simply run the binary when safplus_amf is not running. It will detect that it is in standalone mode and handle some of the platform services that safplus_amf normally provides.
Set up paths and environment variables (as described in #Environment Variables) and then start the safplus_mgt and netsnmpd binaries. In this case we'll use provided scripts:
cd /opt/safplus/7.0/mgt ./mgt_start
Testing
At this point, you should be able to access the system via NETCONF over SSH. A quick connectivity test is to manually SSH into the machine like this:
ssh -s -p 830 localhost netconf
You will see the server's "hello" message:
<?xml version="1.0" encoding="UTF-8"?> <hello xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"> <capabilities> ...
Now, you can use any NETCONF client that can handle NETCONF over SSH. For example, to test access via the SAFplus CLI, run:
safplus_cli
Then within the CLI run:
connect <any_name_you_choose_to_identify_this_node> <user_name> <password> <machine's IP>
For example, if your system is configured to allow root SSH logins you could do:
connect localAdmin root <password> localhost
Next, "cd" to that machine and display the YANG data:
cd localAdmin ls
You should see something like:
/networkElement/localAdmin:
localAdmin (localhost)
arp
nacmThe only YANG modules shown are those that are handled "natively" by safplus_mgt. In particular, the "nacm" (NETCONF Access Control Module is available. If you are running safplus_mgt under SAFplus High Availability, you will also see the "safplusAmf" module.
Congratulations! You have successfully installed SAFplus Management.
Next Steps
Writing your first management-enabled program
Let's create a real application that can be managed. Please see the SAFplus Management Developer's Guide for details.
Using the SAFplus CLI
The SAFplus CLI is much more powerful than show in in this simple example. Please see the SAFplus Management CLI Guide for details.
Configuration
Access Configuration (NETCONF NACM module)
Troubleshooting
NETCONF clients connect and then disconnect instantly
You have either not enabled NETCONF in sshd_config, or the binding program does not exist. Add this to /etc/ssh/sshd_config and restart:
Port 830 Subsystem netconf <replace with your path to SAFplus>/bin/netconf-subsystem
Assertion `!"Cannot load dbal plugin"' failed.
You have not selected a database layer. copy libclSQLiteDB.so (or other layer) to /opt/safplus/7.0/mgt/plugin
Assertion `!"Required env variable [SAFPLUS_BACKPLANE_INTERFACE] not defined"' failed.
You have not selected a network interface to communicate with management enabled applications. To access local apps only, try:
export SAFPLUS_BACKPLANE_INTERFACE=lo ifconfig lo broadcast 127.255.255.255
Assertion `!"Cannot load message transport plugin"' failed.
You have not selected a network transport to communicate with management enabled applications. Copy clMsgUdp.so to /opt/safplus/7.0/mgt/plugin
My YANG modules are not visible in the CLI but 'standard' (e.g. nacm) modules are visible
Modules that use the SAFplus back end are handled specially. They must be copied to the directory specified by the SAFPLUS_MODULES_PATH environment variable. See Environment Variables
Error: open logfile '/opt/safplus/7.0/mgt/bin/netconfd.log' failed (cannot open file)
You do not have permission to create the log file in the directory you specified.
NET-SNMP Configuration
Tue Mar 15 09:04:59.582 2016 [mgtMain.cxx:222] (node0.5310.5323 : MGT.MGT.SNMP:00059 : INFO) netsnmp_get_table_handler(NULL) called Tue Mar 15 09:04:59.582 2016 [mgtMain.cxx:222] (node0.5310.5323 : MGT.MGT.SNMP:00060 : ERROR) netsnmp_inject_handler() called illegally Tue Mar 15 09:04:59.582 2016 [mgtMain.cxx:222] (node0.5310.5323 : MGT.MGT.SNMP:00061 : ERROR) netsnmp_assert handler != ((void *)0) failed agent_handler.c:418 netsnmp_inject_handler_before()
You may have different net-snmp version with this package. To solve this, please try to install net-snmp version 5.7.3 on this host.