Size: 5628
Comment:
|
Size: 6409
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 51: | Line 51: |
For example, assuming there are 4 nodes: A(s), B, C, D. s is the opened listen socket (on the specified port). Open 3 client sockets on B, C and D connecting to s. *Assuming A wants to send message to B, C and D. Because A is server socket, so, it sends message to B, C and D directly. *Assuming B wants to send message to A, C and D. From B, we build the message header: fill the unused field (msg_control for instance) with the pointer to array of target nodeID. Then send this message. A receives the message, parses the message header to get the list of targer nodeID and checks: if a nodeID is its own one, it takes the message, otherwise, it lookup to get the client socket sd corresponding to the nodeID and then forward the message to the that node. |
|
Line 54: | Line 57: |
Open issues for both models: how to retrieve nodes' ip addresses so that we can create a hashmap to map nodeID and its ip address or is there any way to retrieve the ip address from the specified nodeID? | Open issues for both models: how to retrieve nodes' ip addresses so that we can create a hashmap to map nodeID and its ip address or is there any way to retrieve the ip address from the specified nodeID and vice versa? |
SAFplus 7 Feature Discussion
SAFplus Messaging
Advanced Socket Layer
The socket abstraction (MsgSocket class) presents a simple read/write API for multiple scatter gather messages, consisting of receive, send, and flush APIs. At the message transport layer, an instance of the socket object directly communicates with the Linux level socket via system calls. However additional socket objects can be created that implement additional functionality, such as traffic shaping, message segmentation and reassembly and reliable messaging. These message sockets can modify the messages sent and received and then call a lower level MsgSocket instance.
Functionality can therefore be selected at socket construction time by instantiation of a stack or tree of MsgSocket-derived classes, and a single implementation of an abstract concept (such as traffic shaping) can be applied to any message transport type.
After socket construction, application code is unaware that it is accessing layered sockets rather than a direct socket implementation, making it easy to change add or remove functionality as application requirements change.
The diagram above shows 3 applications with different socket stacks. Application 1 is using UDP transport, traffic shaping and allows large messages (segmentation and reassembly). Application 2 does not need large messages; it is just using traffic shaping. Application 3 shows a complex configuration. Its node supports both UDP and TIPC transports, so the first layer above the transport ("splitter") uses the destination nodeId to determine which transport to route the message to. Large messages are supported so the "Segmentation And Reassembly" object is next. After that, the application may need both reliable and unreliable messages, so a "Joiner" object is added that is capable of providing 2 independent MsgSocket interfaces and combining them into one (by prefixing a "message type" character to the front of every message, for example). Finally, a "reliable message" MsgSocket object provides reliable messaging. The application code sees 2 MsgSocket interfaces; one for unreliable and one for reliable messages.
Note that is is expected that the receiving application will have the same socket stack, in cases where the MsgSocket objects modify message content.
SCTP transport plugin
SCTP stands for Stream Control Transmission Protocol. SCTP is the extension of TCP, therefore, there are some typical features:
- Data transfer is reliable
- Multi-stream support
- Multi-homing support
- Ordered delivery is not strictly enforced
Because of the data transport in stream, so there is 2 separate methods to open socket at server and client:
- Server:
- open a socket
- set sctp socket options
- bind the opened socket with IP address and port
- listen connections
- accept connections
- receive/send from/to clients
- Client:
- open a socket
- set sctp socket options
- connect to the listening server socket
- send/receive to/from server socket
However, to implement the peer-to-peer model using SCTP, the model has to be refined:
- 1.Model 1: each client would need to support its own listen socket, so that another client (once aware of another client), can connect directly to its peer.
For example, assuming there are 4 nodes: A(s1), B(s2), C(s3), D(s4). s1,s2,s3,s4 are the opened listen sockets (on the specified port) on each node. Assuming A wants to send message to B, C and D, we have to open 3 client sockets (on the same port as servers, of course) that connect to s2,s3,s4 respectively. After that, from A, messages can be sent to s2,s3,s4 as well as received from them.
- 2.Model 2: based on the central server idea:
- open only one server socket on only one node. This socket listens and accepts all the connections; receives messages from other nodes and forwards these messages to the specified target nodes (may be parsed from the message header).
- open one client socket connecting to the listen socket on other nodes and set the target nodes to the message header and then send the message to the listen socket.
For example, assuming there are 4 nodes: A(s), B, C, D. s is the opened listen socket (on the specified port). Open 3 client sockets on B, C and D connecting to s.
- Assuming A wants to send message to B, C and D. Because A is server socket, so, it sends message to B, C and D directly.
- Assuming B wants to send message to A, C and D. From B, we build the message header: fill the unused field (msg_control for instance) with the pointer to array of target nodeID. Then send this message. A receives the message, parses the message header to get the list of targer nodeID and checks: if a nodeID is its own one, it takes the message, otherwise, it lookup to get the client socket sd corresponding to the nodeID and then forward the message to the that node.
This model has some drawbacks:
- Handling the case when the node on which the listen socket is running dies is so complicated (we have to be aware of this; then find and select another node to create another listen socket; all socket clients have to be recreated and reconnected to the new server socket...)
Open issues for both models: how to retrieve nodes' ip addresses so that we can create a hashmap to map nodeID and its ip address or is there any way to retrieve the ip address from the specified nodeID and vice versa?
SAFplus Management
Management Access APIs
We need a c++ API to issue management object set and gets from any application. This will allow the management data to be generically manipulated and so we will not need to support a lot of application specific APIs like log stream creation or Dynamic HA operations.
Applications can also use these APIs to implement a new northbound protocol (like SNMP and NETCONF) if they need to.
Proposal:
mgtSet(const string& pathSpec, const string& value, Transaction& t); string mgtGet(const string& pathSpec); void mgtCreate(const string& pathSpec); void mgtDelete(const string& pathSpec);
All APIs raise an exception if they cannot complete.
For example:
mgtSet("/Log/stream["mystream"]/filename", "/var/log/myapplog")