Pluggable Session Storage

Pluggable session storage is the storage which implemented by the user. For example, it could be used to implement High-Availability (HA) Solutions. One can implement a Pluggable Session Storage, which stores data to shareable storage. If one server with the FIX Engine fails and another reserve server stands up, the FIX Engine can restore session states from the same shareable storage. Such storage should implement the OnixS::FIX::ISessionStorage interface.

The custom implementation must define its persistence, flush, failure, and restart behavior. OnixS::FIX::Session::flushSessionStorage calls OnixS::FIX::ISessionStorage::flush, but does not pass its syncData argument to the custom implementation. If the storage requires device synchronization, define whether OnixS::FIX::ISessionStorage::flush performs it. Compare these requirements in Selecting Storage for Session Messages Logging and apply the implementation's recovery contract alongside the procedures in Restoring Session State from Log Files.

Implementing Pluggable Storage

There are few basic rules that should be applied to the implementation of Pluggable Storage.

• The user should manage the lifecycle of the storage

  The Engine core treats the instance of Pluggable Storage as a "foreign" object and, therefore, never attempts to delete it. On the other hand, there is a strong requirement to keep the storage instance valid until the OnixS::FIX::Session instance is destroyed.

• Be as fast as possible

  Since Pluggable Storage is incorporated into the message send/receive route, it directly affects sending/receiving latency and overall throughput of the system. So, to keep high performance, it is needed to handle callbacks as fast as possible.

• Prevent against exceptions

  Exceptions are thrown from Pluggable Storage callbacks, always immediately break a session. So you usually need to use an exception-free code for callbacks.

• Don't make assumptions about parameters lifecycle

  Messages and pointers passed to Pluggable Storage callbacks should be completely handled before these callbacks return control to Engine core. Whenever you need to keep such data for separate usage, you should make copies of messages or memory blocks and store them in an appropriate place, but don't keep raw pointers, passed to callbacks.

• No specific synchronization is needed

  The Engine core manages all synchronization issues by itself, so all Pluggable Storage methods are called sequentially. Therefore, you don't need to implement separate synchronization to ensure that OnixS::FIX::ISessionStorage::storeInbound and OnixS::FIX::ISessionStorage::storeOutbound methods are not called simultaneously. Of course, it is still possible to use synchronizations for other needs.

• Track message numbers when the message is logged

  Message sequence numbers should be tracked while messages are logged, i.e. during OnixS::FIX::ISessionStorage::storeInbound or OnixS::FIX::ISessionStorage::storeOutbound methods are handled. These numbers should be kept and returned to Engine through OnixS::FIX::ISessionStorage::inSeqNum() and OnixS::FIX::ISessionStorage::outSeqNum() calls for incoming and outgoing messages respectively.

  OnixS::FIX::ISessionStorage::inSeqNum(int) and OnixS::FIX::ISessionStorage::outSeqNum(int) are called only during the session startup, initial login, and session shutdown of the processes.

Call Order of Pluggable Storage Methods

This section explains how different methods of Pluggable Storage are called in different situations.

Incoming message handling

• Reading a message from wire. At this stage, verification of the message content and the message classification are performed.
• When there is a session-level message
  • OnixS::FIX::ISessionStorage::storeInbound.
  • OnixS::FIX::ISessionListener methods, if the listener is present.
• When there is an application-level message, call order to listeners methods is reversed against the previous case
  • OnixS::FIX::ISessionListener methods, if the listener is present.
  • OnixS::FIX::ISessionStorage::storeInbound.

Note

The message object passed to the OnixS::FIX::ISessionStorage::storeInbound may be invalid (created by the default constructor) if a malformed FIX message was received. Callers must check OnixS::FIX::FieldSet::valid() before using the message object. This applies to the OnixS::FIX::Message class only.

If an exception is thrown by the user's code on any stage (during ISessionStorage or ISessionListener handling), the session becomes disconnected, and subsequent stages are omitted. Logout message will be sent to the counterparty in dependency on the value of the OnixS::FIX::EngineSettings::sendLogoutOnException setting.

Outgoing message handling

• OnixS::FIX::ISessionListener methods. If the method throws an exception, the handling is stopped.
• OnixS::FIX::ISessionStorage::storeOutbound. If the method throws an exception, the handling is stopped. Passing message to the internal queue for subsequent sending.

Note

If a message is sent from the OnixS::FIX::ISessionListener::onInboundApplicationMsg() callback, the session checks it and calls the OnixS::FIX::ISessionStorage::storeInbound() first before calling the OnixS::FIX::ISessionStorage::storeOutbound(). The session does it to keep the logging order. Otherwise, the OnixS::FIX::ISessionStorage::storeOutbound() will be called before the OnixS::FIX::ISessionStorage::storeInbound(), and the outbound message will appear in the log before the inbound message.

Resend Request handling

• OnixS::FIX::ISessionStorage::storeInbound with incoming ResendRequest<2> message.
• OnixS::FIX::ISessionStorage::getOutbound method to restore all or a part of the requested messages. These messages are kept in an internal buffer until the OnixS::FIX::ISessionStorage::getOutbound method returns control to Engine.
• The following loop is repeated for each restored message:
  • OnixS::FIX::ISessionListener::onOutboundApplicationMsg or OnixS::FIX::ISessionListener::onOutboundSessionMsg.
  • OnixS::FIX::ISessionStorage::storeOutbound method.

Attaching Pluggable Storage

There is a quite schematic illustration for the pluggable storage usage.

// Declare the own class and implement ISessionStorage interface callbacks.
class MyPluggableStorage : public OnixS::FIX::ISessionStorage
{
public:
 
    void clear() override {}
 
    void close(bool /*keepSequenceNumbers*/, bool /*doBackup*/) override {}
 
    void getOutbound(SequenceNumber /*beginSequenceNumber*/, SequenceNumber /*endSequenceNumber*/, ISessionStorageListener * /*listener*/) override {}
 
    SequenceNumber inSeqNum() override { return 0; }
 
    void inSeqNum(SequenceNumber /*messageSequenceNumber*/) override {}
 
    void storeInbound(const Message &, SequenceNumber /*messageSequenceNumber*/, const RawMessagePointer & /*pointer*/, bool /*logMessage*/) override {}
 
    void storeInbound(const FlatMessage &, SequenceNumber /*messageSequenceNumber*/, const RawMessagePointer& /*pointer*/, bool /*logMessage*/) override {}
 
    void storeOutbound(const Message &, const RawMessagePointer & /*pointer*/, bool /*logMessage*/) override {}
 
    void storeOutbound(const FlatMessage &, SequenceNumber /*sequenceNumber*/, const RawMessagePointer & /*pointer*/, bool /*logMessage*/) override {}
 
    void setSessionTerminationFlag(bool /*terminated*/) override {}
 
    SequenceNumber outSeqNum() override { return 0; }
 
    void outSeqNum(SequenceNumber /*messageSequenceNumber*/) override {}
 
    void sessionCreationTime(Timestamp) override {}
 
    Timestamp sessionCreationTime() override { return Timestamp(); }
 
    void flush() override {}
 
    size_t resendingQueueSize() const override { return 1000; }
 
    void resendingQueueSize(size_t) override {}
};
 
// Declare the instance of the storage.
MyPluggableStorage myStorage;
 
// Create the session which uses the storage.
Session initiator("SenderCompId", "TargetCompId", ProtocolVersion::FIX_42, &listener, SessionStorageType::Pluggable, &myStorage);

Example

Please, check the Pluggable Storage Sample included in the distribution package of the FIX Engine.