Pluggable Session Storage

Pluggable session storage is the storage which implemented by the user. For example, it could be used to implement High-Availability (HA) Solutions. One can implement a Pluggable Session Storage, which stores data to shareable storage. If one server with the FIX Engine fails and another reserve server stands up, the FIX Engine can restore session states from the same shareable storage. Such storage should implement the OnixS::FIX::ISessionStorage interface.

Implementing Pluggable Storage

There are few basic rules that should be applied to the implementation of Pluggable Storage.

• The user should manage the lifecycle of the storage

  The Engine core treats the instance of Pluggable Storage as a "foreign" object and, therefore, never attempts to delete it. On the other hand, there is a strong requirement to keep the storage instance valid until the OnixS::FIX::Session instance is destroyed.

• Be as fast as possible

  Since Pluggable Storage is incorporated into the message send/receive route, it directly affects sending/receiving latency and overall throughput of the system. So, to keep high performance, it is needed to handle callbacks as fast as possible.

• Prevent against exceptions

  Exceptions are thrown from Pluggable Storage callbacks, always immediately break a session. So you usually need to use an exception-free code for callbacks.

• Don't make assumptions about parameters lifecycle

  Messages and pointers passed to Pluggable Storage callbacks should be completely handled before these callbacks return control to Engine core. Whenever you need to keep such data for separate usage, you should make copies of messages or memory blocks and store them in an appropriate place, but don't keep raw pointers, passed to callbacks.

• No specific synchronization is needed

  The Engine core manages all synchronization issues by itself, so all Pluggable Storage methods are called sequentially. Therefore, you don't need to implement separate synchronization to ensure that ISessionStorage::logInbound and ISessionStorage::logOutbound methods are not called simultaneously. Of course, it is still possible to use synchronizations for other needs.

• Track message numbers when the message is logged

  Message sequence numbers should be tracked while messages are logged, i.e. during OnixS::FIX::ISessionStorage::storeInbound or OnixS::FIX::ISessionStorage::storeOutbound methods are handled. These numbers should be kept and returned to Engine through OnixS::FIX::ISessionStorage::inSeqNum() and OnixS::FIX::ISessionStorage::outSeqNum() calls for incoming and outgoing messages respectively.

  OnixS::FIX::ISessionStorage::inSeqNum(int) and OnixS::FIX::ISessionStorage::outSeqNum(int) are called only during the session startup, initial login, and session shutdown of the processes.

Call Order of Pluggable Storage Methods

This section explains how different methods of Pluggable Storage are called in different situations.

Incoming message handling

• Reading a message from wire. At this stage, verification of the message content and the message classification are performed.
• When there is a session-level message
  • OnixS::FIX::ISessionStorage::storeInbound.
  • OnixS::FIX::ISessionListener methods, if the listener is present.
• When there is an application-level message, call order to listeners methods is reversed against the previous case
  • OnixS::FIX::ISessionListener methods, if the listener is present.
  • OnixS::FIX::ISessionStorage::storeInbound.

If an exception is thrown by the user's code on any stage (during ISessionStorage or ISessionListener handling), the session becomes disconnected, and subsequent stages are omitted. Logout message will be sent to the counterparty in dependency on the value of the OnixS::FIX::EngineSettings::sendLogoutOnException setting.

Outgoing message handling

• OnixS::FIX::ISessionListener methods. If the method throws an exception, the handling is stopped.
• OnixS::FIX::ISessionStorage::storeOutbound. If the method throws an exception, the handling is stopped.
• Passing message to the internal queue for subsequent sending.

Note

If a message is sent from the OnixS::FIX::ISessionListener::onInboundApplicationMsg() callback, the session checks it and calls the OnixS::FIX::ISessionStorage::storeInbound() first before calling the OnixS::FIX::ISessionStorage::storeOutbound(). The session does it to keep the logging order. Otherwise, the OnixS::FIX::ISessionStorage::storeOutbound() will be called before the OnixS::FIX::ISessionStorage::storeInbound(), and the outbound message will appear in the log before the inbound message.

Resend Request handling

• OnixS::FIX::ISessionStorage::storeInbound with incoming ResendRequest<2> message.
• OnixS::FIX::ISessionStorage::getOutbound method to restore all or a part of the requested messages. These messages are kept in an internal buffer until the OnixS::FIX::ISessionStorage::getOutbound method returns control to Engine.
• The following loop is repeated for each restored message:
  • OnixS::FIX::ISessionListener::onOutboundApplicationMsg or OnixS::FIX::ISessionListener::onOutboundSessionMsg.
  • OnixS::FIX::ISessionStorage::storeOutbound method.

Attaching Pluggable Storage

There is a quite schematic illustration for the pluggable storage usage.

    // Declare the own class and implement ISessionStorage interface callbacks.
    class MyPluggableStorage : public OnixS::FIX::ISessionStorage
    {
    public:

        void clear() ONIXS_FIXENGINE_OVERRIDE {}

        void close(bool /*keepSequenceNumbers*/, bool /*doBackup*/) ONIXS_FIXENGINE_OVERRIDE {}

        void getOutbound(SequenceNumber /*beginSequenceNumber*/, SequenceNumber /*endSequenceNumber*/, ISessionStorageListener * /*listener*/) ONIXS_FIXENGINE_OVERRIDE {}

        SequenceNumber inSeqNum() ONIXS_FIXENGINE_OVERRIDE { return 0; }

        void inSeqNum(SequenceNumber /*messageSequenceNumber*/) ONIXS_FIXENGINE_OVERRIDE {}

        void storeInbound(const Message &, SequenceNumber /*messageSequenceNumber*/, const RawMessagePointer & /*pointer*/, bool /*logMessage*/) ONIXS_FIXENGINE_OVERRIDE {}

        void storeInbound(const FlatMessage &, SequenceNumber /*messageSequenceNumber*/, const RawMessagePointer& /*pointer*/, bool /*logMessage*/) ONIXS_FIXENGINE_OVERRIDE {}

        void storeOutbound(const Message &, const RawMessagePointer & /*pointer*/, bool /*logMessage*/) ONIXS_FIXENGINE_OVERRIDE {}

        void storeOutbound(const FlatMessage &, SequenceNumber /*sequenceNumber*/, const RawMessagePointer & /*pointer*/, bool /*logMessage*/) ONIXS_FIXENGINE_OVERRIDE {}

        void setSessionTerminationFlag(bool /*terminated*/) ONIXS_FIXENGINE_OVERRIDE {}

        SequenceNumber outSeqNum() ONIXS_FIXENGINE_OVERRIDE { return 0; }

        void outSeqNum(SequenceNumber /*messageSequenceNumber*/) ONIXS_FIXENGINE_OVERRIDE {}

        void sessionCreationTime(Timestamp) ONIXS_FIXENGINE_OVERRIDE {}

        Timestamp sessionCreationTime() ONIXS_FIXENGINE_OVERRIDE { return Timestamp(); }

        void flush() ONIXS_FIXENGINE_OVERRIDE {}

        size_t resendingQueueSize() const ONIXS_FIXENGINE_OVERRIDE { return 1000; }

        void resendingQueueSize(size_t) ONIXS_FIXENGINE_OVERRIDE {}
    };

    // Declare the instance of the storage.
    MyPluggableStorage myStorage;

    // Create the session which uses the storage.
    Session initiator("SenderCompId", "TargetCompId", ProtocolVersion::FIX_42, &listener, SessionStorageType::Pluggable, &myStorage);

Example

Please, check the Pluggable Storage Sample included in the distribution package of the FIX Engine.