Solarflare Onload is a sockets acceleration technology for the transparent acceleration of sockets-based applications.
On Linux, the FIX Engine uses Solarflare Onload Extensions API to support additional features of Solarflare network cards. If you use such network cards, in your system, you can get an additional benefit from advanced features of such network cards.
FIX Engine automatically detects if the corresponding Onload library installed in your system and if so, then loads it in run-time and provides an ability to use additional features. In this case, there should be the record, in the FIX Engine log, that the corresponding version of the Onload extension library is used.
EF_TCP_CLIENT_LOOPBACK
and EF_TCP_SERVER_LOOPBACK
parameters should be used, e.g. : EF_TCP_CLIENT_LOOPBACK=1 EF_TCP_SERVER_LOOPBACK=1 onload ..
Please see the "Onload User Guide".
Currently, the FIX Engine supports the following Onload features:
When you use the OnixS::FIX::Session::warmUp method, the FIX Engine automatically enables this feature if possible. As a result, you can get the maximal warmup effect including the complete warm-up of the sending path of the TCP stack.
The onload_stackdump
utility could be used to monitor the usage of this feature.
For example:
onload_stackdump lots | grep warm
In order to use this feature you need to use the overloaded OnixS::FIX::Session::send(FlatMessage * msg, SessionSendMode::Enum mode) method with the OnixS::FIX::SessionSendMode::OnloadZeroCopy mode. As a result, you can avoid the additional data copying from an application allocated buffer to the network adapter buffer and can decrease the sending latency.
This feature also is supported when one is sending messages in a batch. In this case, you need to use the overloaded OnixS::FIX::Session::send(FlatMessageBatch & msgs, SessionSendMode::Enum mode, size_t maxPacketSize) method.
Solarflare's TCPDirect is a highly accelerated network middleware. It is similar to Onload but provides lower latency.
Before a TCPDirect-enabled session is created, the application must create an OnixS::FIX::TCPDirect::Stack instance.
Attributes (OnixS::FIX::TCPDirect::Attributes) control the configuration of the stack and its behavior. Please see the "Solarflare TCPDirect User Guide" for a complete list of available attributes and their description.
The pointer to the stack is provided in Session's constructor, for example:
By default, each TCPDirect stack can handle up to 64
TCP endpoints, so this is the maximum number of sessions that could be created using the same stack instance.
The max_tcp_endpoints
attribute can set the maximum number of TCP endpoints. Please refer to Solarflare TCPDirect documentation.
See also max_tcp_listen_endpoints
, max_tcp_syn_backlog
, max_udp_rx_endpoints
, max_udp_tx_endpoints
attributes.
These attributes could be set to a minimum value to save hardware and software resources.
cat /sys/kernel/mm/transparent_hugepage/enabled
Applications must call asynchronous Session's methods to logon (OnixS::FIX::Session::logonAsInitiatorAsync) and logout (OnixS::FIX::Session::logoutAsync).
Applications must call OnixS::FIX::TCPDirect::Stack::dispatchEvents frequently for each stack that is in use.
The TCPDirect stack requires finishing all outstanding work and handling all outstanding events. Before destroying the OnixS::FIX::TCPDirect::Stack instance, the following code should be executed:
Method OnixS::FIX::TCPDirect::Stack::isQuiescent returns a boolean value indicating whether a stack is quiescent.
This can be used to ensure that all connections have been closed gracefully before destroying a stack (or exiting the application). Destroying a stack while it is not quiescent is permitted by the API, but when doing so there is no guarantee that sent data has been acknowledged by the peer or even transmitted, and there is the possibility that peers' connections will be reset.
There is an ability to use the TCPDirect mode for acceptor sessions too. However, to dispatch listening events and accept incoming TCP connections, one must pass the pointer to the OnixS::FIX::TCPDirect::Stack to the OnixS::FIX::Engine::init() method. The pointer to the same stack should also be passed to the acceptor session's constructor. In this case, the FIX Engine does not create an additional thread to listen for incoming connections and uses the corresponding stack to dispatch these events: