Solarflare Onload is a sockets acceleration technology for the transparent acceleration of sockets-based applications.
On Linux, the Handler uses Solarflare Onload Extensions API to support additional features of Solarflare network cards. If one uses such network cards, in your system, you can benefit from advanced features of such network cards. CME iLink 3 Handler automatically detects if the corresponding Onload library installed in your system and, if so, then loads it in run-time and provides an ability to use additional features. In this case, there should be the record, in the Handler log, that the corresponding version of the Onload extension library is used.
EF_TCP_SERVER_LOOPBACKparameters should be used, e.g.:
EF_TCP_CLIENT_LOOPBACK=2 EF_TCP_SERVER_LOOPBACK=2 onload ... Please see the "Onload User Guide".
Currently, the Handler supports the following Onload features:
When one uses the OnixS::CME::iLink3::Session::warmUp method, the Handler automatically enables this feature. As a result, one can get the maximal warmup effect, including the complete warmup of the TCP stack's sending path.
onload_stackdump utility could be used to monitor the usage of this feature.
$ onload_stackdump lots | grep warm
Solarflare TCPDirect is an ultra-low latency network stack.
Before TCPDirect sessions can be created, the application must first create a TCPDirect stack (OnixS::CME::iLink3::TcpDirectStack):
Attributes (OnixS::CME::iLink3::TcpDirectAttr) control the configuration of the stack and its behavior, please see the Solarflare TCPDirect User Guide for a complete list of available attributes and their description.
TCPDirect sessions should be created in the same thread where TcpDirectStack has been created. The reference to the stack is provided in Session's constructor, for example:
By default, each TCPDirect stack can handle up to 64 TCP endpoints, so those number of sessions could be created. Maximum number of TCP endpoints can be set by
max_tcp_endpoints attribute. Please refer to Solarflare TCPDirect documentation. See also
max_udp_tx_endpoints attributes. These attributes could be set to a minimum value to save hardware and software resources.
External code on top of TCPDirect can be implemented as an event loop.
The need to scroll through the loop means the inability to use blocking calls because blocking calls will stop the event's dispatching. Therefore, user code must call asynchronous Session's methods to connect and disconnect, for example:
In general, TCPDirect stack requires to finish all outstanding work and to handle all outstanding events. Before destroying the TcpDirectStack instance, the following code should be executed:
TcpDirectStack::isQuiescent() (OnixS::CME::iLink3::TcpDirectStack::isQuiescent) returns a boolean value indicating whether a stack is quiescent.
This can be used to ensure that all connections have been closed gracefully before destroying a stack (or exiting the application). Destroying a stack while it is not quiescent is permitted by the API, but when doing so there is no guarantee that sent data has been acknowledged by the peer or even transmitted, and there is the possibility that peers' connections will be reset.
While TCPDirect technology provides the highest possible efficiency it imposes some limitations on flexibility.
Accesses to the stack instance at each moment of time are possible from only one thread. It must be remembered that this technology does not use inter-threaded synchronization, does not serialize calls, and does not check the correctness of access to its API by the user. Therefore, the event dispatching and message sending should be performed from the same thread.
Such decisions made it possible to implement the technology as efficiently as possible while at the same time increasing the requirements for the correctness of the code using it.
The TCPDirect API does not support loopback connections as opposed to regular sockets.
To facilitate the development and debugging of applications based on TCPDirect OnixS CME iLink3 Handler contains the TCPDirect emulator for Windows on top of regular sockets with call correctness verification.
Unlike the original technology, the TCPDirect emulator allows you to create a loopback connection, which also helps the application debugging process.
TCPDirect emulator does not require additional configuration, it activated automatically under Windows when TcpDirectStack object is created and used.
Native TCPDirect-connections should belong to the corresponding Solarflare hardware interface. In Windows emulation mode it could be any existing network interface that allows binding the sockets to it. In most common debugging scenarios is recommended to use the same interface (and its IP address) both for an exchange emulator (OnixS::CME::iLink3::Testing::Gateway) and TCPDirect-emulated client connection.