Solarflare Feed Engine

The default implementation of the Feed Engine is based on the standard socket API. Network card manufacturers offer ultra-low latency alternatives to this API. For example, Solarflare offers the ef_vi API to send and receive multicast packets bypassing the kernel and avoiding copying data from internal buffers to user space.

The SDK includes the implementation of the Feed Engine abstraction based on the Solarflare ef_vi API. The functionality is encapsulated into the OnixS::CME::MDH::SolarflareFeedEngine class. The design of this class is similar to the OnixS::CME::MDH::SocketFeedEngine class. The only difference is Solarflare-specific parameters.

Note

The Solarflare ef_vi API is available only for Solarflare network cards on Linux. To simplify the development process, the SDK exposes the OnixS::CME::MDH::SolarflareFeedEngine and related classes for all platforms. However, the instance creation will fail at runtime on platforms where the corresponding functionality is not supported.

Configuring the Solarflare Feed Engine

The Solarflare ef_vi API uses the concept of virtual interfaces to receive network data. The corresponding configuration parameters are available via the OnixS::CME::MDH::SolarflareFeedEngineSettings class:

• The OnixS::CME::MDH::SolarflareFeedEngineSettings::mode defines the behaviour of the Feed Engine. The Feed Engine will fail to operate on non-Solarflare network interfaces in the conventional mode as the use of the ef_vi services on non-Solarflare network interfaces is not supported by the manufacturer and thus leads to a failure. In contrast, the mixed mode combines ef_vi with the standard socket API, thus making possible multicast packets receiving on both Solarflare and non-Solarflare network interfaces. The mixed mode gives more flexibility for the users whose operating environments are configured in a particular way (for example, when Incremental data is available on a Solarflare network adapter and recovery data comes via a non-Solaflare network interface).
• The OnixS::CME::MDH::SolarflareFeedEngineSettings::receiveRingSize parameter defines the number of buffers in the virtual interface receive ring. Buffers from the receive ring are filled with network packets (a single buffer is used for a single network packet). The feed engine checks which buffers are filled with incoming data and delivers those buffers to the handlers for further processing. The receive ring must be large enough to accommodate incoming data during market data bursts.
• The OnixS::CME::MDH::SolarflareFeedEngineSettings::cacheSize parameter sets the size of the pool of buffers used to receive and manipulate incoming data. When the buffers are put into the receive ring and filled with incoming data, the feed engine delivers the buffers to the handlers for further processing. Thus, the buffers filled with the data are kept busy until the received data is processed by handlers. In case of rapid transmission, buffers may be allocated by the feed engine quicker than they are processed by the handlers and released back to the pool. Therefore, the pool must be large enough to avoid data loss caused by an empty pool and receive ring.
• The OnixS::CME::MDH::SolarflareFeedEngineSettings::socketBufferSize parameter establishes the size of the socket buffer used to store incoming packets before they are retrieved by the Feed Engine. This parameter is used by the Feed Engine operating in the mixed mode when ordinary sockets are used as the way of multicast data receiving on non-Solarflare network interfaces.

Mixed mode

The mixed mode for the Solarflare Feed Engine combines the ef_vi SDK with standard network sockets. This mode allows taking advantage of using the ultra-low latency ef_vi API on Solarflare network adapters while receiving some data from non-Solarflare network interfaces using the socket API.

The following code shows how to configure data receiving when Incremental data comes via the Solarflare network adapter (eth0) and recovery data is available on a non-Solarflare network interface (eth2).

Configuring consists of two steps.

The first step is to tell Solarlfare Feed Engine to work in the mixed mode:

SolarflareFeedEngineSettings feSettings;
 
feSettings.mode(SolarflareFeedEngineOperatingMode::Mixed);
 
SolarflareFeedEngine feedEngine(feSettings);

The second step is to bind Handler's feeds to proper network interfaces:

Handler handler;
 
FeedSettings& feedSettings = handler.settings().feeds();
 
// We need to update all interfaces at once instead of assigning
// values for each feed type (Incremenal, Snapshot, Instrument, etc).
feedSettings.feedANetworkInterfaces("eth2").feedBNetworkInterfaces("eth2");
 
// Now, updates the settings for Incremental feeds only.
feedSettings.incrementalFeeds().feedANetworkInterfaces("eth0").feedBNetworkInterfaces("eth0");