By default, feed engines assign timestamps to the incoming multicast packets as soon as they are received. In such a way, an application can measure the latency of market data processing by the Handler.

However, the different implementations of the feed engine abstraction receive incoming packets differently, and thus, the way incoming data is timestamped depends on the implementation of the feed engine. Therefore, although the Handler processes market data in the same way, the results of latency measuring rely on a feed engine implementation. Also, the given approach of assigning timestamps to the incoming data does not reflect the time data spends in the network card and the kernel before it's retrieved by the application and gets processed.

To provide customers with better latency analysis capabilities, the SDK was enhanced with the new feature, which allows getting timestamps assigned by the network adapter or, more straightforward, hardware timestamps. The hardware timestamps allow getting an understanding of how much time data spends in the kernel/adapter before it's get received and processed. It also lets to evaluate the benefits of using kernel-bypass solutions like Solarflare ef_vi and Openonload ones.

Warning: The support of the hardware timestamps by the multi-threaded feed engine, which is the default implementation of the feed engine abstraction using generic socket API, is limited to the Linux platforms only.

Enable Hardware Timestamping

For both the socket-based and ef_vi-based feed engines, hardware timestamps are enabled by assigning a particular instance of the watch service – OnixS::CME::ConflatedUDP::NicWatch.

The following code snippet depicts how to enable hardware timestamps for the OnixS::CME::ConflatedUDP::MultithreadedFeedEngine:

MultithreadedFeedEngineSettings settings;
settings.net().watch(NicWatch::service());
MultithreadedFeedEngine feedEngine(settings);

The code below enables hardware timestamps for the OnixS::CME::ConflatedUDP::SolarflareFeedEngine:

SolarflareFeedEngineSettings settings;
settings.vi().watch(NicWatch::service());
SolarflareFeedEngine feedEngine(settings);

Special Notes on Hardware Timestamping

The hardware timestamps must also be enabled for the network interface. Otherwise, a feed engine will raise a warning or report an error depending on the implementation.

Enabling hardware timestamps on a NIC affects all applications and requires additional (root) privileges.

Also, a network adapter may require clock synchronization manipulations before using the hardware timestamping feature.

Benchmarking the Handler with the Solarflare NICs

The Benchmarking sample from the distribution packet was updated, and the new parameter (--hardwareTimestamps) was added to enable hardware timestamping for the chosen feed engine.

Before running the sample, it's necessary to enable hardware timestamps for a network interface and to synchronize both adapter and system clocks.

The best way to get both preparation steps done is to run the Solarflare PTP daemon. Please, follow the Solarflare documentation on how to configure and run the daemon.

Once the Solarflare PTP daemon is started, the Benchmarking sample can be used to measure latency.

Note: The Solarflare PTP daemon requires some time to get an adapter and system clock synchronized. Therefore, please, wait until the daemon reports clocks are synced before benchmarking market data processing.

The socket-based feed engine (OnixS::CME::ConflatedUDP::MultithreadedFeedEngine) can also be used in combination with the Solarflare Openonload tools. The EF_RX_TIMESTAMPING environment variable must be set to 1 before benchmarking under the Solarflare Openonload. The given environment variable activates hardware timestamping for the Solarflare Openonload tools. The hardware timestamping activation using ioctl function has no effect when the Solarflare Openonload is used.