Low Latency Best Practices

This topic describes how to configure the Handler to achieve the best performance characteristics and the lowest processing latency.

Disable logging

Logging adds overhead on the processing path, so disabling it is one of the simplest latency wins. See Disabling Logging for the configuration snippet and details.

Tuning Working Threads

Market data processing is done asynchronously by using working threads. Under normal conditions, threads may be executed on any processor available in the system. That may have a negative influence on overall performance due to unnecessary thread context switching.

To avoid switching threads between processors, Feed Engine as manager of working threads, allows establishing processor affinity for each working thread:

    FeedEngineThreadPoolSettings feSettings1;
    FeedEngineThreadPoolSettings feSettings2;
 
    // Scheduling Feed Engine thread pools to use different processors.
    feSettings1.threadAffinity().insert(1);
    feSettings2.threadAffinity().insert(2);

In addition to the ability to manipulate thread affinity for working threads of Feed Engine, it also provides a set of events triggered by working threads at the beginning of processing and before ending processing loop. See Feed Engine Events for more information.

With the help of working threads events, it's possible to perform more advanced thread tuning like updating thread priority:

struct ThreadPriorityManager : FeedEngineThreadPoolListener
{
    void onFeedEngineThreadBegin(const FeedEngineThreadPool&) override
    {
        setPriorityForCurrentThread();
    }
};

    FeedEngineThreadPoolSettings feSettings;
    ThreadPriorityManager priorityManager;
 
    FeedEngineThreadPool fePool(feSettings, &feedEngine, &priorityManager);

Decreasing Time Handler Spends on Waiting for Incoming Data

This setting applies to SocketFeedEngine. When the Handler accomplishes processing of previously received market data, it initiates reception of new incoming data if it's available in the feed. In reality, data packets do not come each after another, but there's a certain time interval between two neighbor packets. Pauses between incoming packets cause the socket feed engine bound to the Handler to suspend activities and sleep in the kernel while waiting for incoming data. As a result, data and execution code can be ejected from the processor's cache. That brings to the fact that the next iteration of processing loop is performed slower in comparison to the previous one. To influence the amount of time SocketFeedEngine waits in the kernel on incoming data, use the dataWaitTime constructor parameter, whose value defines time the socket feed engine spends in I/O operation while waiting for the incoming data. Reducing the value of the noted parameter increases the number of wake-ups and thus reduces the probability for the socket feed engine to be thrown out of processor's cache. If the parameter is zero, the socket feed engine checks for data availability but does not enter kernel for a sleep. The drawback of reducing waiting time is the CPU utilization (up to 100% in the case of zero parameter value).

More Topics

• Solarflare's Ultra-low Latency `ef_vi` API