Low Latency Best Practices

Inner Contents
	Benchmarking Results

Detailed Description

The given topic uncovers how to configure the Handler in order to achieve maximal performance characteristics and lowest processing latency.

Configuring Logging Subsystem

Under normal conditions, the Handler logs important events and market data transmitted by MDP into a log file. As far as logging entries represent textual information, binary data like incoming market data packets are encoded using base64-encoding before stored in a log. That adds extra time to a processing cycle. Finally, if the Logger implementation stores its data into a file, that may be a relatively slow operation.

If the users want to eliminate slowdowns caused by flushing data to filesystem and/or extra encoding operations, they can disable logging by binding instance of the OnixS::CME::ConflatedUDP::NullLogger class to the Handler. In such case log events are not constructed by the Handler, and nothing is logged at all.

In-place Execution of the Feed Engine Machinery

The Feed Engine concept was improved, and the thread-management layer was removed from the available implementations. The new OnixS::CME::ConflatedUDP::NetFeedEngine::process member was introduced to run the Feed Engine machinery explicitly. This improvement allows applications to perform their tasks more efficiently. In-place invocation of the OnixS::CME::ConflatedUDP::NetFeedEngine::process member allows avoiding using additional threads. Instead, it lets combining it with the other tasks like sending/receiving orders through an order management system used in combination with market data handling.

Below pseudo-code depicts the primary principle:

void main()
{
    // Initializes the order management system. 

    OrderManagementSystem oms;

    configure(oms);

    // Initializes market data handling services. 

    const ChannelId channel = 310;

    SocketFeedEngine feedEngine(SocketFeedEngine());

    Handler handler;

    configure(handler, channel, feedEngine);

    // Brings OSM and MDH into running states. 

    oms.start();
    handler.start();

    // Dispatch and handle events until end of 
    // service is requested (e.g. Ctrl+C is pressed).

    while (!interrupted())
    {
        oms.process();
        feedEngine.process();
    }

    // Stops both OSM and MDH. 

    handler.stop();
    oms.stop();

    // The end. 
}

Turning Up Working Threads

Suppose the application uses a thread-safe implementation of the Feed Engine and invokes its OnixS::CME::ConflatedUDP::NetFeedEngine::process member across multiple threads. In that case, threads participating in execution of the Feed Engine machinery may need additional turn-up. Under normal circumstances, threads are executed on any processor available in the system. That may have a negative influence on overall performance due to unnecessary thread context switching.

Establishing thread affinity for each of the working threads avoids or minimizes switching between processors. Suppose the application uses the OnixS::CME::ConflatedUDP::FeedEngineThreads class to run the Feed Engine machinery across multiple threads. In that case, affinity can be specified through the settings supplied at the instance construction:

extern SocketFeedEngine feedEngine;

...

FeedEngineThreadSettings threadSettings;

threadSettings.pool().affinity().cpus().insert(1);
threadSettings.pool().affinity().cpus().insert(2);

FeedEngineThreads threads(feedEngine, threadSettings);

In addition to establishing affinity for working threads, the OnixS::CME::ConflatedUDP::FeedEngineThreads also provides a set of events triggered by working threads at the beginning of the master loop and before ending a processing loop. See Multi-threaded Processing for more information.

With the help of working thread events, it is possible to perform more advanced thread turning like updating thread priority:

extern SocketFeedEngine feedEngine;

...

struct 
ThreadPriorityManager 
: FeedEngineThreadListener
{
    void 
    onFeedEngineThreadBegin(
        const FeedEngineThreads&)
    {
        SetThreadPriority(
            GetCurrentThread(), 
            THREAD_PRIORITY_TIME_CRITICAL);
    }
};

ThreadPriorityManager priorityManager;

...

FeedEngineThreadSettings settings;

settings.pool().size(2);

FeedEngineThreads threads(feedEngine, settings, &priorityManager); 

Decreasing Time Handler Spends on Waiting for Incoming Data

When the Handler accomplishes processing the previously received market data, it initiates receiving new data if it is available in the feed. In reality, data packets do not come each after another. Usually, there's a non-zero time interval between two neighbor packets. Pauses between incoming packets may cause a Feed Engine to sleep in a system kernel while waiting for incoming data. As a result, data and an execution code can be ejected from a processor's cache. That brings to the fact that the next iteration of the processing loop is performed slower than the previous one. The OnixS::CME::ConflatedUDP::SocketFeedEngine provides the OnixS::CME::ConflatedUDP::SocketFeedEngineSettings::dataWaitTime parameter, which defines the time the Feed Engine spends inside an I/O operation (like select) while waiting for incoming data. Reducing the parameter's value increases the number of wake-ups and reduces the probability for the Feed Engine's data and code to be thrown out of a processor's cache. If the parameter is set to zero, the Feed Engine checks for data availability only but does not enter a system kernel to sleep. The drawback of reducing waiting time is a CPU consumption increase (up to 100% for zero parameter value).

Note

It depends on an implementation of the Feed Engine, whether sleeping in a system kernel while waiting for incoming data may take place. For certain implementations, like the OnixS::CME::ConflatedUDP::SolarflareFeedEngine one, that is not applicable.

Suppressing Market Data Copying

Under normal conditions, the Handler effectively utilizes internal structures used to keep incoming market data. Packets and messages are re-used once the Handler processes the contained data. Therefore, no data is allocated during real-time market data processing.

However, data may be copied within callbacks, which the Handler invokes as listeners for various market data events. Thus, when a book is copied, that assumes memory allocation and thus harms performance and latency. Copying should be minimized. Alternatively, strategies with preallocation should be used to improve the results.

For example, book snapshots can be constructed of a particular capacity capable of storing a particular number of price levels. Constructing book snapshots with an initial capacity, sufficient to hold books of maximal possible depth eliminates reallocations.