Low Latency Best Practices

The given topic explains how to configure the Handler to achieve optimal performance and lowest possible processing latency.

Configuring Logging Subsystem

Under normal conditions, the Handler logs important events and market data from the iMpact Multicast Price Feed to a log file. Since the log file uses a text-based format, binary data such as incoming market data is base64-encoded before being stored in the log. That adds extra time to the processing cycle. Finally, logging is file-based and thus is relatively slow.

To eliminate slowdowns caused by flushing data to the filesystem and/or extra encoding operations, the logging subsystem should be disabled by setting HandlerSettings::logLevel parameter to LogLevels::Fatal value and HandlerSettings::advancedLogOptions parameter to AdvancedLogOptions::LogNothing.

Turning Up Working Threads

Market data processing is performed asynchronously by the Feed Engine's worker threads. Under normal conditions, those threads may be executed on any processor available in the system. That may negatively affect overall performance due to unnecessary thread context switching.

To avoid switching threads between processors, the FeedEngineSettings::threadAffinity parameter establishes processor affinity for the Feed Engine's working threads:

using namespace OnixS::ICE::iMpact::MarketData;
FeedEngineSettings feedEngineSettings;
feedEngineSettings.threadAffinity.insert(1);

The same CPU set is applied to every worker thread of the engine. See the Threading Model topic for how the Feed Engine, its worker threads, and Handlers relate.

Setting Thread Priority

In addition to CPU affinity, the priority of each worker thread can be tuned. The FeedEngineListener::onFeedEngineThreadBegin callback is invoked in the context of the worker thread just before it enters its processing loop, which makes it the right place to apply thread-specific tuning such as raising the scheduling priority or assigning a thread name:

// A Feed Engine listener lets you tune each worker thread from within the
// worker thread itself: onFeedEngineThreadBegin() is invoked in the context of
// the thread that receives and processes market data. CPU affinity is already
// applied from FeedEngineSettings::threadAffinity before this callback fires,
// so this is the place to raise the thread's scheduling priority or assign it a
// name.
class WorkerThreadTuner
  : public OnixS::ICE::iMpact::MarketData::FeedEngineListener
{
public:
    void onFeedEngineThreadBegin(
        const OnixS::ICE::iMpact::MarketData::FeedEngine& /*engine*/
    ) override
    {
        // Apply platform-specific tuning for the current thread here, e.g.
        // raise the scheduling priority via pthread_setschedparam() on POSIX
        // systems or SetThreadPriority() on Windows.
    }
};
 
void SettingThreadPriority()
{
    using namespace OnixS::ICE::iMpact::MarketData;
 
    FeedEngineSettings settings;
    settings.threadAffinity.insert(1);
 
    // The listener must outlive the Feed Engine; declaring it first guarantees
    // it is destroyed after the engine has stopped its worker threads.
    WorkerThreadTuner tuner;
    FeedEngine feedEngine(settings, &tuner);
}

Tuning the Data Waiting Interval

When a Feed Engine worker thread finishes processing the previously received market data, it tries to receive new data. Pauses between incoming network packets may cause the worker thread to block while waiting for incoming data. As a result, data and executable code can be evicted from the processor's cache, and the subsequent processing cycle could be slower.

The FeedEngineSettings::dataWaitTime parameter defines how long, in milliseconds, the Feed Engine blocks while waiting for incoming data. Reducing it increases the number of wake-ups and lowers the probability that the Feed Engine's data and code are evicted from the cache. When the parameter is 0, the Feed Engine constantly checks for data availability (busy waits).

The FeedEngineSettings::spinBeforeIdleTime parameter defines how long, in milliseconds, a redundant worker thread keeps cycling before it goes to sleep once it runs out of work. It applies only when several worker threads share the engine (threadCount > 1); with the default single worker thread it has no practical effect, because that thread always has incoming I/O to wait on.

using namespace OnixS::ICE::iMpact::MarketData;
 
FeedEngineSettings settings;
 
// Busy-poll for incoming data instead of blocking. This minimizes wake-up
// latency at the cost of up to 100% CPU utilization on each worker thread.
settings.dataWaitTime = 0;
 
// spinBeforeIdleTime only affects redundant workers, so it takes effect
// only with more than one worker thread.
settings.threadCount = 2;
 
// Keep an otherwise-redundant worker spinning for a few milliseconds before
// it sleeps, instead of parking it as soon as it runs out of work.
settings.spinBeforeIdleTime = 5;

Warning

Reducing the data waiting interval or increasing the spin time lowers latency but increases CPU consumption, up to 100% utilization per worker thread when FeedEngineSettings::dataWaitTime is 0. Tune these parameters against your own workload.

Suppressing Market Data Copying

Under normal conditions, Handler effectively utilizes internal structures used to keep incoming market data. Packets and ICE iMpact messages are reused once the contained data is processed by the Handler. Therefore, no data is allocated during real-time market data processing.

However, data may be copied within the callbacks that the Handler invokes as a listener for various market data events. Thus, when a book is copied, it triggers memory allocation, which negatively affects performance and latency. To improve results, copying should be minimized, or preallocation strategies should be used.