Setting Up Feed Engine

The Feed Engine concept encapsulates data delivery aspects to let the Handler focus on market data processing. It also provides advanced facilities on utilizing system resources to achieve the best performance and latency characteristics.

Using the Feed Engine

The SDK offers the OnixS::CME::ConflatedUDP::SocketFeedEngine class as the primary and default implementation of the Feed Engine concept to receive market data transmitted through the multicast feeds.

The following code sample depicts how to construct an instance of the Feed Engine, link it with an instance of the Handler, and run the Feed Engine machiner to let the Handler process incoming market data and other events.

// Initializes the socket-based Feed Engine.
SocketFeedEngineSettings feedEngineSettings;
SocketFeedEngine feedEngine(feedEngineSettings);

// Initializes the Handler. 
Handler handler;

// Lets the Handler to extract connectivity-related information. 
handler.settings().channel(310);
handler.settings().feeds().connectivityConfigurationFile("config.xml");

// Binds the Handler to the Feed Engine.
handler.feeds().engine(&feedEngine);

// Configures processing session. 
setSessionToAccurateLateJoin(handler.settings().session());

// Moves the Handler to processing state.
handler.start();

// Runs the Feed Engine machinery to receive market data 
// from the network and to push it to the Handler.

while (!userRequestedToEndLooping())
  feedEngine.process();

// Stops the Handler. 
handler.stop();

Sharing the Feed Engine Between Multiple Handlers

Multiple Handler instances can share a single instance of the Feed Engine:

void 
configure(
    Handler& handler,
    ChannelId channel,
    NetFeedEngine& feedEngine)
{
    HandlerSettings& settings = handler.settings();

    settings.channel(channel);

    settings.feeds().connectivityConfigurationFile("config.xml");
    settings.feeds().engine(&feedEngine);    

    setSessionToAccurateLateJoin(settings.session());
}

...

// Initializes the Feed Engine.
SocketFeedEngineSettings feedEngineSettings;
SocketFeedEngine feedEngine(feedEngineSettings);

// Initializes the Handlers. 
Handler handlerForChannel310;
Handler handlerForChannel312;

// Configures Handlers. 
configure(
  handlerForChannel310, 
  310, 
  feedEngine);

configure(
  handlerForChannel312, 
  312, 
  feedEngine);

// Moves the Handler to processing state.
handlerForChannel310.start();
handlerForChannel312.start();

// Runs the Feed Engine machinery to receive market data 
// from the network and to push it to the Handler.

while (!userRequestedToEndLooping())
  feedEngine.process();

// Stops the Handler. 
handlerForChannel310.stop();
handlerForChannel312.stop();

Multi-threaded Processing

If the implementation of the Feed Engine concept supports multithreaded processing, then it's possible to take advantage of distributing Feed Engine machinery tasks across multiple threads.

Note

In general, the Feed Engine concept does not obligate implementations to support multithreaded execution. Therefore, the OnixS::CME::ConflatedUDP::NetFeedEngine::process function may not support simultaneous invocation by multiple threads. Please, check the documentation of a particular implementation, whether it's safe to call OnixS::CME::ConflatedUDP::NetFeedEngine::process simultaneously, and the other conditions.

The SDK exposes a ready-to-use service which cyclically invokes the OnixS::CME::ConflatedUDP::NetFeedEngine::process member function of the associated instance of the OnixS::CME::ConflatedUDP::NetFeedEngine class by a set of threads. The functionality is encapsulated in the OnixS::CME::ConflatedUDP::FeedEngineThreads class.

The given class manages a pool of threads. When an instance of the class is created, it spawns working threads. The number of threads is defined by the ThreadPoolSettings::size parameter, which can be accessed through the OnixS::CME::ConflatedUDP::FeedEngineThreadSettings::pool member. Each thread cyclically calls the OnixS::CME::ConflatedUDP::NetFeedEngine::process member for the instance of the OnixS::CME::ConflatedUDP::NetFeedEngine class delivered to the service (class) at construction time.

All events exposed by the service encapsulated into the OnixS::CME::ConflatedUDP::FeedEngineThreadListener class. Subscribing to events is done at the service construction time.

The following table uncovers meaning of each event:

Event	Description
OnixS::CME::ConflatedUDP::FeedEngineThreadListener::onFeedEngineThreadBegin	Invoked by the working thread before entering master processing loop. Event callback is invoked in the context of working thread allowing the subscriber to perform thread-specific turning like setting affinity or priority for the thread.
OnixS::CME::ConflatedUDP::FeedEngineThreadListener::onFeedEngineThreadEnd	Invoked by when a particular working thread is about to end. The given event callback is invoked in the context of the working thread allowing the subscriber to perform thread-specific cleanup like deleting data stored in thread local storage.
OnixS::CME::ConflatedUDP::FeedEngineThreadListener::onFeedEngineThreadIdle	Invoked when the working thread is idle. At an active phase, a working thread is running market data processing loop. It analyses the returned value of OnixS::CME::ConflatedUDP::NetFeedEngine::process call. If neither OnixS::CME::ConflatedUDP::NetFeedEngineProcessResult::ioWaited nor OnixS::CME::ConflatedUDP::NetFeedEngineProcessResult::eventsDispatched flags are raised for a predefined number of iterations the working thread invoked the OnixS::CME::ConflatedUDP::NetFeedEngine::process member, then the thread raises the given event. The given callback assumes returning a boolean value which indicates whether a working thread invoked the given callback should sleep for a bit of time in order to reduce load onto CPU or resume processing without any delay. Warning There's no predictability in firing of the given event by the service. Each call of the OnixS::CME::ConflatedUDP::NetFeedEngine::process may wait for incoming data or receive incoming data or process data received by the calls from the other threads. Availability of active tasks, which are executed by working threads, depends on many factors including behavior of Handler instances, system capacity, and a number of allocated working threads, clock resolution, and other factors.
OnixS::CME::ConflatedUDP::FeedEngineThreadListener::onFeedEngineThreadIssue	Invoked when a working thread experiences an issue while carrying out its tasks of if invocation of the OnixS::CME::ConflatedUDP::NetFeedEngine::process member leaded to exception throwing.