Using TCP Recovery Facility

Recovering from Data Loss

The Handler receives market data through the network multicast feeds. Due to a multicast nature, data may come in the wrong order or may be completely lost. The Handler does its best to fix all the network-related issues. However, packet gaps may still happen while processing market data.

The CME MDP provides its clients with an ability to request and obtain lost data using a reliable TCP connection. This facility is called a TCP Recovery. If the Handler is configured to use a TCP Recovery facility, in case of data loss, it doesn't report an error immediately and does not fall into a book recovery. Instead, it suspends regular processing and requests a remote system to resend missed data. The Handler resumes regular processing without restarting its execution, once it successfully receives lost packets from the service.

If a request to recover missed data through a TCP Recovery feed fails, the Handler tries to recover lost packets for a number of times defined by the OnixS::CME::ConflatedUDP::TcpRecoverySessionSettings::attempts parameter.

It may happen the missing data is not sent due to the TCP recovery service malfunctioning. It happens, the counterpart sends heartbeats but requested data is not transmitted. In such cases, the Handler may stop processing real-time data for a undefined time. To prevent from being locked in the TCP recovery, the Handler allows to limit the time it spends in recovering missing data. The limit can be established or updated by using the OnixS::CME::ConflatedUDP::TcpRecoverySessionSettings::servingTimeLimit parameter.

If the Handler is not able to recover lost packets for a predefined number of attempts or the allocated time for recovery is out, it spawns a recovery from snapshots or resumes real-time processing as defined by other session settings.

Note

The CME MDP limits the number of TCP recovery requests it serves per second. The OnixS::CME::ConflatedUDP::TcpRecoverySettings class exposes the OnixS::CME::ConflatedUDP::TcpRecoverySettings::maxRequests parameter allowing to control the number of requests allowed per second. If the number of requests exceeds the defined limit, the recovery request is rejected by the service and thus the Handler will have either to repeat an attempt to recover missing data or to recover market state from snapshots or resume natural incremental processing.

Enabling TCP Recovery

It is necessary to contact CME support for TCP Recovery credentials to be able to use a TCP Recovery facility.

Also, the MDP does not allow multiple recovery requests served simultaneously using the same credentials. Only one TCP recovery session can be active at a time. For this reason, the SDK exposes a TCP recovery feature as a shared service which can be bound to multiple Handler instances. Implementation of this feature covers all synchronization related aspects to avoid multiple requests at the same time.

The following example demonstrates how to use TCP Recovery with the Handler:

using namespace OnixS::CME::MDPH;

// Configuring TCP Recovery.

TcpRecoverySetting tcpRecoverySettings;

// Credentials are to be set as there's no defaults for
// these parameters. Other parameters have default values.

tcpRecoverySettings.username("CME");
tcpRecoverySettings.password("CME");

// Constructing instance of TCP Recovery
// service and binding to the Handler.

TcpRecoveryService tcpRecovery(tcpRecoverySettings);

// Configuring Handler.

Handler handler;

handler.settings().channel(310);
handler.settings().feeds().connectivityConfigurationFile("config.xml");

// This tells the Handler to use TCP recovery before doing other steps. 
handler.settings().session().tcpRecovery().service(&tcpRecovery);

handler.start();