Detecting packet gaps

Due to the unreliable nature of multicast, network packets transmitted by the CME MDP 3 may come in an order different from the original or be lost. The handler's detection of such packets that are sent via the Incremental feed is controlled by the following parameters:

These parameters can be accessed via handler.settings().feeds().incrementalFeeds().

The handler uses the packet sequence number to detect packet loss. When the handler receives a packet with a sequence number greater than expected, it decides on whether to wait for expected packets or continue accepting incoming packets for further processing, thus ignoring the expected ones. Decision making is based on timing and comparing sequence numbers of incoming packets to the expected ones.

Suppose the handler receives a packet with a sequence number greater than expected. In that case, it compares the packet's sequence number with the threshold, which represents the sum of the seqNumberOfLastAcceptedPacket and the outOfOrderPacketsMaxInterval values, where the outOfOrderPacketsMaxInterval is the value of the configuration parameter mentioned above. The seqNumberOfLastAcceptedPacket is the sequence number of the last packet the handler accepted for further processing. If the threshold is not exceeded, the handler stores the incoming packet in the internal queue and waits until a new packet is received. Otherwise, the handler reports the packet loss.

The handler also establishes the time limit it waits for missing packets. If the handler receives missing data within the established time interval, it resumes normal data processing. However, if no data is received for the predefined time frame, the handler reports the packet loss. The OnixS::CME::MDH::RealtimeFeedSettings::lostPacketWaitTime parameter defines the time interval the handler waits for missing data.

The time limit and sequence number checking are two independent criteria. Suppose the missed data is not received within the predefined time interval. In that case, the handler reports the data loss no matter whether the sequence numbers of incoming packets exceed the threshold or not. And vice versa.

Gap detection and feed arbitrage

The sequence number checking is performed for each feed independently. If arbitrage between Incremental feeds is enabled, the handler does not report the packet loss until incoming packets from both feeds exceed the sequence number threshold.

Gap interval

The handler enqueues incoming packets while waiting for the missed ones. The handler reports the packet loss when a gap detection criterion is satisfied.

Before the message gap is reported, the handler determines the missing packet range. It analyses enqueued packets and extracts the most recent ones representing the continuous sequence number subsequence. The extracted packets will be accepted for further processing. Other packets will be disregarded. Thus, the gap will represent an interval, the beginning of which is the last accepted (processed) packet plus one, and the end is the first packet of the extracted continuous sequence minus one:

[ seqNumberOfLastProcessedPacket + 1, seqNumberOfHeadOfContinuousSubsequence - 1 ]

For example, if the last processed (accepted) packet is # 1000 and pending packets are # 1002, # 1005, # 1007, and # 1008, then the gap interval will represent [1001, 1006]. Pending packets # 1002 and # 1005 will be disregarded, whereas packets # 1007 and # 1008 will be accepted for further processing.