PCAP Replay

What is PCAP replay?

The Handler supports replaying market data not only from log files but also from PCAP files. PCAP is a widely used capture file format, and data in this format can be generated by many networking tools.

The Handler requires the following data for replay:

1. Product definitions data (TCP);
2. Snapshot packets (UDP);
3. Live packets (UDP);

In the current implementation, the Handler supports replay from PCAP files for snapshot data and for live UDP data. TCP data should be provided as a raw network dump.

PCAP data providers

As there are many ways to create PCAP data, the Handler uses identifiers for different data providers. Currently, the Handler supports only one data provider. However, you can request the addition of any custom data provider. Please contact our Sales Team if your data source is not currently supported.

Supported PCAP data providers:

1. ICE Data Services (IDS).

Convert PCAP data to log file format

The Handler can convert PCAP data to the log file format. The tools/PcapConverter directory contains a tool for this purpose. For details, see the tools/PcapConverter/README.md documentation.

Replay directly from PCAP files

The Handler supports direct replay from IDS files. To enable this, create an instance of PcapDataProvider with the PcapDataProviderFactory::create method. Then assign it to HandlerSettings::inputStream.

For example:

using namespace OnixS::ICE::iMpact::MarketData;
 
// Create an instance of custom log input stream.
std::unique_ptr<PcapDataProvider> pcapDataProvider(
    PcapDataProviderFactory::create(
        KnownPcapDataProviderIds::IDS,
        "replay",
        "../Configuration/ConnectivityConfiguration.Production.xml"));
 
HandlerSettings handlerSettings;
 
// Set custom log replay input stream.
handlerSettings.inputStream = pcapDataProvider.get();

The PcapDataProvider interface inherits from the custom log replay input stream interface, i.e., LogReplayInputStream. This interface is used when instead of plain log file a custom data source should be used.

For more details how all pieces work together please see the PcapReplay sample. This sample also includes a sample of PCAP data which can be used to check how it works.

Timestamps

For replayed packets, the Handler uses the same timestamps as available in the PCAP file for UDP data. But for TCP, there is no such source so the Handler just uses the current timestamp. The timestamp is available in MessageInfo::receiveTime and in Packet::time if the user uses PacketProcessingListener.