Using Logs to Replay Market Data

Under normal flow, Handler logs all the important aspects of its execution onto the file system. Log files also include original market data processed by the Handler. This information is usually saved for analysis of non-standard situations which may occur during use of the Handler. However, it also can be used to reproduce standard Handler's behavior for a certain period.

Once Handler was executed with logging enabled, it's possible to use log files for further replay. First of all, logs must be backed up (copied to another location) or the Handler's configuration must be updated to use another directory for new log files because the current implementation of the Handler doesn't support replay from the same folder in which new logs are stored.

Market data replay can be done with the help of start member. It accepts an instance of OnixS::Eurex::MarketData::ReplayOptions class containing a list of logs to be replayed.

Collection of log files can be either manually assigned or fulfilled from the provided folder.

ReplayOptions replayOptions("logStore");

handler.start(replayOptions);

Alternatively, it's possible to manually define a list of log files to be replayed. Manually manipulating list of log files allows replaying log files whose names differ from names used by the Handler.

ReplayOptions replayOptions;
replayOptions.logs.push_back("1.txt");
replayOptions.logs.push_back("2.txt");
replayOptions.logs.push_back("3.txt");

Note: Files to be replayed must be in exact order as they were recorded by the Handler in bounds of single processing session.

From listener callbacks perspective, there's no difference whether the Handler processes market from the network or log files.

Note: Currently, log replay must be run with the same set of Handler's settings which were originally used.

Sample application demonstrating Log Replay feature can be found in samples/Backtesting subfolder of distributive library package.

Note: For more and up-to-date information demonstrating Log Replay feature see Backtesting sample from the samples collection available in the distributive package.

Using Packet Captures to Replay Market Data

It is also possible to replay market data stored as network packet captures (.PCAP files). It is very similar to replaying the log files:

handler.replayPcap(ReplayOptions("pcapStore", ".pcap"));

The packet capture file can contain data for for several channels, therefore it is possible to replay a single set of files to several handlers:

std::vector<EmdiHandler*> handlers;
handlers.push_back(&handler1);
handlers.push_back(&handler2);
replayPcap(handlers, ReplayOptions("pcapStore", ".pcap"));

If handler manager is used to create the handlers:

EmdiHandlerManager emdi(emdiSettings);
MarketSegments productNames;
productNames.insert("FDAX");
emdi.replayPcap(&rdiHandler, productNames, ReplayOptions("pcapStore", ".pcap"));

Note: it is also possible to replay gzipped files.