File-based Session Storage

When a File-based Session Storage is used, messages and session state-related data are stored in files in StorageDirectory; writing is done during the execution of the Send(IMessage) method.

For each FIX session with a File-based Session Storage the following files are created in the StorageDirectory :

STORAGE_ID.summary - contains inbound and outbound FIX messages.
STORAGE_ID.state - contains session state-related data.

STORAGE_ID is formed on the base of SenderCompId, TargetCompId, session's FIX Version and a timestamp. It is available via the StorageId property.

To create a session with file-based session storage, the storageType parameter in the Session(string, string, ProtocolVersion, bool, SessionStorageType, string) constructor should be set to the FileBasedStorage value.

For example:

const bool keepSequenceNumbersBetweenFixConnections = true;
var session = new Session("SenderCompID", "TargetCompID", ProtocolVersion.Fix44, keepSequenceNumbersBetweenFixConnections, SessionStorageType.FileBasedStorage);

Segmented File-based Storage

File-based Storage could be configured to store messages in a set of *.summary files, instead of single one. In this mode, logged messages are split through multiple files called "segments".

Segmented mode of File-based Storage is activated when the FileBasedStorageSegmentSize property is set to a positive value during the Engine initialization.

For example:

var settings = new EngineSettings();
settings.FileBasedStorageSegmentSize = 256 * 1024;
settings.IgnoreFileBasedStorageIntegrityErrors = true;
Engine.Init(settings);

const bool keepSequenceNumbersBetweenFixConnections = true;
Session initiator = new Session("SenderCompID", "TargetCompID", ProtocolVersion.Fix44, keepSequenceNumbersBetweenFixConnections, SessionStorageType.FileBasedStorage);

Segments names are in the following format: STORAGE_ID[-part-N].summary, where:

STORAGE_ID- the name of the storage (available via the StorageId property).
-part-N- the segment suffix, the N is 1-based segment index. The segment where the latest messages are written does not have this suffix.

For example, the following set of files could be created: STORAGE_ID.summary, STORAGE_ID-part-1.summary, STORAGE_ID-part-2.summary, etc. Oldest messages are stored in STORAGE_ID-part-1.summary, and the latest ones - in STORAGE_ID.summary.

The current number of segments (N+1) is stored in the STORAGE_ID.state file (the FilePartCount key).

When a session is re-started, to restore previously sent messages segment files are analyzed in the following order:

STORAGE_ID.summary.
STORAGE_ID-part-N.summary
...
STORAGE_ID-part-.summary`

The number of segments to analyze depends on the ResendingQueueSize value and on message types (for example, session-level messages are not restored).

While the session is active, all segments that could be used to service inbound Resend Request messages are kept opened to prevent unexpected deleting.

When new messages arrive, new storage segments are created if needed. If the older ones become unused, they are closed and could be archived.

Restoring Session State from File-based Storage

When a Session instance with a File-based Session Storage is created, the FIX Session state (sequence numbers of last received and sent messages, previously sent messages, etc.) is restored from files that were written earlier.

To start a FIX session as a new one (so-called "clean start"), it is necessary to remove the session files from the previous run.

Connecting parties must bi-laterally agree as to when sessions are to be started/stopped, and log files are backed up based upon individual system and time zone requirements.

When the FIX session is finished, the session files are not needed anymore. They can be backed up, and later a new FIX session with the same SenderCompId and TargetCompId will start sequence numbers from 1.

The usual practice is to back up the log files at the end of each business day (so-called "End Of Day procedure") to start the sequence numbers from 1 at the beginning of the next day.

Note

To reset the local sequence numbers to 1 and back-up the log files, use the ResetLocalSequenceNumbers() member. This method can be called only when the session is disconnected.

Restoring Large Numbers of Messages from File-based Storage

When a Session instance is created, the previously sent messages are restored from the STORAGE_ID.summary file. If this file and the number of messages are large, the restoring process during the session creation can take a long time.

The ResendingQueueSize parameter affects the restoring process time. However, it only affects the processing time of outgoing messages.

If there are incoming messages in the *.summary file after the last required outgoing message, the overall processing time will include the processing time of all incoming messages. The processing time of incoming messages depends on the number of such messages in the *.summary file - the larger, the longer.

By default, the restoring algorithm processes messages one by one, including incoming messages. This is because the algorithm considers that FIX messages can contain RawData(96) fields or other fields that may contain embedded SOH characters. Such fields can contain any symbols, so the algorithm should process messages one by one to detect the correct beginning/end of each FIX message.

The SessionStorageReadingMode property can be used to control the File-based session storage reading mode. If no FIX message fields may contain embedded SOH characters, one can set it to the NoEmbeddedSohCharacters mode. In this case, the restoring algorithm will skip incoming messages from the *.summary file, which can significantly decrease the restoring process time.

Integrity Check

When the session state is restored, the Engine checks that all files required to fill the resending queue are available and could be read.

If some files are not available or inaccessible the Engine behavior depends on the IgnoreFileBasedStorageIntegrityErrors property: