Programming Guide

Introduction

OnixS Java FIX Engine is a simple fully Java compliant tool that will FIX-enable applications written in Java.

The engine provides the following services:

  • Manages a network connection.
  • Manages the FIX session layer (for the delivery of application messages), automatically handles Sequence Reset, Resend Request, Test Request, Logon and Logout session-level messages.
  • Manages the FIX application layer (defines business related data content).
  • Creates (outgoing) FIX messages.
  • Parses (incoming) FIX messages.
  • Validates FIX messages.
  • Persists messages and FIX session state (the ability to log data to a flat file).
  • Recovers the FIX session state after restart in accordance with the FIX state model.
  • SSL-based encryption.
  • FAST compression and decompression.

The following image is a high-level overview of the .NET FIX Engine architecture:

Getting Started

The typical way of using the engine is as follows:

  1. Initialize the engine (see Engine Initialization and Shutdown).
  2. Select the FIX Version.
  3. Create the FIX Session.
  4. Establishing FIX Connection as Initiator or Acceptor session.
  5. Exchanging Messages FIX Message.
  6. Disconnect the FIX Session.
  7. Shutdown the engine (see Engine Initialization and Shutdown).

Check also following chapters:

  1. Error Reporting
  2. Licensing

Engine Initialization and Shutdown

Initializing the engine

To initialize the engine services the

biz.onixs.fix.engine.Engine.init()

method is used. This method must be called before all other methods.

The engine configuration is set during initialization phase. See Configuring the Engine for details.

Shutting down the engine

To shutdown the engine services the

biz.onixs.fix.engine.Engine.shutdown()

method is used. No other engine methods must be called after this method.

Example.

22
23
24
25
26
27
28
29
30
import biz.onixs.fix.engine.Engine;
  
public class EngineInitAndShutdown {
    public static void main(String[] args) {
        final Engine engine = Engine.init();
        // App logic
        engine.shutdown();
    }
}

FIX Version

As the FIX protocol evolves its new versions will be released to you to extend the current functionality.

Supported FIX versions are: FIX 4.0 - 5.0, 5.0 SP1, 5.0 SP2. The

biz.onixs.fix.dictionary.Version

enum is used to identify them.

Example.

21
22
23
24
25
26
27
28
29
30
import biz.onixs.fix.dictionary.Version;
import biz.onixs.fix.engine.Engine;
  
public class VersionGetter {
    public static void main(String[] args) {
        final Engine engine = Engine.init();
        final Version fixVersion = Version.getByNumber("4.2");
        engine.shutdown();
    }
}

FIX Session

A FIX session is defined as a bi-directional stream of ordered messages between two parties within a continuous sequence number series. It is represented by the

biz.onixs.fix.engine.Session

class.

Constructing Session

To create a session object the appropriate constructor is used.

Session Role

A session has one of two possible roles: Acceptor or Initiator.

Acceptor is the receiving party of the FIX session. It listens for the incoming connection on the pre-defined port. The acceptor has responsibility to perform first level authentication and formally declare the connection request "accepted" through transmission of an acknowledgment Logon message.

Initiator establishes the telecommunications link and initiates the session via transmission of the initial Logon message.

To obtain current role the

biz.onixs.fix.engine.Session.getRole()

method is used.

Terminate Session

To terminate the session the

biz.onixs.fix.engine.Session.dispose()

method is used.

Establishing FIX Connection

Establishing Connection

To establish a FIX connection as Acceptor the

biz.onixs.fix.engine.Session.logonAsAcceptor()

method is used.

To establish a FIX connection as Initiator the

biz.onixs.fix.engine.Session.logonAsInitiator(String, int)

method is used.

Closing Connection

To disconnect the session the

biz.onixs.fix.engine.Session.logout()

method is used. This member implements a graceful closing of the FIX connection as defined by the Standard.

Note
logonAsInitiator(..) and logout() methods are blocked until the Logon/Logout response is received. These methods have a timeout during which the response should be received. This timeout is equal to the heartbeat interval (by default 30 sec) + ReasonableTransmissionTime value (by default 20%) as percentage from the heartbeat interval. If the acknowledgment Logon/Logout message is not received during the timeout then the Logout message will be sent and the FIX connection will be closed.

Reconnection Facility

When a FIX session is in the active state and there are some network issues the engine will try to restore the FIX connection in accordance with ConnectionRetries.Number and ConnectionRetries.Interval settings.

Note
Reconnection facility works for already established connections. In other cases, you need to use the Sessions Scheduler component or manually handle LinkErrorException or TimeoutException exceptions of logonAsInitiator(..) method and try to connect again.

Example.

29
30
31
32
33
34
35
36
37
38
39
final Session acceptor = new Session("SenderCompID", "TargetCompID", Version.FIX40);
final Session initiator = new Session("TargetCompID", "SenderCompID", Version.FIX40);
acceptor.logonAsAcceptor();
// Sends the Logon message and waits for the acknowledgment Logon
initiator.logonAsInitiator("localhost", engine.getSettings().getListenPorts()[0]);
// Sends the Logout message and waits for the confirming Logout
initiator.logout();
acceptor.logout();
// Free resources
initiator.dispose();
acceptor.dispose();

Connection and Session Lifetime

A single FIX session can exist across multiple sequential (not concurrent) physical connections. Parties can connect and disconnect multiple times while maintaining a single FIX session. That is, once the logout() method is called, the FIX Connection is terminated, however, the FIX session life continues. It is possible to continue the same session later using either logonAsAcceptor() or logonAsInitiator(..) methods again. In other words, a FIX Session is comprised of one or more FIX connections.

See also

Understanding Session States, Connecting using Custom Logon Message.

Exchanging Messages

Sending Messages to a Counterparty

To send a message to counterparty the

biz.onixs.fix.engine.Session.send(Message)

or

biz.onixs.fix.engine.Session.send(FlatMessage)

methods are used. This method is asynchronous. As soon as a session is created it is possible to start sending messages via the session.

If the session is not established, the messages are stored in the session storage and will be sent in replay to the resend request when the connection is established with the counterparty and the sequence numbers mismatch is detected (please see the Resending Messages page).

Receiving Messages from a Counterparty

To receive incoming application-level messages it is necessary to implement the

biz.onixs.fix.engine.Session.InboundApplicationMessageListener

or

biz.onixs.fix.engine.Session.InboundApplicationFlatMessageListener

interface and register the handler with the

biz.onixs.fix.engine.Session.setInboundApplicationMessageListener(InboundApplicationMessageListener)

or

biz.onixs.fix.engine.Session.setInboundApplicationFlatMessageListener(InboundApplicationFlatMessageListener)

To receive incoming session-level messages it is necessary to implement the

biz.onixs.fix.engine.Session.InboundSessionMessageListener

or

biz.onixs.fix.engine.Session.InboundSessionFlatMessageListener

interface and register the handler with the

biz.onixs.fix.engine.Session.setInboundSessionMessageListener(InboundSessionMessageListener)

or

biz.onixs.fix.engine.Session.setInboundSessionFlatMessageListener(InboundSessionFlatMessageListener)
Note
Both listeners flat and structured ones provides the same FIX message but different representations. The flat message listener is fired first.
Note
It is inefficient from the performance point of view to register for both types of messages.

Example.

Structured message exchange:

41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
private static class MessageProcessor implements
        Session.InboundApplicationMessageListener,
        Session.InboundSessionMessageListener {
    @Override
    public void onInboundApplicationMessage(Object sender, Session.InboundApplicationMessageArgs args) {
        System.out.println("Incoming application-level message: " + args.getMsg());
    }
  
    @Override
    public void onInboundSessionMessage(Object sender, Session.InboundSessionMessageArgs args) {
        System.out.println("Incoming session-level message: " + args.getMsg());
    }
}
  
public static void main(String[] args) {
    final Engine engine = Engine.init(PORT);
    //
    final Session acceptor = new Session(SENDER_COMP_ID, TARGET_COMP_ID, VERSION);
    final MessageProcessor messageProcessor = new MessageProcessor();
    acceptor.setInboundApplicationMessageListener(messageProcessor);
    acceptor.setInboundSessionMessageListener(messageProcessor);
    //
    final Session initiator = new Session(TARGET_COMP_ID, SENDER_COMP_ID, VERSION);
    //
    acceptor.logonAsAcceptor();
    initiator.logonAsInitiator(HOST, PORT);
    //
    final Message order = createOrder();
    initiator.send(order);

Flat message exchange:

41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
private static class FlatMessageProcessor implements
        Session.InboundApplicationFlatMessageListener,
        Session.InboundSessionFlatMessageListener {
    @Override
    public void onInboundApplicationFlatMessage(Object sender, Session.InboundApplicationFlatMessageArgs args) {
        System.out.println("Incoming application-level flat message: " + args.getMsg());
    }
  
    @Override
    public void onInboundSessionFlatMessage(Object sender, Session.InboundSessionFlatMessageArgs args) {
        System.out.println("Incoming session-level flat message: " + args.getMsg());
    }
}
  
public static void main(String[] args) {
    final Engine engine = Engine.init(PORT);
    //
    final Session acceptor = new Session(SENDER_COMP_ID, TARGET_COMP_ID, VERSION);
    final FlatMessageProcessor messageProcessor = new FlatMessageProcessor();
    acceptor.setInboundApplicationFlatMessageListener(messageProcessor);
    acceptor.setInboundSessionFlatMessageListener(messageProcessor);
    //
    final Session initiator = new Session(TARGET_COMP_ID, SENDER_COMP_ID, VERSION);
    //
    acceptor.logonAsAcceptor();
    initiator.logonAsInitiator(HOST, PORT);
    //
    final FlatMessage flatOrder = createFlatOrder();
    initiator.send(flatOrder);
Message Sequence Numbers

Understanding Message Sequencing

A single FIX session can exist across multiple sequential (not concurrent) physical connections. Parties can connect and disconnect multiple times while maintaining a single FIX session. That is, once the logout() method is called, the FIX Connection is terminated, however, the FIX session life continues. It is possible to continue the same session later using either logonAsAcceptor() or logonAsInitiator(..) methods again. In other words, a FIX Session is comprised of one or more FIX connections.

All FIX messages are identified by a unique sequence number (MsgSeqNum field) within the bounds of a single FIX session. Sequence numbers are initialized at the start of each FIX session starting at 1 (one) and increment throughout the session. Each session establishes an independent incoming and outgoing sequence series.

The engine automatically synchronizes sequence numbers when reconnecting during a FIX session on the base of the information that was previously stored in log files.

Manipulating Message Sequence Numbers

Monitoring sequence numbers enables parties to synchronize applications gracefully when reconnecting during a FIX session.

The expected sequence number of the next incoming message can be get

biz.onixs.fix.engine.Session.getInSeqNum()

or set (session must be disconnected):

biz.onixs.fix.engine.Session.setInSeqNum(long)

The sequence number of the next outgoing message can be get

biz.onixs.fix.engine.Session.getOutSeqNum()

or set (session must be disconnected):

biz.onixs.fix.engine.Session.setOutSeqNum(long)

Resetting inbound and outbound sequence numbers back to 1, for whatever reason, constitutes the beginning of a new FIX session. Method

biz.onixs.fix.engine.Session.resetLocalSequenceNumbers()

must be used to backup the previous log files and reset the sequence numbers to 1.

Note
Both setInSeqNum(..) and setOutSeqNum(..) properties can be used to set expected sequence numbers manually before a FIX connection is established. However, it is NOT recommended to use these properties to reset the message sequence numbers to 1 for the session. Instead, method Session.resetLocalSequenceNumbers() must be used.

Some implementations of the FIX protocol expect that a FIX Session coincides with a FIX Connection. In other words, message sequence numbers must be reset back to 1 after the Logout messages exchange and the FIX connection is terminated.

To interact with such FIX venues, keepSequenceNumbersAfterLogout parameter of session constructor should be set to false. In this case after the logout() method sequence numbers are set back to 1. Next time when the Session object is created incoming and outgoing message sequence numbers will start from 1 (in other words, a new FIX Session will be created).

See also

Resetting Message Sequence Numbers, Resetting Message Sequence Numbers via ResetSeqNumFlag Field.

Automatic Message Fields

The FIX session sets or updates specific fields in the outgoing messages automatically.

The following fields belong to the message header and trailer. They are mandatory for all FIX versions and all message types.

The following fields are set or updated only if they are configured explicitly for the session. They are applicable for all message types.

Field FIX Version
SenderSubID(50) all
TargetSubID(57) all
SenderLocationID(142) since 4.1
TargetLocationID(143) since 4.1
LastMsgSeqNumProcessed(369) since 4.2

FIX Message

In general, FIX message represents a sequence of fields whose values are associated with unique numbers (tags).

A common way of presenting FIX message is called the tag-value FIX message format. In such a format all fields are presented as tag=value pairs and are delimited by special SOH symbol.

The following image depicts the structure of FIX message in tag-value format:

The engine provides two representation of the FIX message by the following classes:

biz.onixs.fix.parser.Message

and

biz.onixs.fix.parser.FlatMessage

FlatMessage class

The class to encapsulate basic services related with handling FIX messages. It does not require any dialect to be parsed from raw FIX message. It does not concern about whether the body (the payload) of a FIX message contains a sequence of tag/value pairs that resemble on a higher semantic level a repeating group. And does not support validation against a dialect.

It allows to add, set and get FIX message fields associated with a tag number or index in the FlatMessage. The order of fields will be kept as they are received.

In order to perform validation and/or accessing repeating groups it can be converted into a Message class with corresponding FIX version/dialect. Multiple dialects can be used.

Message class

The class to encapsulate all services related with handling FIX messages. It has structure based on the type and the version of the FIX protocol/dialect which allows to perform validation and access to repeating groups.

Constructing Blank Message

To create blank flat message object, the

biz.onixs.fix.parser.FlatMessage.FlatMessage()

constructor must be used, empty flat message will be initialized and ready for interaction.

To create blank structured message object, the

biz.onixs.fix.parser.Message.create(String, Version)

constructor must be used. It initializes a structured message of the specified type and a FIX protocol version with no fields presented, except a couple of service fields like those, which identifies type and version of FIX protocol to which constructed message belongs to.

Note
By reason of the internal design you cannot change the message type of the created structured message and perform the validation correctly, when a FIX message is created with the certain type then the particular message structure is associated with this message to perform validation and determine undefined fields, so you need to create a new message when you want to change the message type.

Converting Message from and to Raw (Tag-Value) Format

biz.onixs.fix.parser.FlatMessage.FlatMessage(byte[], int)

and

biz.onixs.fix.parser.Message.Message(String)

constructors are available to construct flat and structured message instances correspondingly from its raw (tag-value) representation.

biz.onixs.fix.parser.FlatMessage.assemble(ByteBuffer)

and

biz.onixs.fix.parser.Message.assemble(ByteBuffer)

methods are available to build tag-value presentation of a message instance. Also several overloads of toString() methods are available which are simplify debugging and monitoring.

Note
While building the tag-value presentation of a message instance BodyLength(9) and Checksum(10) tags will be updated.

Validating Message

In addition to the presence of a particular field in the message of a certain type, the FIX protocol also defines whether the presence of fields are strictly required in the message, whether they are required under certain conditions, or whether they are optional. To check for the presence of all required fields, the

biz.onixs.fix.parser.Message.validate()

method is used. See also Required Fields. In the engine terminology, this process is called message validation.

Note
Validation is available only for the structured message and not for the flat one.

Message life cycle

Sometimes there is a need to use message, after it was sent to a session. As soon as the message was sent, the engine will not use it later, so it can be used again; for example, it will be sent to other sessions.

Note
The message object is not thread safe, so it cannot be used in one thread while it is used in another.

Example.

29
30
31
32
33
34
35
36
// Create "New Order - Single", MsgType = "D"
final Message order = Message.create("D", Version.FIX40);
// Another way to create FIX message
final String rawMsg = "8=FIX.4.0\u00019=86\u000135=D\u000149=0\u000156=0\u000134=1\u000152=99990909-17:17:17\u0001"
        + "11=90001008\u000121=1\u000155=IBM\u000154=1\u000138=10\u000140=1\u000159=0\u000110=191\u0001";
final Message order2 = new Message(rawMsg);
// Validate created message
order2.validate();
Manipulating Message Fields

A FIX message represents a sequence of fields whose values are associated with unique numbers (tags). Such an interpretation treats a message as an collection of values associated with tag numbers. For this reason, the primary approach in handling message fields is similar to managing associative collections.

Adding Field Into Message

To associate a field with a value in structured message, the

biz.onixs.fix.parser.Message.set(int, String)

method must be used. If there is no field of a given tag number available in the message, this member creates an association. If the field already exists, this member updates its value with the new value.

To associate a field with a value in flat message, the

biz.onixs.fix.parser.FlatMessage.add(int, String)

method must be used. It will add tag/value pair at the end of the message no matter it exists or not. OR

biz.onixs.fix.parser.FlatMessage.set(int, String)

method must be used. If there is no field of a given tag number it will throw an exception. If the field already exists, this member updates its value with the new value.

Accessing Field Value

To get a field value the following method can be used for structured message:

biz.onixs.fix.parser.Message.get(int)

and the following methods for the flat message:

biz.onixs.fix.parser.FlatMessage.get(int)

and

biz.onixs.fix.parser.FlatMessage.getByIndex(int)

where the indexes can be obtained using the following method:

biz.onixs.fix.parser.FlatMessage.findIndex(int)
Note
Implementations of all field value getters are avoiding creation of internal objects as much as it is possible as object creation is an expensive operation in Java.

Accessing Field Value Directly

The following class has been implemented in order to provide direct access/pointer to the region in a received message of value associated with the field.

biz.onixs.util.ValuePtr

Example usage which avoids a string allocation:

31
32
33
34
35
36
37
38
39
40
41
final Message order = Message.create("D", Version.FIX40);
  
final ValuePtr clOrdIDToLookForPtr = new ValuePtr("client_order_id");  // one-time
  
ValuePtr valuePtr = new ValuePtr(); // one-time
// many times something like:
if (order.get(FIX40.Tag.ClOrdID, valuePtr) && clOrdIDToLookForPtr.equals(valuePtr)) {
    System.out.println("Found!");
} else {
    System.out.println("NOT found");
}

Removing Field From Message

To remove a field value the following method can be used for structured message:

biz.onixs.fix.parser.Message.remove(int)

and the following methods for the flat message:

biz.onixs.fix.parser.FlatMessage.remove(int)

and

biz.onixs.fix.parser.FlatMessage.remove()

Example.

29
30
31
final Message order = Message.create("D", Version.FIX40);
order.set(11, "client_order_id");
final String clOrdID = order.get(11);

Named Tags Constants

The following class contains the constants for the tag values:

biz.onixs.fix.tag.Tag

Also there are similar classes defined for each FIX version:

biz.onixs.fix.tag.FIX40.Tag

etc. The use of these constants makes the source code more readable.

Example.

30
31
32
33
final Message order = Message.create("D", Version.FIX40);
order.set(Tag.ClOrdID, "90001008");
order.set(Tag.Side, "1");
order.set(Tag.TimeInForce, "0");

Iterating over Fields of Message

To iterate over all fields of a FIX message please use the following method for the structured message:

biz.onixs.fix.parser.Message.iterator()

This iterator does not iterate over fields from a repeating group. To iterate over fields of the repeating group you need to get the corresponding GroupInstance object from the repeating group, please see the FIX Repeating Group.

The following method for the flat message:

biz.onixs.fix.parser.FlatMessage.iterator()

Message class provides iterator over raw message, please use the following method:

biz.onixs.fix.parser.Message.iteratorOverRawMsg()

This iterator allows to iterate over all fields including repeating groups of received raw FIX message as it is. Before any modifications to FIX message the set of fields will be in the same order as they are received/parsed and will include duplicates. Compared to the above mentioned iterator it is slower.

Summary

Message type Advantages Disadvantages
Message - Validation can be performed against a dialect. 
- Provides methods to access a repeating group. 
- Get/Set values using the associated tag number will be performed in constant time. 
- Message type(i.e. tag# 35) and FIX protocol version(i.e. tag #8) cannot be changed for already created message. 
- Changes the order of fields after vice versa conversion from raw message according to dialect file specified. 
- It is not possible to get all values of duplicated fields. 
- It is not possible to add duplicated fields. 
FlatMessage - Any field of the message can be changed including message type(i.e. tag# 35) and FIX protocol version(i.e. tag #8). 
- Keeps the order of fields as they are in original raw message after vice versa conversion. 
- Not concern about FIX message body/payload. 
- Provides access to all values of duplicated fields. 
- Duplicated fields can be added. 
- No validation can be performed only basic tag=value format will be checked. 
- Get/Set values using the associated tag number will be performed in linear time but it is possible to get corresponding index for multiple access in constant time. 

See also

FIX Repeating Group.

Error Reporting

Exception is used as a fundamental error-reporting mechanism. In the event of any error the

biz.onixs.fix.engine.EngineException

is thrown.

Licensing

The pre-defined license file name is "OnixS.lic". The license file is loaded during engine initialization as described in the Unified Resource Loading.

The default license file name can be changed using LicenseFile parameter in the Properties File or property of the Settings Class:

biz.onixs.fix.engine.EngineSettings.setLicenseFile(String)

The following table explains different options to keep the license file.

License File Location License File Name LicenseFile Property Note
application classpath OnixS.lic no change required
current directory *.lic no change required Any file with the ".lic" extension in the current directory is checked automatically.
user home directory *.lic no change required Any file with the ".lic" extension in the user home is checked automatically.
application classpath AltName.ext AltName.ext
absolute path "/foo/bar" AltName.ext /foo/bar/AltName.ext
relative path "foo/bar" AltName.ext foo/bar/AltName.ext

Configuring the Engine

There is a number of parameters (settings) which control the behaviour of the engine. EngineSettings class provides the way to change settings of the engine programmatically. However, the default settings of the engine can also be changed using an external configuration file.

The engine is configured during initialization phase. The initialization method is overloaded. Thus, there are multiple ways to pass the configuration.

Properties File

The engine settings are taken from the properties.

27
28
final PropertyBasedSettings settings = new PropertyBasedSettings("site/engine.properties");
final Engine engine = Engine.init(settings);

The properties file is loaded as described in the Unified Resource Loading.

The configuration file example follows.

1
2
3
4
5
Dialect = site/dialect.xml
Log.InboundMessages = true
Validate.NumberOfRepeatingGroupInstances = false
Validate.RequiredFields = true
Validate.UnknownFields = true

Settings Class

The engine settings are available as the class properties.

27
28
final EngineSettings settings = new EngineSettings();
final Engine engine = Engine.init(settings);

The settings can be changed before initialization method call.

Settings

The configuration parameter (key) is case-sensitive. The list of available parameters follow.

Please note the values should be specified without quotation marks.

Setting Description Programmatically Type Default value
LicenseFile Specifies the name of the license file resource. EngineSettings.setLicenseFile(String) 
EngineSettings.getLicenseFile()
string OnixS.lic
Versions Specifies the FIX dictionaries to load during the engine initialization. The dictionary names are separated with the '|' character. EngineSettings.setVersions(String) 
EngineSettings.getVersions()
string FIX.4.0 | FIX.4.1 | FIX.4.2 | FIX.4.3 | FIX.4.4 | FIX.5.0 | FIX.5.0_SP1 | FIX.5.0_SP2
Dialect Specifies the location of one or several files with the FIX dialects definition. The file names are separated with the '|' character. The dialect files are loaded during engine initialization as described in the Unified Resource Loading. EngineSettings.setDialects(String) 
EngineSettings.getDialects()
string empty
ListenPort FIX engine listens on this port(s) for incoming connections. If it is set to "0" then only session-initiators can be created. If it is set to "-1" then the telecommunication level is disabled and only message parsing/assembling can be used. Use comma delimited list if more than one listen port is required. It is important to avoid using ports used by other services. EngineSettings.setListenPorts(String) 
EngineSettings.getListenPorts()
string 0
Log.Directory Inbound/outbound messages and session's state data are stored in this directory. EngineSettings.setStorageFolder(String) 
EngineSettings.getStorageFolder()
string FixEngineStorage
Log.InboundMessages Option to store inbound messages in the session storage. EngineSettings.setLogInboundMessages(boolean) 
EngineSettings.isLogInboundMessages() 
Session.setLogInboundMessages(boolean) 
Session.isLogInboundMessages()
boolean true
Log.OutboundMessages Option to store outbound messages in the session storage. EngineSettings.setLogOutboundMessages(boolean) 
EngineSettings.isLogOutboundMessages() 
Session.setLogOutboundMessages(boolean) 
Session.isLogOutboundMessages()
boolean true
AsyncFileStorage.QueueSize This parameter sets asynchronous operations queue size for session storage. It is effective for AsyncFileBased session storage type only. EngineSettings.setAsyncFileStorageQueueSize(int) 
EngineSettings.getAsyncFileStorageQueueSize()
integer 2000
Validate.InboundMessages Option to validate inbound messages. EngineSettings.setValidateInboundMessages(boolean) 
EngineSettings.isValidateInboundMessages()
boolean false
Validate.OutboundMessages Option to validate outbound messages. EngineSettings.setValidateOutboundMessages(boolean) 
EngineSettings.isValidateOutboundMessages() 
boolean false
Validate.NumberOfRepeatingGroupInstances Option to validate the declared number of repeating group instances is equal to the actual one. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. EngineSettings.setValidateNumberOfRepeatingGroupInstances(boolean) 
EngineSettings.isValidateNumberOfRepeatingGroupInstances()
boolean false
Validate.RequiredFields Option to validate the presence of required fields. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. EngineSettings.setValidateRequiredFields(boolean) 
EngineSettings.isValidateRequiredFields()
boolean false
Validate.UnknownFields Option to validate the presence of unknown fields. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. EngineSettings.setValidateUnknownFields(boolean) 
EngineSettings.isValidateUnknownFields()
boolean false
Validate.FieldValues Option to fail validation on empty field value. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. EngineSettings.setValidateFieldValues(boolean) 
EngineSettings.isValidateFieldValues()
boolean false
Validate.DuplicatedField Option to validate the presence of fields which appears more than once. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. EngineSettings.setValidateDuplicatedField(boolean) 
EngineSettings.isValidateDuplicatedField()
boolean false
ConnectionRetries.Interval The initial time interval between the attempts to restore the telecommunication link, in milliseconds. EngineSettings.setConnectionRetriesInterval(int) 
EngineSettings.getConnectionRetriesInterval()
integer 90000
ConnectionRetries.Number Number of attempts to restore the telecommunication link. The range is [0..MAX_INT]. EngineSettings.setConnectionRetriesNumber(int) 
EngineSettings.getConnectionRetriesNumber()
integer 3
Connection.TcpNoDelay Option to improve latency at the expense of message throughput. EngineSettings.setConnectionTcpNoDelay(boolean) 
EngineSettings.isConnectionTcpNoDelay() 
Session.setTcpNoDelay(boolean) 
Session.getTcpNoDelay()
boolean true
Connection.TcpSendBufferSize The size of the TCP buffer allocated to FIX connection for sending data, in bytes. EngineSettings.setConnectionTcpSendBufferSize(int) 
EngineSettings.getConnectionTcpSendBufferSize() 
Session.setSendBufferSize(int) 
Session.getSendBufferSize()
integer 65535
Connection.TcpReceiveBufferSize The size of the TCP buffer allocated to FIX connection for receiving data, in bytes. EngineSettings.setConnectionTcpReceiveBufferSize(int) 
EngineSettings.getConnectionTcpReceiveBufferSize() 
Session.setReceiveBufferSize(int) 
Session.getReceiveBufferSize()
integer 65535
Connection.OutputQueueDrainAmount The max number of messages to take from the outgoing queue and write to the outgoing TCP buffer in a single call. EngineSettings.setConnectionOutputQueueDrainAmount(int) 
EngineSettings.getConnectionOutputQueueDrainAmount() 
Session.setOutputQueueDrainAmount(int) 
Session.getOutputQueueDrainAmount()
integer 10
ReasonableTransmissionTime The reasonable transmission time as % from heartBtInt field value. When either end of the connection has not received any data for (HeartBtInt * (100 + ReasonableTransmissionTime)/100) seconds, it will transmit a Test Request message. If there is still no heartbeat message received after (HeartBtInt * (100 + ReasonableTransmissionTime)/100) seconds then the connection should be considered lost and corrective action be initiated. EngineSettings.setReasonableTransmissionTime(int) 
EngineSettings.getReasonableTransmissionTime() 
Session.setReasonableTransmissionTime(int) 
Session.getReasonableTransmissionTime()
integer 20
SessionListenerDelayWarningThreshold The limit of time to spend in the session listener code. If the listener takes longer then the warning is generated. In milliseconds. EngineSettings.setSessionListenerDelayWarningThreshold(int) 
EngineSettings.getSessionListenerDelayWarningThreshold()
integer 1000
Engine.DefaultHeartbeatInterval The default heartbeat interval in seconds. EngineSettings.setDefaultHeartbeatInterval(int) 
EngineSettings.getDefaultHeartbeatInterval()
integer 30
SessionEventArgReuse Option to re-use session event argument objects in each session. EngineSettings.setSessionEventArgReuse(boolean) 
EngineSettings.isSessionEventArgReuse()
boolean false
SessionInboundMessageReuse Option to use the same message object for the incoming messages.  In case of re-using mode the message object can be operated only until control is returned from the inbound message listener. EngineSettings.setSessionInboundMessageReuse(boolean) 
EngineSettings.isSessionInboundMessageReuse() 
Session.setInboundMessageReuse(boolean) 
Session.isInboundMessageReuse()
boolean false
Session.SendLogoutOnException Option to send the logout message before dropping the telecommunication link in case of exception during the processing of incoming FIX message. EngineSettings.setSessionSendLogoutOnException(boolean) 
EngineSettings.isSessionSendLogoutOnException()
boolean true
CheckCompIdsMatch Option to check that SenderCompId and TargetCompId of every inbound FIX message match session values. EngineSettings.setCheckCompIdsMatch(boolean) 
EngineSettings.isCheckCompIdsMatch()
boolean false
ProcessLogonNextExpectedMsgSeqNum Option to process NextExpectedMsgSeqNum(789) field in Logon(A). EngineSettings.setProcessLogonNextExpectedMsgSeqNum(boolean) 
EngineSettings.isProcessLogonNextExpectedMsgSeqNum()
boolean false
Resending.QueueSize Number of sent messages that are available for resending on counterparty's Resend Request(2) message. EngineSettings.setResendingQueueSize(int) 
EngineSettings.getResendingQueueSize() 
Session.setResendingQueueSize(int) 
Session.getResendingQueueSize()
integer 1000
Session.UpdateSendingTime If this option set to true then the SendingTime(52) field in the outgoing messages is always updated. If this option set to false then the SendingTime(52) field is set only if it is not set already. EngineSettings.setSessionUpdateSendingTime(boolean) 
EngineSettings.isSessionUpdateSendingTime() 
Session.setUpdateSendingTime(boolean) 
Session.isUpdateSendingTime()
boolean true
Session.UpdateHeaderFields If this option set to true then common header fields in the outgoing messages are always updated. If this option set to false then the common header fields are set only if they are not set already. The fields are SenderCompID(52), TargetCompID(56) and others. EngineSettings.setSessionUpdateHeaderFields(boolean) 
EngineSettings.isSessionUpdateHeaderFields()
boolean true
Session.ResendRequestMaximumRange The maximum number of messages to be requested in one Resend Request(2) message. For no limits use 0. EngineSettings.setSessionResendRequestMaximumRange(int) 
EngineSettings.getSessionResendRequestMaximumRange() 
Session.setResendRequestMaximumRange(int) 
Session.getResendRequestMaximumRange()
integer 0
Session.SpecifyLastMsgSeqNumProcessed Option to specify the LastMsgSeqNumProcessed(369) field on every message sent. EngineSettings.setSessionSpecifyLastMsgSeqNumProcessed(boolean) 
EngineSettings.isSessionSpecifyLastMsgSeqNumProcessed() 
Session.setSpecifyLastMsgSeqNumProcessed(boolean) 
Session.isSpecifyLastMsgSeqNumProcessed()
boolean false
EngineShutdownHook Option to shutdown engine automatically on JVM shutdown. EngineSettings.setEngineShutdownHook(boolean) 
EngineSettings.isEngineShutdownHook()
boolean false
Connection.OutStreamDelayWarningThreshold Option to specify the output stream write delay threshold after which the warning is logged. In milliseconds. EngineSettings.setConnectionOutStreamDelayWarningThreshold(int) 
EngineSettings.getConnectionOutStreamDelayWarningThreshold() 
integer 1000
UserSessionLevelMessageWarning Option to specify whether to warn if user sends session level message. EngineSettings.setUserSessionLevelMessageWarning(boolean) 
EngineSettings.isUserSessionLevelMessageWarning() 
Session.setUserSessionLevelMessageWarning(boolean) 
Session.isUserSessionLevelMessageWarning()
boolean true
Acceptor.SendLogoutOnInvalidLogon Option to send a Logout(5) message, before dropping the telecommunication link in reply to an invalid logon attempt. EngineSettings.setAcceptorSendLogoutOnInvalidLogon(boolean) 
EngineSettings.isAcceptorSendLogoutOnInvalidLogon()
boolean false
Session.ResendRequestLogic Session resend request logic. Available values can be found here: ResendRequestLogic EngineSettings.setSessionResendRequestLogic(ResendRequestLogic) 
EngineSettings.setSessionResendRequestLogic(String) 
Session.setResendRequestLogic(ResendRequestLogic) 
Session.getResendRequestLogic()
enumeration  STANDARD_FIX
IgnoreFileBasedStorageIntegrityErrors behaviour when File Based Storage errors are detected. If 'true' creates new session storage when errors are detected. If 'false' throws exception and prevents session against creating. EngineSettings.setIgnoreFileBasedStorageIntegrityErrors(boolean) 
EngineSettings.isIgnoreFileBasedStorageIntegrityErrors() 
boolean false
SpecifyApplVerIdField Option to specify ApplVerID(1128) tag in all FIX messages when a FIX protocol version FIX 5.0 and above is used. EngineSettings.setSpecifyApplVerIdField(boolean) 
EngineSettings.isSpecifyApplVerIdField() 
Session.setSpecifyApplVerIdField(boolean) 
Session.isSpecifyApplVerIdField()
boolean true
SslNeedClientAuth Option to specify whether client authentication should be required. EngineSettings.setSslNeedClientAuth(boolean) 
EngineSettings.isSslNeedClientAuth()
boolean false

Unified Resource Loading

By default, any external resource is looked for at the following places and in the following order (if other is not set explicitly):

  1. Classpath
  2. Absolute file path
  3. Current directory
  4. User home directory

If it can't be found at the 1st location then the 2nd one is checked, etc.

For example, you can put a dialect file to the classpath or current directory and it will be located by engine.

Venue Specific Settings

This section describes venue specific settings, which should be used in work with a particular venue specific counterparty.

CME Specific Settings

Secure Logon

CME Globex implemented secure authentication for iLink and Drop Copy sessions on Convenience Gateway (CGW) and Market Segment Gateway (MSGW). The new logon procedure secures the client system logon with:

  • Customer identity verification - a client system logon request will be signed with security credentials issued and validated by CME Group.
  • Message confidentiality and integrity - to credential the logon message, the client system sends a keyed-hash message authentication code (HMAC) generated from a combination of the logon FIX tag values. When CME Globex receives the logon message, it uses the identical inputs to calculate the HMAC value to validate against the logon request. If the values do not match, CME Globex rejects the logon.

Customers must create secure key pairs for iLink and Drop Copy Sessions in the CME Customer Center.

CME Secure Logon requires to add several specific tags to Logon message, so our FIX Engine is ready to implement Secure Logon procedure. We have updated our CME Trading Client sample to help implementing Secure Logon.

Implementing Secure Logon consist of two parts: creating canonical request string and calculating HMAC. Canonical request string is just a union of several tags from Logon message. Please note, that Session fills values of some fields like SendingTime right before message send, so its values can be taked from onOutboundSessionMessage() callback. To calculate HMAC a standard Java cryptographic classes are used.

Example.

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
public void onOutboundSessionMessage(Object o, Session.OutboundSessionMessageArgs args) {
    if (FIX42.MsgType.Logon.equals(args.getMsg().getType())) {
  
        Message customLogon = args.getMsg();
  
        if (accessKeyID != null && secretKey != null) {
            customLogon.set(FIX42.Tag.EncodedTextLen, accessKeyID.length());
            customLogon.set(FIX42.Tag.EncodedText, accessKeyID);
            customLogon.set(encryptedPasswordMethodTag, "CME-1-SHA-256");
            customLogon.remove(FIX42.Tag.EncryptMethod);
  
            String canonicalRequest = createCanonicalRequest(customLogon, session.getHeartBtInt());
            String hash = calculateHmac(canonicalRequest, secretKey);
            customLogon.set(encryptedPasswordLenTag, hash.length());
            customLogon.set(encryptedPasswordTag, hash);
        }
    }
}
  
private String createCanonicalRequest(Message customLogon, int heartbeatInterval) {
    StringBuilder sb = new StringBuilder();
  
    char delimiter = '\n';
  
    sb.append(session.getOutSeqNum()).append(delimiter);
    sb.append(session.getSenderCompID()).append(delimiter);
    sb.append(session.getSenderSubID()).append(delimiter);
    sb.append(customLogon.get(FIX42.Tag.SendingTime)).append(delimiter);
    sb.append(session.getTargetSubID()).append(delimiter);
    sb.append(heartbeatInterval).append(delimiter);
    sb.append(session.getSenderLocationID()).append(delimiter);
    sb.append(customLogon.get(FIX42.Tag.LastMsgSeqNumProcessed)).append(delimiter);
    sb.append(customLogon.get(tradingSystemNameTag)).append(delimiter);
    sb.append(customLogon.get(tradingSystemVersionTag)).append(delimiter);
    sb.append(customLogon.get(tradingSystemVendorTag));
  
    return sb.toString();
}
  
private String calculateHmac(String canonicalRequest, String userKey) {
    String hash = null;
    try {
        // Init HMAC instance
        Mac sha256HMAC;
        sha256HMAC = Mac.getInstance("HmacSHA256");
  
        // Initialize HMAC instance with the key
        // Decode the key first, since it is base64url encoded
        byte[] decodedUserKey = Base64.getUrlDecoder().decode(userKey);
        SecretKeySpec secretKey = new SecretKeySpec(decodedUserKey, "HmacSHA256");
        sha256HMAC.init(secretKey);
  
        // Calculate HMAC, base64url encode the result and strip padding
        hash = Base64.getUrlEncoder().withoutPadding().encodeToString((sha256HMAC.doFinal(canonicalRequest.getBytes("UTF-8"))));
  
    } catch (NoSuchAlgorithmException | InvalidKeyException | IllegalStateException | UnsupportedEncodingException e) {
        e.printStackTrace();
    }
    return hash;
}

Logging Services

This section explains how logging services are organized in the engine. It depicts the structure of logs, as well as how logging can be customized.

By default, the engine uses file system directory specified by the "Log.Directory" configuration parameter for storing as well as the engine log file:

  • incoming and outgoing messages
  • session state data

File-based Logs Structure

For each FIX session with a file-based session storage the following files are also created:

  • SESSION_NAME.Rx.summary – contains inbound and outbound FIX messages. The sent messages could be read from this file and re-send again on counterparty request.
  • SESSION_NAME.state – contains session state data.

Where:

Restoring Session State from Logs

When the

biz.onixs.fix.engine.Session

object is created anew after the engine restart, FIX session state (the sequence numbers of last received and sent messages, previously sent messages, etc) is restored from these files.

To start a FIX session as a new one (so called "clean start"), it is necessary to remove the log files from the previous runs.

Connecting parties must bi-laterally agree as to when sessions are to be started/stopped and log files are backed up based upon individual system and time zone requirements.

When the FIX session is finished, the log files are not needed anymore. They can be backed up and later a new FIX session with the same SenderCompID and TargetCompID will start sequence numbers from 1. The usual practice is to backup the log files at the end of each business day (so called "End Of Day procedure") to start the sequence numbers from 1 at the beginning of the next day.

Note
To reset the local sequence numbers to 1 and backup the log files, the Session.resetLocalSequenceNumbers() method must be used. This method can be called only when the session is disconnected.

Controlling Engine Logging

By default, the engine logs all important aspects of its activity while working. The SLF4J (Simple Logging Facade for Java) is used by Handler internally. The SLF4J detects and uses the concrete logging implementation configured by user. By default, the Logback logging implementation is recommended.

Please see Logging section for more details.

Advanced Programming

FIX Repeating Group

Sometimes fields of the same tag number can appear multiple times in the same message within a so-called repeating group.

The following diagram depicts the general structure of a repeating group in a FIX message:

Each repeating group starts from the field which identifies the number of repeating entries within repeating group. Such a leading field must immediately precede the repeating group entries. In the referenced example, the leading field is defined by the NoRoutingIDs tag.

Each entry of a repeating group has a selected field which in turn identifies the beginning of a group new entry. This field must occur in a raw message before any other tag from a single repeating group entry. In other words, these fields separate repeating group entries from each other. RoutingType field is an example of such a separator.

Please also see quote from FIX standard:

Fix 4.4 Specification vol 1 page 19
If the repeating group is used, the first field of the repeating group is required. This allows implementations of the protocol to use the first field as a "delimiter" indicating a new repeating group entry. The first field listed after the NoXXX, then becomes conditionally required if the NoXXX field is greater than zero.

Important aspects of FIX repeating groups structure:

  • Well-defined FIX messages assume all fields of a repeating group that follow one another. The appearance of an arbitrary field, which doesn't belong to repeating group (according to the FIX specification), automatically indicates the end of the repeating group. Therefore, the further appearance of fields from the repeating group is disallowed by the FIX Standard.
  • It is possible for repeating groups to appear inside of another repeating group. In such case, all entries of the inner repeating group belong to the single entry of the outer repeating group.
  • The number of entries or instances, which are defined by the value of the leading field (tag), must correspond to the number of instances/entries, which actually appear in the message. In particular, number of tags-separators must equal to value of the leading field.

Group class

The FIX repeating group is represented with the class:

biz.onixs.fix.parser.Group

Since a repeating group is identified by the leading field, which defines the number of repeating instances, the engine follows this approach in handling of repeating groups. In particular, the Group object is accessed using its leading (number of instances) field. Changing the value of this field affects the length of a repeating group. Removing this field from a message or other repeating group removes all entries of the repeating group.

Adding Group into Message

To create a new repeating group or modify the number of instances in the existing one the

biz.onixs.fix.parser.Message.setGroup(int, int)

method is used.

Accessing Repeating Group Entries

To get the group object that represents the existing repeating group of the message the

biz.onixs.fix.parser.Message.getGroup(int)

method is used.

Removing Repeating Group from Message

To remove a repeating group method

biz.onixs.fix.parser.Message.remove(int)

is used.

The group class works with fields and embedded repeating groups in the same manner as the

biz.onixs.fix.parser.Message

class, but each method has an additional parameter that defines the index of the repeating group instance (starting from 0).

Ill-Formed Repeating Group

Declared number of instances is the number of repeating group instances, that are defined by the FIX field which identifies a repeating group size ("NoField"). Actual number of instances is the number of repeating group instances which actually exist in FIX message, and can be calculated by counting presence of FIX fields (which make up repeating group) in the fields sequence.

By default, the engine strictly follows requirements of the FIX Standard. Thus, it requires the actual number of repeating group instances to match declared number of instances. FIX message parsing services report an error by throwing exception, if FIX message doesn't meet the requirements.

The engine exposes configuration parameter Validate.NumberOfRepeatingGroupInstances, which allows to change default behaviour. By changing parameter value as shown below, it's possible to have ill-formed FIX messages parsed successfully.

Note
In case a repeating group from a message is not parsed completely try to parse the message using message constructor with enabled validateDuplicatedField parameter.
Note
If there is a parse exception with message Duplicate field found ... and it refers to a field from repeating group probably the field is not defined correctly in the dialect file.

Example.

Setting Value

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
final Message message = Message.create("V", Version.FIX42);
message.set(Tag.MDReqID, "ABSD");
message.set(Tag.SubscriptionRequestType, 1);
message.set(Tag.MarketDepth, 1);
message.set(Tag.MDUpdateType, 1);
message.set(Tag.AggregatedBook, "N");
// create a repeating group NoMDEntryTypes with two instances
final Group groupMDEntryTypes = message.setGroup(Tag.NoMDEntryTypes, 2);
groupMDEntryTypes.set(Tag.MDEntryType, 0, "EntryType_1"); // first instance
groupMDEntryTypes.set(Tag.MDEntryType, 1, "EntryType_2"); // second instance
// create a repeating group NoRelatedSym with two instances
final Group groupRelatedSym = message.setGroup(Tag.NoRelatedSym, 2);
groupRelatedSym.set(Tag.Symbol, 0, "EURUSD_0"); // first instance
groupRelatedSym.set(Tag.Symbol, 1, "EURUSD_1"); // second instance
message.validate();
LOG.info("Message: {}", message);

Getting Value

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
final Message message = new Message("8=FIX.4.2\u00019=142\u000135=V\u000149=0\u000156=0\u000134=0\u0001" +
        "52=99990909-17:17:17\u0001262=ABSD\u0001263=1\u0001264=1\u0001265=1\u0001266=N\u0001267=2\u0001" +
        "269=EntryType_1\u0001269=EntryType_2\u0001146=2\u000155=EURUSD_0\u000155=EURUSD_1\u000110=163\u0001");
LOG.info("MDEntryTypes group");
final Group mdEntryTypes = message.getGroup(Tag.NoMDEntryTypes);
if (mdEntryTypes != null) {
    for (int i = 0; i < mdEntryTypes.getNumberOfInstances(); i++) {
        LOG.info("Instance # {}", i);
        LOG.info("MDEntryType = '{}'", mdEntryTypes.get(Tag.MDEntryType, i));
    }
}
LOG.info("RelatedSym group");
final Group relatedSym = message.getGroup(Tag.NoRelatedSym);
if (relatedSym != null) {
    for (int i = 0; i < relatedSym.getNumberOfInstances(); i++) {
        LOG.info("Instance # {}", i);
        LOG.info("Symbol = '{}'", relatedSym.get(Tag.Symbol, i));
    }
}

FIX Session

Understanding Session States

During the FIX session lifetime, state changes occur. The Session class exposes the

biz.onixs.fix.engine.Session.getState()

method to determine in which state it currently resides.

The following tables describes all possible FIX session states that can occur during its lifetime, as well as what a specific state means for a session with a particular role.

State Acceptor
DISCONNECTED The session is disconnected.
AWAIT_LOGON The session is waiting for the initial Logon message.
ESTABLISHED The session is fully established (after the successful logon exchange).
RECONNECTING The session is trying to restore the telecommunication link.
AWAIT_CONFIRMING_LOGOUT The initial logout message was sent and the session is waiting for the acknowledgment logout message.
WAIT_FOR_RETRANSMISSION The session is waiting for the message retransmission from the counterparty.
State Initiator
DISCONNECTED The session is disconnected.
AWAIT_CONFIRMING_LOGON The initial logon message was sent and the session is waiting for the acknowledgment logon message.
ESTABLISHED The session is fully established (after the successful logon exchange).
AWAIT_CONFIRMING_LOGOUT The initial logout message was sent and the session is waiting for the acknowledgment logout message.
WAIT_FOR_RETRANSMISSION The session is waiting for the message retransmission from the counterparty.

Tracking Session State Change

To be notified about the changes in the session state implement

biz.onixs.fix.engine.Session.StateChangeListener

and the register it as event handler with the

biz.onixs.fix.engine.Session.addStateChangeListener(StateChangeListener)

Example.

26
27
28
29
30
31
32
33
34
35
36
37
38
39
public class SessionStateChangeListener implements Session.StateChangeListener {
    public static void main(String[] args) {
        final Engine engine = Engine.init();
        final Session sn = new Session("SenderCompId", "TargetCompId", Version.FIX42);
        sn.addStateChangeListener(new SessionStateChangeListener());
        engine.shutdown();
    }
  
    @Override
    public void onStateChange(Object sender, Session.StateChangeArgs e) {
        System.out.println("Prev session state: " + e.getPrevState());
        System.out.println("New session state: " + e.getNewState());
    }
}
Session Events and Listeners

The events and their listeners of the

biz.onixs.fix.engine.Session

class are listed below.

Avoid time-consuming tasks and session management method calls (logon, logout, reset, etc.) in the session event listener calling thread.

Session state is changed:

biz.onixs.fix.engine.Session.StateChangeListener

Application-level message is received from the counterparty:

biz.onixs.fix.engine.Session.InboundApplicationMessageListener

Application-level flat message is received from the counterparty:

biz.onixs.fix.engine.Session.InboundApplicationFlatMessageListener

Session-level message is received from the counterparty:

biz.onixs.fix.engine.Session.InboundSessionMessageListener

Session-level flat message is received from the counterparty:

biz.onixs.fix.engine.Session.InboundSessionFlatMessageListener

Application-level message will be sent to the counterparty:

biz.onixs.fix.engine.Session.OutboundApplicationMessageListener

Application-level flat message will be sent to the counterparty:

biz.onixs.fix.engine.Session.OutboundApplicationFlatMessageListener

Session-level message will be sent to the counterparty:

biz.onixs.fix.engine.Session.OutboundSessionMessageListener

Application-level message resending request:

biz.onixs.fix.engine.Session.MessageResendingListener

Error condition is detected:

biz.onixs.fix.engine.Session.ErrorListener

Warning condition is detected:

biz.onixs.fix.engine.Session.WarningListener

Example.

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
public class SessionEventListener implements Session.StateChangeListener, Session.MessageResendingListener,
        Session.InboundSessionMessageListener, Session.InboundApplicationMessageListener,
        Session.OutboundSessionMessageListener, Session.OutboundApplicationMessageListener,
        Session.InboundApplicationFlatMessageListener, Session.InboundSessionFlatMessageListener,
        Session.OutboundApplicationFlatMessageListener,
        Session.WarningListener, Session.ErrorListener {
  
    public static void main(String[] args) {
        final Engine engine = Engine.init();
        final Session session = new Session("SenderCompId", "TargetCompId", Version.FIX42);
        final SessionEventListener listener = new SessionEventListener();
        session.addStateChangeListener(listener);
        session.setInboundSessionMessageListener(listener);
        session.setInboundApplicationMessageListener(listener);
        session.setOutboundSessionMessageListener(listener);
        session.setOutboundApplicationMessageListener(listener);
        session.setWarningListener(listener);
        session.setErrorListener(listener);
        session.setMessageResendingListener(listener);
        engine.shutdown();
    }
  
    @Override
    public void onStateChange(Object sender, Session.StateChangeArgs args) {
        System.out.println("New session state: " + args.getNewState());
    }
  
    @Override
    public void onInboundApplicationMessage(Object sender, Session.InboundApplicationMessageArgs args) {
        System.out.println("Incoming application-level message: " + args.getMsg());
    }
  
    @Override
    public void onInboundApplicationFlatMessage(Object sender, Session.InboundApplicationFlatMessageArgs args) {
        System.out.println("Incoming application-level flat message: " + args.getMsg());
    }
  
    @Override
    public void onInboundSessionMessage(Object sender, Session.InboundSessionMessageArgs args) {
        System.out.println("Incoming session-level message: " + args.getMsg());
    }
  
    @Override
    public void onInboundSessionFlatMessage(Object sender, Session.InboundSessionFlatMessageArgs args) {
        System.out.println("Incoming session-level flat message: " + args.getMsg());
    }
  
    @Override
    public void onOutboundApplicationMessage(Object sender, Session.OutboundApplicationMessageArgs args) {
        System.out.println("Outgoing application-level message: " + args.getMsg());
    }
  
    @Override
    public void onOutboundApplicationFlatMessage(Object sender, Session.OutboundApplicationFlatMessageArgs args) {
        System.out.println("Outgoing application-level flat message: " + args.getMsg());
    }
  
    @Override
    public void onOutboundSessionMessage(Object sender, Session.OutboundSessionMessageArgs args) {
        System.out.println("Outbound session-level message: " + args.getMsg());
    }
  
    @Override
    public void onWarning(Object sender, Session.WarningArgs args) {
        System.out.println("Session warning: " + args.getReason());
    }
  
    @Override
    public void onError(Object sender, Session.ErrorArgs args) {
        System.out.println("Session error: " + args.getReason());
    }
  
    @Override
    public boolean onMessageResending(Object sender, Session.MessageResendingArgs args) {
        return false;
    }
}
Accepting FIX Session Without a Prior Creation of Session Object

Sometimes there is a requirement to accept an incoming FIX session "on the fly", without the prior creation of the session object. The engine exposes an ability to create session object in response to the incoming FIX connection. To take advantage of this feature it is necessary to implement and subscribe to the Engine.DynamicAcceptorListener listener in the engine object.

The event arguments object provides information to make decision whether to accept connection:

  • incoming FIX logon message
  • session identification information (session id) — SenderCompId, TargetCompId, etc.
Note
Please note that the listener will not trigger for already created sessions (session with same identification information) even if they are in DISCONNECTED state.

To accept the incoming connection:

  1. session object needs to be created using session identification information (session id) provided in the event arguments or logon message
  2. created Session object is passed to the createdSession property of the event arguments object

Otherwise the incoming connection will be rejected. Optionally a reject reason can be passed to the event arguments object.

Note
There is no need to call Session.logonAsAcceptor() method in DynamicAcceptorListener interface implementation, it will be called internally after the session passed to createdSession property.

Example.

28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
public class DynamicAcceptorSample {
    public static void main(String[] args) {
        final EngineSettings engineSettings = new EngineSettings();
        engineSettings.addListenPort(4500);
        final Engine engine = Engine.init(engineSettings);
        engine.addDynamicAcceptorListener(new AcceptingLogic());
        Utils.waitForEnterToTerminateApp();
        engine.shutdown();
    }
  
    static class AcceptingLogic implements Engine.DynamicAcceptorListener {
        @Override
        public void onDynamicAcceptor(Object sender, Engine.DynamicAcceptorArgs args) {
            final SessionId sessionId = args.getSessionId();
            System.out.println("Dynamic acceptor: " + sessionId + ", logon message " + args.getIncomingLogonMessage());
            final Session session = new Session(sessionId);
            System.out.println("Session created: " + session);
            args.setCreatedSession(session);
        }
    }
  
    private static class RejectingLogic implements Engine.DynamicAcceptorListener {
        @Override
        public void onDynamicAcceptor(Object sender, Engine.DynamicAcceptorArgs args) {
            final SessionId sessionId = args.getSessionId();
            System.out.println("Dynamic acceptor: " + sessionId + ", logon message " + args.getIncomingLogonMessage());
            args.setRejectReason("Face control failed");
        }
    }
}
Connecting using Custom Logon Message

Sometime there is a need to set additional fields (e.g. Username(553), Password(554)) in the initiation logon message. In this case the

biz.onixs.fix.engine.Session.logonAsInitiator(String, int, int, Message)

method can be used.

Example.

33
34
35
36
37
final Session initiator = new Session("InitiatorCompId", "AcceptorCompId", Version.FIX43);
final Message customLogonMsg = Message.create("A", Version.FIX43);
customLogonMsg.set(Tag.Username, "USERNAME");
customLogonMsg.set(Tag.Password, "PASSWORD");
initiator.logonAsInitiator("localhost", 11011, 30, customLogonMsg);

Check also Samples :: Credential, the CredentialBuySide example.

Disconnecting using Custom Logout Message

If there is a need to set additional fields in the Logout(5) message then the custom logout message can be configured.

This custom logout message can be set on the session object:

biz.onixs.fix.engine.Session.setLogoutMessage(Message)

Example.

33
34
35
36
37
38
final Session initiator = new Session("InitiatorCompId", "AcceptorCompId", Version.FIX43);
final Message customLogoutMessage = Message.create(SessionLevelMsgType.Logout, Version.FIX43);
customLogoutMessage.set(2000, "Custom field");
initiator.setLogoutMessage(customLogoutMessage);
initiator.logonAsInitiator("localhost", 11011);
initiator.logout();

Or this custom logout message can be passed to the logout method:

biz.onixs.fix.engine.Session.logout(Message)

Example.

33
34
35
36
37
final Session initiator = new Session("InitiatorCompId", "AcceptorCompId", Version.FIX43);
final Message customLogoutMessage = Message.create(SessionLevelMsgType.Logout, Version.FIX43);
customLogoutMessage.set(2000, "Custom field");
initiator.logonAsInitiator("localhost", 11011);
initiator.logout(customLogoutMessage);
Logon Password Authentication

When Accepting an incoming FIX connection additional authentication checks may be required. The typical checks are logon username/password and source IP address. Depending on the result of the logon verification check the decision can be made whether to accept the FIX connection or reject and close the connection. This is typically performed at the logon stage.

In order to achieve this you should subscribe to the inbound session messages and add the required check logic. If the check fails and the FIX connection needs to be rejected (closed) then an exception can be thrown from the inbound session message listener.

In the following example the password authentication check is demonstrated.

31
32
final Session session = new Session("AcceptorCompId", "InitiatorCompId", Version.FIX42);
session.setInboundSessionMessageListener(new LogonPasswordAuthentication());

The text from the exception goes to the FIX logout message text field (Text(58)).

Check also Samples :: Credential for paired live applications with authentication support.

Resetting Message Sequence Numbers

Session class exposes the

biz.onixs.fix.engine.Session.resetLocalSequenceNumbers()

member to reset both incoming and outgoing message local sequence numbers to 1 and backup the previous log files.

Note
The Session.resetLocalSequenceNumbers() member can be called only when the session is disconnected.
Resetting Message Sequence Numbers via ResetSeqNumFlag Field

ResetSeqNumFlag(141) field in the Logon(A) message indicates that the both sides of the FIX session should reset sequence numbers.

To send a logon message with this field during Establishing FIX Connection, use the setResetSeqNumFlag parameter of the

biz.onixs.fix.engine.Session.logonAsInitiator(String, int, boolean)

method.

Note
Some FIX venues do not support this flag.
Note
Only the first Logon message will be sent with the ResetSeqNumFlag flag. If subsequent Logon messages are sent during the automatic attempts to restore the telecommunication link they will be sent without this flag.
Resending Messages

When the Resend Request(2) message is received from the counterparty (e.g. due to a sequence number mismatch), MessageResending event is raised if an application has previously subscribed to this event. Otherwise, the SequenceReset-GapFill(4) message will be sent instead of the requested message.

If the sent application-level message needs to be resent to the counterparty then the return value of the MessageResending event listener should be set to true while handling the event. Otherwise the SequenceReset-GapFill(4) message will be sent instead (e.g. in case of an aged order).

The message that is about to be resent is available via Msg property of the

biz.onixs.fix.engine.Session.MessageResendingArgs

parameter.

Example.

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
public class MessageResendingSample implements Session.MessageResendingListener {
    public static void main(String[] args) {
        final Engine engine = Engine.init();
        final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX50);
        session.setMessageResendingListener(new MessageResendingSample());
        engine.shutdown();
    }
  
    @Override
    public boolean onMessageResending(Object sender, Session.MessageResendingArgs args) {
        System.out.println("Message is about to be resent to the counterpart: " + args.getMsg());
        // Return false if it's necessary to skip this message (GapFill will be sent).
        return false;
    }
}
Memory Based Session Storage

Memory based session storage could be used to maintain a high-performance FIX session when persisting of session state and messages in the file system is not required.

To create such a session storageType parameter of session constructor should be set to

biz.onixs.fix.engine.storage.SessionStorageType.MemoryBasedStorage
30
31
final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42,
        SessionStorageType.MemoryBasedStorage);
Async File-based Session Storage

Asynchronous File-Based Session Storage combines the good-performance with the File-Based Session Storage functionality. This storage has the same abilities as File-Based Session Storage except the asynchronous file operations. Each this storage creates the separated thread, which will be used to perform all operations with the file system.

To create such a session storageType parameter of session constructor should be set to

biz.onixs.fix.engine.storage.SessionStorageType.AsyncFileBased
30
31
final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42,
        SessionStorageType.AsyncFileBased);
Pluggable Session Storage

Pluggable session storage is the storage which implemented by the user. Such a storage should implement the following interfaces:

biz.onixs.fix.engine.storage.SessionStorage

and

biz.onixs.fix.engine.storage.StorageRepository

After implementing above mentioned interfaces instance of the implemented storage repository should be registered in storage repository manager. Then session with corresponding storage type can be created (see below for more details).

35
36
37
38
final StorageRepositoryManager repositoryManager = engine.getStorageRepositoryManager();
final StorageRepository myStorageRepository = new MyStorageRepository();
SessionStorageType myStorageType = repositoryManager.register("MyStorageID", myStorageRepository);
final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42, myStorageType);

Example.

Please, check the pluggable-storage sample, included into distribution package of FIX Engine.

Scrambling Password in Session Storage

For the security reasons the values of the following fields can be scrambled in the file-based session storage:

  • field Password(554) in Logon(A) message

In order to achieve this on the engine level the following option can be used:

28
29
30
final FileBasedStorageRepository fileStorageRepository
        = engine.getStorageRepositoryManager().getFileStorageRepository();
fileStorageRepository.setScramblePassword(true);
Inbound Message Log Filtering

There is an option to skip saving of some inbound FIX messages to session storage. For instance, for a performance reason.

This can be configured on a per-session basis:

biz.onixs.fix.engine.Session.setInboundMessageLogFilter(MessageFilter)

For instance, the

biz.onixs.fix.filter.TypeMessageFilter

implementation is used to filter specified message types:

31
32
33
final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX40);
final MessageFilter messageFilter = new TypeMessageFilter("W X");
session.setInboundMessageLogFilter(messageFilter);

The space-delimited list of message types to filter is specified.

Default Threading Model

Each session instance starts two separate threads: for sending and receiving messages. The following table explains which listener could be called from the each one thread.

Listener Application thread Sending thread Receiving thread
ErrorListener No No Yes
WarningListener Yes No Yes
InboundApplicationMessageListener No No Yes
InboundSessionMessageListener No No Yes
OutboundApplicationMessageListener Yes No Yes
OutboundSessionMessageListener Yes No Yes
MessageResendingListener No No Yes
StateChangeListener Yes Yes Yes
Custom Timestamp Provider

The engine by default updates SendingTime of FIX messages to be sent to counterparty using internally implemented timestamp provider. Also it uses timestamp provider while writes timestamp of a FIX message written into a log file.

Custom timestamp provider can be implemented and passed to session in order to be used in the above mentioned cases.

It is required to implement the following interface:

biz.onixs.fix.engine.TimestampProvider

and pass an instance to the session using the following method:

biz.onixs.fix.engine.Session.setTimestampProvider(TimestampProvider)

FIX Dialects

It is common for firms to implement slightly different interpretations of the FIX protocol (FIX dialects).

Within the FIX protocol such deviations from the FIX specification must be described in the FIX Engine, to ensure that the parsing or creation of the corresponding FIX messages are correct.

The Dialect configuration parameter specifies one or several dialect definition files in XML format. The dialect definition file must conform to the dialect XML schema.

FIX dialect can be set on:

  • engine level;
  • session level.

Please read details below.

Engine Level

The engine level dialect is defined with the FIX element with no id attribute specified. The dialect defined for the engine level is set implicitly for all engine sessions with such FIX version.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.2">
        <Message type="N">
            <Group numberOfInstancesTag="73">
                <Field tag="37"/>
            </Group>
        </Message>
    </FIX>
</Dialect>

In this example the dialect is set for all FIX 4.2 sessions. I.e. it changes the base FIX version definitions in the engine.

You need no additional steps to start using this dialect in your code. It must be only set in the configuration.

30
31
32
33
34
35
final EngineSettings engineSettings = new EngineSettings();
engineSettings.setDialects("site/dialect_no_id.xml");
final Engine engine = Engine.init(engineSettings);
final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42);
final Message message = Message.create("D", Version.FIX42);
session.send(message);
Session Level

The session level dialect is defined with the FIX element with id attribute specified. The dialect defined for the session level needs to be set explicitly for the selected sessions.

1
2
3
4
5
6
7
8
9
10
11
12
13
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.2" id="MyFIX">
        <Message type="N">
            <Group numberOfInstancesTag="73">
                <Field tag="37"/>
            </Group>
        </Message>
    </FIX>
</Dialect>

In this example the dialect is defined over base FIX 4.2 version definition and has specified id. I.e. it doesn't change the base FIX version definitions in the engine.

Using FIX dialect in the code is easy.

30
31
32
33
34
35
36
final EngineSettings engineSettings = new EngineSettings();
engineSettings.setDialects("site/dialect_with_id.xml");
final Engine engine = Engine.init(engineSettings);
final Version version = Version.getById("MyFIX");
final Session session = new Session("SenderCompID", "TargetCompID", version);
final Message message = Message.create("D", version);
session.send(message);
Dialect Description

To show how to define a FIX variant Dialect using the XML-based description , there is a step-by-step explanation below. It is used by the engine. For a complete reference of the definition capabilities refer to dialects definition XML schema.

Enhancing Message with New Field

To add a new field to a FIX Message add the corresponding <Field> entity to the description of FIX message in the dialect description file.

Note
Inclusion of a Field element for the already registered (e.g. standard) field or group causes the engine to override its attribute. In this way, the existing field attributes can be modified. This behaviour allows you to make a mandatory field or an optional group, and vice versa. As a separate action, the same approach gives the opportunity to replace existing FIX Repeating Groups with regular fields, and vice versa.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.0">
        <Message type="D">
            <!-- Field #526 does not belong to the Standard FIX 4.0. -->
            <Field tag="526" name="SecondaryClOrdID" isRequired="true"/>
            <!-- Field #21 is required in Standard FIX 4.0, but will be optional in this FIX dialect. -->
            <Field tag="21" name="HandlInst" isRequired="false"/>
        </Message>
    </FIX>
</Dialect>
Enhancing Message with Repeating Group

To add a new repeating group to a FIX message, add the corresponding <Group> entity to the FIX dialect description file. The same approach is used to add new fields into an existing repeating group.

Note
In the case of a new group definition, the first field (subgroup) will be used as a leading tag (i.e. the tag which separates repeating groups entries). If the current definition describes the changes to an existing group, then the original leading tag remains operative.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.0">
        <Message type="D">
            <!-- Number of repeating groups for pre-trade allocation, does not belong to Standard FIX 4.0 -->
            <Group numberOfInstancesTag="78" name="NoAllocs">
                <!-- Does not belong to Standard FIX 4.0 -->
                <Field tag="79" name="AllocAccount"/>
                <!-- Does not belong to Standard FIX 4.0 -->
                <Field tag="661" name="AllocAcctIDSource"/>
                <!-- Does not belong to Standard FIX 4.0 -->
                <Field tag="736" name="AllocSettlCurrency"/>
                <!-- Other group fields -->
            </Group>
        </Message>
    </FIX>
</Dialect>
Making Field Optional / Required

To make a field, which is required according to the FIX protocol specification, optional, and visa-versa, add the corresponding <Field> entity to the FIX dialect description and set the isRequired attribute accordingly.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.0">
        <Message type="D">
            <!-- Required in Standard FIX 4.0, but optional in this FIX dialect -->
            <Field tag="21" isRequired="false" name="HandlInst"/>
            <!-- Optional in Standard FIX 4.0, but required in this FIX dialect -->
            <Field tag="109" isRequired="true" name="ClientID"/>
        </Message>
    </FIX>
</Dialect>
Adding Definition of New Message

To define a custom (user-defined) message, add the corresponding <Message> entity to the FIX dialect description.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.0">
        <Message type="UserDefinedMessage_1">
            <Field tag="100" isRequired="true"/>
            <Field tag="101"/>
        </Message>
    </FIX>
</Dialect>

Such a user-defined message can be used exactly the same way as the standard FIX message:

32
33
34
final Message userDefinedMsg = Message.create("UserDefinedMessage_1", Version.FIX40);
userDefinedMsg.set(100, "Field 100");
userDefinedMsg.set(101, "Field 101");
Removing Fields, Repeating Groups and Messages

As it was previously described, the presence of a field definition in a dialect description file overrides and substitutes any entity of the same tag number which is already available in the message or outer repeating group. In this way, repeating groups can be replaced with regular fields and vice versa. However, sometimes there is a requirement to exclude completely certain fields and/or repeating groups from messages or another repeating group. Moreover, sometimes it is necessary to exclude completely certain messages from use. To satisfy this requirement, the dialect description syntax offers the mode attribute, which allows the removal of a single field, repeating group or an entire message from the dialect. To achieve this result, the "remove" value must be specified for the attribute in the corresponding dialect entry for <Field> , <Group> or <Message> .

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.2">
  
        <Message type="8">
            <!-- Partial fill not supported, order is either
                 filled or canceled. Thus, CumQty not needed. -->
            <Field tag="14" mode="remove"/>
  
            <!-- No need in MiscFees group as well. -->
            <Group numberOfInstancesTag="136" mode="remove"/>
        </Message>
  
        <!-- "New Order - List" is not supported. -->
        <Message type="E" mode="remove"/>
  
    </FIX>
</Dialect>
Overriding Existent Definitions of FIX standard dictionary

Using the mode attribute FIX standard dictionary can be overridden. If override mode is defined for either FIX version, the engine will replace the entire definition of FIX version with the new one. In this case, the FIX version will contain only messages, repeating groups and files defined by the description. For example, the description of FIX version 5.0 below will be empty and not contain any definitions at all.

Example.

1
2
3
4
5
6
7
8
9
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="5.0" mode="override" id="Custom_FIX_5.0">
    </FIX>
  
</Dialect>
Overriding Existent Definitions of Messages and Repeating Groups

With the use of the mode attribute, fields can be removed from repeating groups and messages. It is also possible to redefine the structure of the message and/or repeating group. The dialect description language supports the "override" value for the mode attribute. When the override mode is defined for either a message or repeating group, the engine will replace the specified definition of the message or repeating group. In this case, the message and/or repeating group will consist only of the fields and inner repeating groups defined by the description.

Note
Once a repeating group is overridden, its leading tag will be changed in the same way as if it is a new group definition, i.e. the first field, defined within the overridden group, will be considered as the leading one.
Note
Message overriding impacts the basic message structure. Only 4 fields will always be present in the overridden message structure: BeginString (8), BodyLength (9), MsgType (35), CheckSum (10) 
All other fields should be added to the overridden message structure manually.

If the append mode is used and if the declared element has been already presented in the description of the message, it will be removed from its current position and added to the end of the message. For example, if standard definition of message is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="5.0" id="Custom_FIX_5.0">
  
        <Message type="7">
            <Field tag="1"/>
            <Field tag="2"/>
            <Field tag="3"/>
        </Message>
  
    </FIX>
  
</Dialect>

And user has defined their custom definition of this message as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="5.0" id="Custom_FIX_5.0">
  
        <Message type="7">
            <Field tag="1"/>
            <Field tag="4"/>
        </Message>
  
    </FIX>
  
</Dialect>

The result message definition will be as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="5.0" id="Custom_FIX_5.0">
  
        <Message type="7">
            <Field tag="2"/>
            <Field tag="3"/>
            <Field tag="1"/>
            <Field tag="4"/>
        </Message>
  
    </FIX>
  
</Dialect>
Note
In order to control tags order you need to create a custom dialect and describe all tags in a Message or Repeating group in the necessary order with the override mode.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.4">
  
        <!-- Advertisement is now only text info. -->
        <Message type="7" mode="override">
            <Field tag="58" name="Text" isRequired="true"/>
        </Message>
  
        <Message type="S">
            <!-- Completely redefines group to do not use Legs component. -->
            <Group numberOfInstancesTag="555" name="NoLegs" mode="override">
                <!-- Now tag #588 will be as leading tag in the group. -->
                <Field tag="588"/>
                <Field tag="8000"/>
            </Group>
        </Message>
  
    </FIX>
  
</Dialect>
Using Component Block

You can use the <Component> entity to describe common components in order to use it in all necessary messages. There is need to add the name attribute by which you can identify a particular component and use it in all necessary messages. Component Block can contain <Field> , <Group> and even other <Component> entities. When a Component Block occurs in a message description, all described fields from this component are considered as fields directly described in this message.

Example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
<?xml version="1.0" encoding="utf-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX version="4.0">
  
        <!-- Header component description. -->
        <Component name="Header">
            <Field tag="8" name="BeginString" isRequired="true" />
            <Field tag="9" name="BodyLength" isRequired="true" />
            <Field tag="35" name="MsgType" isRequired="true" />
            <Field tag="49" name="SenderCompID" isRequired="true" />
            <Field tag="56" name="TargetCompID" isRequired="true" />
            <Field tag="115" name="OnBehalfOfCompID" />
            <Field tag="128" name="DeliverToCompID" />
            <Field tag="90" name="SecureDataLen" />
            <Field tag="91" name="SecureData" />
            <Field tag="34" name="MsgSeqNum" isRequired="true" />
            <Field tag="50" name="SenderSubID" />
            <Field tag="57" name="TargetSubID" />
            <Field tag="116" name="OnBehalfOfSubID" />
            <Field tag="129" name="DeliverToSubID" />
            <Field tag="43" name="PossDupFlag" />
            <Field tag="97" name="PossResend" />
            <Field tag="52" name="SendingTime" isRequired="true" />
            <Field tag="122" name="OrigSendingTime" />
        </Component>
  
        <!-- Trailer component description. -->
        <Component name="Trailer">
            <Field tag="93" name="SignatureLength" />
            <Field tag="89" name="Signature" />
            <Field tag="10" name="CheckSum" isRequired="true" />
        </Component>
  
        <Message type="A">
            <!-- Using fields from Header component. -->
            <Component name="Header" />
            <Field tag="98" name="EncryptMethod" isRequired="true" />
            <Field tag="108" name="HeartBtInt" isRequired="true" />
            <Field tag="95" name="RawDataLength" />
            <Field tag="96" name="RawData" />
            <!-- Using fields from Trailer component. -->
            <Component name="Trailer" />
        </Message>
    </FIX>
  
</Dialect>
Using QuickFIX Dialect

The engine supports dialect in the QuickFIX xml format, so you can use it as it is. Identifier (id) will be generate when a QuickFIX dialect is used. This id is equal to the dialect file name without extension. Therefore, the QuickFIX dialect becomes a session-level.

Samples

Check also Samples :: Dialect for paired live applications with dialect support.

SSL Encryption

The SSL (Secure Sockets Layer) support helps to make FIX communication secure.

The SSL support is implemented using Java Security / JSSE (Java Secure Socket Extension) implementation. Learn more here about Java Security and JSSE.

The SSL configuration procedure is slightly different for FIX acceptor and initiator sessions.

Creating SSL Context

Before establishing SSL-secured FIX session the SSL context needs to be created. One of the methods follows.

34
35
final SSLContext sslContext = SslContextFactory.getInstance(keyStoreName, trustStoreName, keyPassword,
        keyStorePassword, trustStorePassword);

Parameters:

  • keyStoreName — key store resource name;
  • trustStoreName — trust store resource name; can be set to null if you don't want to validate counterparty certificate;
  • keyPassword — key pair password used during key pair generation;
  • keyStorePassword — key store password;
  • trustStorePassword — trust store password; can be set to null if the trust store name is set to null.

The key and trust stores are loaded as described in the Unified Resource Loading.

Acceptor with SSL

The acceptor SSL configuration is performed on the engine level via the following method.

38
39
40
41
42
43
final SSLContext sslContext = SslContextFactory.getInstance(keyStoreName, trustStoreName, keyPassword,
        keyStorePassword, trustStorePassword);
Engine.setSSLContext(sslContext);
final Engine engine = Engine.init(11011);
final Session session = new Session("SellSide", "BuySide", Version.FIX40);
session.logonAsAcceptor();

Make sure that you configure SSL before doing engine init.

Initiator with SSL

The initiator SSL configuration is performed on the session level via the following method.

38
39
40
41
42
final Session session = new Session("BuySide", "SellSide", Version.FIX40);
final SSLContext sslContext = SslContextFactory.getInstance(keyStoreName, trustStoreName, keyPassword,
        keyStorePassword, trustStorePassword);
session.setSSLContext(sslContext);
session.logonAsInitiator("localhost", 11011);

Make sure that you configure SSL before doing session logon.

SSL Configuration via System Properties

The SSL encryption can be configured via system properties instead of the

biz.onixs.fix.engine.SslContextFactory

In this case the default SSL context can be used:

36
37
38
39
40
41
42
43
System.setProperty("javax.net.ssl.keyStore", keyStoreName);
System.setProperty("javax.net.ssl.keyStorePassword", keyStorePassword);
System.setProperty("javax.net.ssl.trustStore", trustStoreName);
System.setProperty("javax.net.ssl.trustStorePassword", trustStorePassword);
final Session session = new Session("BuySide", "SellSide", Version.FIX40);
final SSLContext sslContext = SSLContext.getDefault();
session.setSSLContext(sslContext);
session.logonAsInitiator("localhost", 11011);
System Property Description
javax.net.ssl.keyStore Key store file location.
javax.net.ssl.keyStorePassword Password to access the private key from the key store file specified by "javax.net.ssl.keyStore". This password is used twice: to unlock the key store file (store password), and to decrypt the private key stored in the key store (key password).
javax.net.ssl.trustStore Trust store file location.
javax.net.ssl.trustStorePassword Password to unlock the key store file (store password) specified by "javax.net.ssl.trustStore".
javax.net.ssl.trustStoreType For Java keystore file format, this property has the value jks (or JKS). You do not normally specify this property, because its default value is already jks.
javax.net.debug To switch on logging for the SSL/TLS layer, set this property to ssl.
There is no way to pass the key password via system property. It can be equal to the key store password though.
Key and Certificate Management

To perform key and certificate management the keytool can be used. This tool is a part of JDK package and the complete documentation can be found in the JDK documentationTools & Tool APIsSecurity.

Below you can find the simplified pre-steps to do in order to organize secure communication between two parties.

Key Pair Generation

Generate the key pair for the "SellSide" and "BuySide":

keytool -genkeypair -dname "cn=SellSide, ou=FixEngine, o=OnixS, c=US" -alias sellside -keypass pass11 -keystore sellside.keystore.bin -storepass pass11 -validity 360

keytool -genkeypair -dname "cn=BuySide, ou=FixEngine, o=OnixS, c=US" -alias buyside -keypass pass22 -keystore buyside.keystore.bin -storepass pass22 -validity 360

Certificate Export

Export the certificate of "SellSide" and "BuySide":

keytool -exportcert -alias sellside -file sellside.cer -keystore sellside.keystore.bin -storepass pass11

keytool -exportcert -alias buyside -file buyside.cer -keystore buyside.keystore.bin -storepass pass22

By default, the certificates are exported in the binary format. However, you can select BASE64-encoded printable format - Internet RFC 1421. Sometimes it is referred as PEM - Privacy Enhanced Mail. Use the "-rfc" key then.

keytool -exportcert -rfc -alias sellside -file sellside.pem -keystore sellside.keystore.bin -storepass pass11

keytool -exportcert -rfc -alias buyside -file buyside.pem -keystore buyside.keystore.bin -storepass pass22

Certificate Import

Import the certificate of "BuySide" into the "SellSide" trust store and vice versa:

keytool -importcert -noprompt -alias buyside -file buyside.cer -keystore sellside.truststore.bin -storepass pass33

keytool -importcert -noprompt -alias sellside -file sellside.cer -keystore buyside.truststore.bin -storepass pass44

By default, the certificates are imported in the binary format. However, you can select BASE64-encoded printable format - Internet RFC 1421. Sometimes it is referred as PEM - Privacy Enhanced Mail. Use the "-rfc" key then.

keytool -importcert -noprompt -rfc -alias buyside -file buyside.pem -keystore sellside.truststore.bin -storepass pass33

keytool -importcert -noprompt -rfc -alias sellside -file sellside.pem -keystore buyside.truststore.bin -storepass pass44

Result

As a result we have two pairs of key store and trust store for each party to establish secure communication:

  • (sellside.keystore.bin, sellside.truststore.bin)
  • (buyside.keystore.bin, buyside.truststore.bin)

This is what was used for the samples.

Samples

Check also Samples :: SSL for paired live applications with SSL support.

Sharing Engine Instance

The FIX engine class is implemented as a singleton:

biz.onixs.fix.engine.Engine

Thus you can't have more than a single instance of the engine in the same JVM. If you have several applications based on the engine and you want to run them in the same JVM then you need to share the engine instance between all of them. It means engine configuration must include everything to satisfy every application needs.

Assume 1st application uses the FIX dialect defined in the "sharing-engine-dialect-1.xml".

1
2
3
4
5
6
7
8
<?xml version="1.0" encoding="UTF-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX id="dialect-1" version="5.0_SP2">
    </FIX>
</Dialect>

And assume 2nd application uses the FIX dialect defined in the "sharing-engine-dialect-2.xml".

1
2
3
4
5
6
7
8
<?xml version="1.0" encoding="UTF-8"?>
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  
    <FIX id="dialect-2" version="5.0_SP2">
    </FIX>
</Dialect>

In order to run both applications in the same JVM at the same time we need to pass both dialect files to the engine during initialization.

25
26
27
28
29
30
31
32
33
34
35
36
37
38
public class SharingEngineMain {
    public static void main(String[] args) {
        final EngineSettings settings = new EngineSettings();
        settings.setDialects("site/sharing-engine-dialect-1.xml | site/sharing-engine-dialect-2.xml");
        final Engine engine = Engine.init(settings);
        //
        final SharingEngineApp1 app1 = new SharingEngineApp1();
        final SharingEngineApp2 app2 = new SharingEngineApp2();
        app1.work();
        app2.work();
        //
        engine.shutdown();
    }
}

The 1st application will use the pre-initialized engine instance.

25
26
27
28
29
30
public class SharingEngineApp1 {
    public void work() {
        final Version version = Version.getById("dialect-1");
        final Session session = new Session("SenderCompId-1", "TargetCompId-1", version);
    }
}

And the 2nd application will use the same pre-initialized engine instance.

25
26
27
28
29
30
public class SharingEngineApp2 {
    public void work() {
        final Version version = Version.getById("dialect-2");
        final Session session = new Session("SenderCompId-2", "TargetCompId-2", version);
    }
}

Best Practices

This section summarizes our findings and recommends best practices to tune the different layers of Java FIX Engine.

Please note that the exact benefits and effects each of these configuration choices, that will be highly dependent upon the specific applications and workloads. In this way, so we strongly recommend experimenting with the different configuration options with your workload before deploying them in a production environment.

Low Latency Best Practices

Session tuning-up
Selecting right storage

Using MemoryBasedStorage instead of FileBasedStorage boosts performance since FIX messages are stored directly in memory. Alternatively, it's possible to use a custom pluggable storage (using PluggableStorage) which does nothing on FIX message-related operations as soon as no resend requests are to be supported.

It is also recommended to use AsyncFileBasedStorage which provides same functionality as FileBasedStorage with good-performance.

Manipulating Threads Affinity

By default, threads are used by FIX session to send and receive FIX messages; they can be executed on any of available processors/cores. Specifying CPU affinity for each session thread may give a significant performance boost:

biz.onixs.fix.engine.Session.setReceivingThreadAffinity(int[])biz.onixs.fix.engine.Session.setSendingThreadAffinity(int[])
Using send spinning timeout
biz.onixs.fix.engine.Session.setSendSpinningTimeout(long)

method can be used to decrease the latency of the data sending. If the value is zero (by default) and the outgoing message cannot be sent immediately it is placed to the outgoing queue. If the value greater than zero, the Session.send(..) method waits for the socket sending buffer availability in the spin loop mode before placing the message to the outgoing queue (to be sent later by the sending thread). The method parameter specifies the spin loop period in nanoseconds. Session.setSendSpinningTimeout(..) using makes sense when your session sends FIX messages frequently in this case waiting in the loop is cheaper than the thread context switch. However please note that the spin wait increases the CPU usage so the spin wait period should not be too long.

Using session warmup
biz.onixs.fix.engine.Session.warmUp(Message)

method can be used to warm up the sending path. It makes sense if your session sends FIX messages infrequently, in this case the sending path and associated data structures will not be in a cache and this can increase the send latency. You can periodically (a recommended period is 500 usec and less) call the Session.warmUp(..) to avoid cache misses and keep the sending path fast.

Reusing FIX message instance for incoming messages

Object creation is an expensive operation in Java, with impact on both performance and memory consumption. The cost varies depending on the amount of initialization that needs to be performed, when the object is to be created. OnixS Java FIX Engine exposes an ability to reuse the incoming message by Session. We highly recommend to turn on SessionInboundMessageReuse to minimize the excess object creation and garbage collection overhead.

Updating Engine configuration
Disabling Resend Requests functionality

If no messages are re-sent on counterparty's Resend Request messages, then it is possible to set the Resending.QueueSize to zero to increase overall performance.

Disabling FIX messages validation

Validation significantly reduces performance due to miscellaneous checks, performed on each FIX message.

Optimizing FIX dictionaries

When FIX Message is constructed, the space for all the fields, that are defined for the message, is allocated. The latest FIX specifications define a lot of fields for each message type. Even when most of fields are not used, FIX Engine reserves the space for them, and this has negative effect on FIX Engine's performance.

Editing dictionaries descriptions and excluding messages and fields (especially repeating groups), which are not used by application, have direct influence onto FIX messages processing speed and thus decrease general latency of FIX message related processing.

Manipulating FIX messages

While constructing FIX messages, each time message, that has to be sent, is not efficient, because it involves memory allocation and subsequent destruction. The

biz.onixs.fix.parser.Message

class provides ability to reuse a message of a particular type. It exposes the

biz.onixs.fix.parser.Message.reset()

method, which wipes out all the FIX fields, assigned to the given instance. This method does not deallocate all memory occupied, it just brings the message to the initial state as it was just constructed. However, after the previously allocated memory is reused, thus, the second use of the object allows to setup FIX fields faster.

Reusing FIX message instance

A common strategy is to use the same message instance for sending multiple times. As soon as a message of a certain type is constructed, it is updated with common fields for the subsequent re-use. Afterwards, before sending it to the counterparty, it is updated with mutable fields (like price or quantity) exactly for the case, and sent to the counterparty.

As mutable fields are updated, each time message is about to be sent to counterparty.

High Throughput Best Practices

Session Tuning-up

There are many settings in the Session class and this section covers only subset that relates to high-throughput workloads.

Selecting Right Storage

Please see corresponding section in Low Latency Best Practices.

Manipulating Threads Affinity

Please see corresponding section in Manipulating Threads Affinity.

Reusing FIX message instance for incoming messages

Please see corresponding section in Low Latency Best Practices.

Updating Engine configuration
Disabling Resend Requests functionality

Please see corresponding section in Low Latency Best Practices.

Disabling FIX messages validation

Please see corresponding section in Low Latency Best Practices.

Optimizing FIX dictionaries

Please see corresponding section in Low Latency Best Practices.

Transport Layer and the TCP/IP Stack

There are many options in the protocol stack that can effect the efficiency of the data delivery. You must understand the characteristics of the version of the stacks you are running, and that they are compatible with the versions and options on the other stacks.

Enable the Nagle's algorithm (disable TCP_NODELAY)

Nagle’s algorithm is very useful for minimizing network overhead by concatenating packets together. We strongly recommend to enable Nagle’s algorithm for high-throughput workloads. Please see Connection.TcpNoDelay setting in the Settings section for more details.

Reducing Garbage Collection Overhead

Session Tuning-up
Reusing FIX message instance for incoming messages

Please see corresponding section in Low Latency Best Practices.