OnixS Java FIX Engine is a simple fully Java compliant tool that will FIX-enable applications written in Java.
The engine provides the following services:
The following image is a high-level overview of the Java FIX Engine architecture:
The typical way of using the engine is as follows:
Check also following chapters:
Initializing the engine
To initialize the engine services the biz.onixs.fix.engine.Engine.init() method must be used.
This method must be called before all other methods.
The engine configuration is set during the initialization phase. See Configuring the Engine for details.
Shutting down the engine
To shutdown the engine services the biz.onixs.fix.engine.Engine.shutdown() method must be used.
No other engine methods must be called after this method.
Example
22 23 24 25 26 27 28 29 30 | import biz.onixs.fix.engine.Engine; public class EngineInitAndShutdown { public static void main(String[] args) { final Engine engine = Engine.init(); // App logic engine.shutdown(); } } |
As the FIX protocol evolves its new versions will be released to you to extend the current functionality.
Supported FIX versions are: FIX 4.0 - 5.0, 5.0 SP1, 5.0 SP2. biz.onixs.fix.dictionary.Version enum is used to identify them.
Example
21 22 23 24 25 26 27 28 29 30 | import biz.onixs.fix.dictionary.Version; import biz.onixs.fix.engine.Engine; public class VersionGetter { public static void main(String[] args) { final Engine engine = Engine.init(); final Version fixVersion = Version.getByNumber("4.2"); engine.shutdown(); } } |
A FIX session is defined as a bi-directional stream of ordered messages between two parties within a continuous sequence number series. It is represented by the biz.onixs.fix.engine.Session class.
Constructing Session
To create a session object the appropriate constructor is used.
Session Role
A session has one of two possible roles: Acceptor or Initiator.
Acceptor is the receiving party of the FIX session. It listens for the incoming connection on the pre-defined port. The acceptor has the responsibility to perform first level authentication and formally declare the connection request “accepted” through the transmission of an acknowledgment Logon message.
Initiator establishes the telecommunications link and initiates the session via the transmission of the initial Logon message.
To obtain the current role the biz.onixs.fix.engine.Session.getRole() method must be used.
Terminate Session
To terminate the session the biz.onixs.fix.engine.Session.dispose() method must be used.
Establishing Connection
To establish a FIX connection as Acceptor the biz.onixs.fix.engine.Session.logonAsAcceptor() method must be used.
To establish a FIX connection as Initiator the biz.onixs.fix.engine.Session.logonAsInitiator(String, int) method must be used.
Closing Connection
To disconnect the session the biz.onixs.fix.engine.Session.logout() method must be used.
This member implements a graceful closing of the FIX connection as defined by the Standard.
Note: The biz.onixs.fix.engine.Session.logonAsInitiator(String, int) and biz.onixs.fix.engine.Session.logout() methods are blocked until the Logon/Logout response is received. These methods have a timeout during which the response should be received. This timeout is equal to the heartbeat interval (by default 30 sec) + ReasonableTransmissionTime value (by default 20%) as percentage from the heartbeat interval. If the acknowledgment Logon/Logout message is not received during the timeout then the Logout message will be sent and the FIX connection will be closed.
Reconnection Facility
When a FIX session is in the active state and there are some network issues the engine will try to restore the FIX connection in accordance with ConnectionRetries.Number and ConnectionRetries.Interval settings.
Note: Reconnection facility works for already established connections. In other cases, you need to use the Sessions Scheduler component or manually handle biz.onixs.fix.engine.LinkErrorException or biz.onixs.fix.engine.TimeoutException exceptions of biz.onixs.fix.engine.Session.logonAsInitiator(String, int) method and try to connect again.
Example
29 30 31 32 33 34 35 36 37 38 39 | final Session acceptor = new Session("SenderCompID", "TargetCompID", Version.FIX40); final Session initiator = new Session("TargetCompID", "SenderCompID", Version.FIX40); acceptor.logonAsAcceptor(); // Sends the Logon message and waits for the acknowledgment Logon initiator.logonAsInitiator("localhost", engine.getSettings().getListenPorts()[0]); // Sends the Logout message and waits for the confirming Logout initiator.logout(); acceptor.logout(); // Free resources initiator.dispose(); acceptor.dispose(); |
Connection and Session Lifetime
A single FIX session can exist across multiple sequential (not concurrent) physical connections. Parties can connect and disconnect multiple times while maintaining a single FIX session. That is, once the biz.onixs.fix.engine.Session.logout() method is called, the FIX Connection is terminated, however, the FIX session life continues. It is possible to continue the same session later using either biz.onixs.fix.engine.Session.logonAsAcceptor() or biz.onixs.fix.engine.Session.logonAsInitiator(String, int) methods again. In other words, a FIX Session is comprised of one or more FIX connections.
See also
Understanding Session States, Connecting using Custom Logon Message.
Sending Messages to a Counterparty
To send a message to a counterparty, the biz.onixs.fix.engine.Session.send(Message) or biz.onixs.fix.engine.Session.send(FlatMessage) method must be used.
This method is asynchronous. As soon as a session is created it is possible to start sending messages via the session.
If the session is not established, the messages are stored in the session storage and will be sent in the replay to the resend request when the connection is established with the counterparty and the sequence numbers mismatch is detected (please see the Resending Messages page).
Receiving Messages from a Counterparty
To receive incoming application-level messages it is necessary to implement the biz.onixs.fix.engine.Session.InboundApplicationMessageListener and/or biz.onixs.fix.engine.Session.InboundApplicationFlatMessageListener interface.
And register the handler with the corresponding method biz.onixs.fix.engine.Session.setInboundApplicationMessageListener(InboundApplicationMessageListener) and/or biz.onixs.fix.engine.Session.setInboundApplicationFlatMessageListener(InboundApplicationFlatMessageListener)
To receive incoming session-level messages it is necessary to implement the biz.onixs.fix.engine.Session.InboundSessionMessageListener and/or biz.onixs.fix.engine.Session.InboundSessionFlatMessageListener interfaces.
And register the handler with the corresponding method biz.onixs.fix.engine.Session.setInboundSessionMessageListener(InboundSessionMessageListener) and/or biz.onixs.fix.engine.Session.setInboundSessionFlatMessageListener(InboundSessionFlatMessageListener).
Note: Both listeners flat and structured ones provide the same FIX message but different representations. The flat message listener is fired first.
Note: It is inefficient from the performance point of view to register for both types of messages.
Example
Structured message exchange:
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | private static class MessageProcessor implements Session.InboundApplicationMessageListener, Session.InboundSessionMessageListener { @Override public void onInboundApplicationMessage(final Object sender, final Session.InboundApplicationMessageArgs args) { System.out.println("Incoming application-level message: " + args.getMsg()); } @Override public void onInboundSessionMessage(final Object sender, final Session.InboundSessionMessageArgs args) { System.out.println("Incoming session-level message: " + args.getMsg()); } } public static void main(final String[] args) { final Engine engine = Engine.init(PORT); // final Session acceptor = new Session(SENDER_COMP_ID, TARGET_COMP_ID, VERSION); final MessageProcessor messageProcessor = new MessageProcessor(); acceptor.setInboundApplicationMessageListener(messageProcessor) .setInboundSessionMessageListener(messageProcessor); // final Session initiator = new Session(TARGET_COMP_ID, SENDER_COMP_ID, VERSION); // acceptor.logonAsAcceptor(); initiator.logonAsInitiator(HOST, PORT); // final Message order = createOrder(); initiator.send(order); |
Flat message exchange:
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | private static class FlatMessageProcessor implements Session.InboundApplicationFlatMessageListener, Session.InboundSessionFlatMessageListener { @Override public void onInboundApplicationFlatMessage(Object sender, Session.InboundApplicationFlatMessageArgs args) { System.out.println("Incoming application-level flat message: " + args.getMsg()); } @Override public void onInboundSessionFlatMessage(Object sender, Session.InboundSessionFlatMessageArgs args) { System.out.println("Incoming session-level flat message: " + args.getMsg()); } } public static void main(String[] args) { final Engine engine = Engine.init(PORT); // final Session acceptor = new Session(SENDER_COMP_ID, TARGET_COMP_ID, VERSION); final FlatMessageProcessor messageProcessor = new FlatMessageProcessor(); acceptor.setInboundApplicationFlatMessageListener(messageProcessor) .setInboundSessionFlatMessageListener(messageProcessor); // final Session initiator = new Session(TARGET_COMP_ID, SENDER_COMP_ID, VERSION); // acceptor.logonAsAcceptor(); initiator.logonAsInitiator(HOST, PORT); // final FlatMessage flatOrder = createFlatOrder(); initiator.send(flatOrder); |
Understanding Message Sequencing
A single FIX session can exist across multiple sequential (not concurrent) physical connections. Parties can connect and disconnect multiple times while maintaining a single FIX session. That is, once the
biz.onixs.fix.engine.Session.logout() method is called, the FIX Connection is terminated, however, the FIX session life continues.It is possible to continue the same session later using either biz.onixs.fix.engine.Session.logonAsAcceptor() or biz.onixs.fix.engine.Session.logonAsInitiator(String, int) methods again. In other words, a FIX Session is comprised of one or more FIX connections.
All FIX messages are identified by a unique sequence number (MsgSeqNum field) within the bounds of a single FIX session. Sequence numbers are initialized at the start of each FIX session starting at 1 (one) and increment throughout the session. Each session establishes an independent incoming and outgoing sequence series.
The engine automatically synchronizes sequence numbers when reconnecting during a FIX session on the base of the information that was previously stored in log files.
Manipulating Message Sequence Numbers
Monitoring sequence numbers enables parties to synchronize applications gracefully when reconnecting during a FIX session. The expected sequence number of the next incoming message can be get using the biz.onixs.fix.engine.Session.getInSeqNum() method. And set using the biz.onixs.fix.engine.Session.setInSeqNum(long) method. The sequence number of the next outgoing message can be get using the biz.onixs.fix.engine.Session.getOutSeqNum() method. And set using the biz.onixs.fix.engine.Session.setOutSeqNum(long) method.
Resetting inbound and outbound sequence numbers back to 1, for whatever reason, constitutes the beginning of a new FIX session. The biz.onixs.fix.engine.Session.resetLocalSequenceNumbers() method must be used to backup the previous log files and reset the sequence numbers to 1.
Note: Both biz.onixs.fix.engine.Session.setInSeqNum(long) and biz.onixs.fix.engine.Session.getOutSeqNum() properties can be used to set expected sequence numbers manually before a FIX connection is established. However, it is NOT recommended to use these properties to reset the message sequence numbers to 1 for the session. Instead, method biz.onixs.fix.engine.Session.resetLocalSequenceNumbers() must be used.
Some implementations of the FIX protocol expect that a FIX Session coincides with a FIX Connection. In other words, message sequence numbers must be reset back to 1 before each Logon messages exchange including reconnect.
To interact with such FIX venues, keepSequenceNumbersBetweenFixConnections parameter of session constructor should be set to false. In this case before the Logon messages exchanges sequence numbers are set back to 1.
See also
Resetting Message Sequence Numbers, Resetting Message Sequence Numbers via ResetSeqNumFlag Field.
The FIX session sets or updates specific fields in the outgoing messages automatically.
The following fields belong to the message header and trailer. They are mandatory for all FIX versions and all message types.
The following fields are set or updated only if they are configured explicitly for the session. They are applicable for all message types.
| Field | FIX Version |
|---|---|
| SenderSubID(50) | all |
| TargetSubID(57) | all |
| SenderLocationID(142) | since 4.1 |
| TargetLocationID(143) | since 4.1 |
| LastMsgSeqNumProcessed(369) | since 4.2 |
In general, a FIX message represents a sequence of fields whose values are associated with unique numbers (tags).
A common way of presenting a FIX message is called the tag-value FIX message format. In such a format all fields are presented as tag=value pairs and are delimited by special SOH symbol.
The following image depicts the structure of FIX message in tag-value format:
The engine provides two representation of the FIX message by the biz.onixs.fix.parser.Message and biz.onixs.fix.parser.FlatMessage classes.
FlatMessage class
The class to encapsulate basic services related to handling FIX messages. It does not require any dialect to be parsed from raw FIX message. It does not concern about whether the body (the payload) of a FIX message contains a sequence of tag/value pairs that resemble on a higher semantic level a repeating group. And does not support validation against a dialect.
It allows adding, set and get FIX message fields associated with a tag number or index in the FlatMessage. The order of fields will be kept as they are received.
In order to perform validation and/or accessing repeating groups, it can be converted into a Message class with corresponding FIX version/dialect. Multiple dialects can be used.
Message class
The class to encapsulate all services related to handling FIX messages. It has a structure based on the type and the version of the FIX protocol/dialect which allows performing validation and access to repeating groups.
Constructing Blank Message
To create a blank flat message object, the biz.onixs.fix.parser.FlatMessage.FlatMessage() constructor must be used, an empty flat message will be initialized and ready for interaction.
To create a blank structured message object, the biz.onixs.fix.parser.Message.create(String, Version) method must be used.
It initializes a structured message of the specified type and a FIX protocol version with no fields presented, except a couple of service fields like those, which identifies type and version of FIX protocol to which constructed message belongs to.
Note: By reason of the internal design, you cannot change the message type of the created structured message and perform the validation correctly, when a FIX message is created with a certain type then the particular message structure is associated with this message to perform validation and determine undefined fields, so you need to create a new message when you want to change the message type.
Converting Message from and to Raw (Tag-Value) Format
The biz.onixs.fix.parser.FlatMessage.FlatMessage(byte[], int) and biz.onixs.fix.parser.Message.Message(byte[], int) constructors and many more are available to construct flat and structured message instances correspondingly from its raw (tag-value) representation.
The biz.onixs.fix.parser.FlatMessage.assemble(ByteBuffer) and biz.onixs.fix.parser.Message.assemble(ByteBuffer) methods are available to build a tag-value presentation of a message instance.
Also, several overloads of toString() methods are available which simplify debugging and monitoring.
Note: While building the tag-value presentation of a message instance BodyLength(9) and Checksum(10) tags will be updated.
Validating Message
In addition to the presence of a particular field in the message of a certain type, the FIX protocol also defines whether the presence of fields are strictly required in the message, whether they are required under certain conditions, or whether they are optional. To check for the presence of all required fields, the biz.onixs.fix.parser.Message.validate() method is must be used.
See also Required Fields. In the engine terminology, this process is called message validation.
Note: Validation is available only for the structured message and not for the flat one.
Message life cycle
Sometimes there is a need to use message after it was sent to a session. As soon as the message was sent, the engine will not use it later, so it can be used again; for example, it will be sent to other sessions.
Note: The message object is not thread-safe, so it cannot be used in one thread while it is used in another.
Example
29 30 31 32 33 34 35 36 | // Create "New Order - Single", MsgType = "D" final Message order = Message.create("D", Version.FIX40); // Another way to create FIX message final String rawMsg = "8=FIX.4.0\u00019=86\u000135=D\u000149=0\u000156=0\u000134=1\u000152=99990909-17:17:17\u0001" + "11=90001008\u000121=1\u000155=IBM\u000154=1\u000138=10\u000140=1\u000159=0\u000110=191\u0001"; final Message order2 = new Message(rawMsg); // Validate created message order2.validate(); |
A FIX message represents a sequence of fields whose values are associated with unique numbers (tags). Such an interpretation treats a message as a collection of values associated with tag numbers. For this reason, the primary approach in handling message fields is similar to managing associative collections.
Adding Field Into Message
To associate a field with a value in a structured message, the biz.onixs.fix.parser.Message.set(int, String) method must be used. If there is no field of a given tag number available in the message, this member creates an association. If the field already exists, this member updates its value with the new value.
To associate a field with a value in a flat message, the biz.onixs.fix.parser.FlatMessage.add(int, String) method must be used. It will add a tag-value pair at the end of the message no matter it exists or not.
OR
The biz.onixs.fix.parser.FlatMessage.set(int, String) method must be used. If there is no field of a given tag number it will throw an exception. If the field already exists, this member updates its value with the new value.
Accessing Field Value
To get a field value the biz.onixs.fix.parser.Message.get(int) method can be used for structured message. The biz.onixs.fix.parser.FlatMessage.get(int) and/or biz.onixs.fix.parser.FlatMessage.getByIndex(int) methods for the flat message (where the indexes can be obtained using the biz.onixs.fix.parser.FlatMessage.findIndex(int) method).
Note: Implementations of all field value getters are avoiding the creation of internal objects as much as it is possible as object creation is an expensive operation in Java.
Accessing Field Value Directly
The biz.onixs.util.ValuePtr class has been implemented in order to provide direct access/pointer to the region in a received message of value associated with the field.
Example usage which avoids a string allocation:
31 32 33 34 35 36 37 38 39 40 41 | final Message order = Message.create("D", Version.FIX40); final ValuePtr clOrdIDToLookForPtr = new ValuePtr("client_order_id"); // one-time ValuePtr valuePtr = new ValuePtr(); // one-time // many times something like: if (order.get(FIX40.Tag.ClOrdID, valuePtr) && clOrdIDToLookForPtr.equals(valuePtr)) { System.out.println("Found!"); } else { System.out.println("NOT found"); } |
Removing Field From Message
To remove a field value the biz.onixs.fix.parser.Message.remove(int) method can be used for structured message. The biz.onixs.fix.parser.FlatMessage.remove(int) and/or biz.onixs.fix.parser.FlatMessage.remove() methods for the flat message.
Example
29 30 31 | final Message order = Message.create("D", Version.FIX40); order.set(11, "client_order_id"); final String clOrdID = order.get(11); |
Named Tags Constants
The biz.onixs.fix.tag.Tag class contains the constants for the tag values.
Also there are similar classes defined for each FIX version biz.onixs.fix.tag.FIX40.Tag etc. The use of these constants makes the source code more readable.
Example
30 31 32 33 | final Message order = Message.create("D", Version.FIX40); order.set(Tag.ClOrdID, "90001008") .set(Tag.Side, "1") .set(Tag.TimeInForce, "0"); |
Iterating over Fields of Message
To iterate over all fields of a FIX message please use the biz.onixs.fix.parser.Message.iterator() method for the structured message.
This iterator does not iterate over fields from a repeating group. To iterate over fields of the repeating group you need to get the corresponding biz.onixs.fix.parser.GroupInstance object from the repeating group, please see the FIX Repeating Group.
biz.onixs.fix.parser.Message class provides iterator over raw message, please use the biz.onixs.fix.parser.Message.iteratorOverRawMsg() method.This iterator allows iterating over all fields including repeating groups of received raw FIX message as it is. Before any modifications to FIX message the set of fields will be in the same order as they are received/parsed and will include duplicates. Compared to the above-mentioned iterator it is slower.
To iterate over all fields of the flat message use the biz.onixs.fix.parser.FlatMessage.iterator() method.
Summary
| Message type | Advantages | Disadvantages |
|---|---|---|
| Message | - Validation can be performed against a dialect. | - Message type(i.e. tag# 35) and FIX protocol version(i.e. tag #8) cannot be changed for already created message. |
| - Provides methods to access a repeating group. | - Changes the order of fields after vice versa conversion from raw message according to dialect file specified. | |
| - Get/Set values using the associated tag number will be performed in constant time. | - It is not possible to get all values of duplicated fields. | |
| - It is not possible to add duplicated fields. | ||
| FlatMessage | - Any field of the message can be changed including message type(i.e. tag# 35) and FIX protocol version(i.e. tag #8). | - No validation can be performed only basic tag=value format will be checked. |
| - Keeps the order of fields as they are in the original raw message after vice versa conversion. | - Get/Set values using the associated tag number will be performed in linear time but it is possible to get the corresponding index for multiple access in constant time. | |
| - Not concern about FIX message body/payload. | ||
| - Provides access to all values of duplicated fields. | ||
| - Duplicated fields can be added. |
See also
An exception is used as a fundamental error-reporting mechanism. In the event of any error the biz.onixs.fix.engine.EngineException exception is thrown.
The pre-defined license file name is “OnixS.lic”. The license file is loaded during engine initialization as described in the Unified Resource Loading.
The default license file name can be changed using LicenseFile parameter in the Properties File or via biz.onixs.fix.engine.EngineSettings.setLicenseFile(String) method.
The following table explains different options to keep the license file.
| License File Location | License File Name | LicenseFile Property | Note |
|---|---|---|---|
| application classpath | OnixS.lic | no change required | |
| current directory | *.lic | no change required | Any file with the “.lic” extension in the current directory is checked automatically. |
| user home directory | *.lic | no change required | Any file with the “.lic” extension in the user home is checked automatically. |
| application classpath | AltName.ext | AltName.ext | |
| absolute path “/foo/bar” | AltName.ext | /foo/bar/AltName.ext | |
| relative path “foo/bar” | AltName.ext | foo/bar/AltName.ext |
There is a number of parameters (settings) which control the behavior of the engine. biz.onixs.fix.engine.EngineSettings class provides the way to change settings of the engine programmatically. However, the default settings of the engine can also be changed using an external configuration file.
The engine is configured during the initialization phase. The initialization method is overloaded. Thus, there are multiple ways to pass the configuration.
The engine settings are taken from the properties.
27 28 | final PropertyBasedSettings settings = new PropertyBasedSettings("site/engine.properties"); final Engine engine = Engine.init(settings); |
The properties file is loaded as described in the Unified Resource Loading.
The configuration file example follows.
1 2 3 4 5 | Dialect = site/dialect.xml Log.InboundMessages = true Validate.NumberOfRepeatingGroupInstances = false Validate.RequiredFields = true Validate.UnknownFields = true |
The engine settings are available as the class properties.
27 28 | final EngineSettings settings = new EngineSettings(); final Engine engine = Engine.init(settings); |
The settings can be changed before the initialization method call.
The configuration parameter (key) is case-sensitive. The list of available parameters follow.
Please note the values should be specified without quotation marks.
| Setting | Description | Default value |
|---|---|---|
| LicenseFile | Specifies the name of the license file resource. | OnixS.lic |
| Versions | Specifies the FIX dictionaries to load during the engine initialization. The dictionary names are separated with the ‘|’ character. | FIX.4.0 | FIX.4.1 | FIX.4.2 | FIX.4.3 | FIX.4.4 | FIX.5.0 | FIX.5.0_SP1 | FIX.5.0_SP2 |
| Dialect | Specifies the location of one or several files with the FIX dialects definition. The file names are separated with the ‘|’ character. The dialect files are loaded during engine initialization as described in the Unified Resource Loading. | empty |
| ListenPort | FIX engine listens on this port(s) for incoming connections. If it is set to “0” then only session-initiators can be created. If it is set to “-1” then the telecommunication level is disabled and only message parsing/assembling can be used. Use comma delimited list if more than one listen port is required. It is important to avoid using ports used by other services. | 0 |
| Log.Directory | Inbound/outbound messages and session's state data are stored in this directory. | FixEngineStorage |
| Log.InboundMessages | Option to store inbound messages in the session storage. | true |
| Log.OutboundMessages | Option to store outbound messages in the session storage. | true |
| Log.BeforeSending | Option to store outbound messages before sending. | true |
| AsyncFileStorage.QueueSize | This parameter sets asynchronous operations queue size for session storage. It is effective for biz.onixs.fix.engine.storage.SessionStorageType.AsyncFileBased session storage type only. | 2000 |
| AsyncFileStorage.WriteErrorRetriesNumber | This parameter sets the number of retries in case an I/O error occurs during storing messages into a file. It is effective for biz.onixs.fix.engine.storage.SessionStorageType.AsyncFileBased session storage type only. | 0 |
| Validate.InboundMessages | Option to validate inbound messages. | false |
| Validate.OutboundMessages | Option to validate outbound messages. | false |
| Validate.NumberOfRepeatingGroupInstances | Option to validate the declared number of repeating group instances is equal to the actual one. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | false |
| Validate.RequiredFields | Option to validate the presence of required fields. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | false |
| Validate.UnknownFields | Option to validate the presence of unknown fields. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | false |
| Validate.EmptyFieldValues | Option to fail validation on empty field value. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | false |
| Validate.FieldValues | Option to validate field values in accordance with the FIX protocol or its FIX Dialect. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | false |
| Validate.DuplicatedField | Option to validate the presence of fields which appears more than once. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | false |
| Validate.UnknownMessage | In case the unknown message validation is disabled, it will allow parsing an unknown message using the message standard skeleton. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | true |
| Validate.CheckSum | Option to validate CheckSum field value with the actual checksum of the raw message received. It is applied to the inbound and outbound messages depending on the Validate.InboundMessages and Validate.OutboundMessages values. | true |
| Connection.Mode | The way in which the engine or particular session will work with sockets. Available values can be found here: biz.onixs.fix.engine.ConnectionMode | DEDICATED_THREADS |
| ConnectionRetries.Interval | The initial time interval between the attempts to restore the telecommunication link, in milliseconds. | 90000 |
| ConnectionRetries.Number | Number of attempts to restore the telecommunication link. The range is [0..MAX_INT]. | 3 |
| Connection.TcpNoDelay | Option to improve latency at the expense of message throughput. | true |
| Connection.TcpSendBufferSize | The size of the TCP buffer allocated to FIX connection for sending data, in bytes. | 65535 |
| Connection.TcpReceiveBufferSize | The size of the TCP buffer allocated to FIX connection for receiving data, in bytes. | 65535 |
| Connection.OutputQueueMaxSize | The maximum size of the internal output queue allocated for a FIX connection, in bytes. | 512*1024*1024 |
| Connection.SendSpinningTimeout | The send spinning timeout (in nanoseconds) of the biz.onixs.fix.engine.Session.send(Message) method to wait for the socket sending buffer availability in the spin loop mode before placing the message to the outgoing queue (to be sent later by the sending thread). | 0 |
| Connection.ReceiveSpinningTimeout | The non-blocking receive spinning timeout (in nanoseconds) before the receiving thread enters into the blocking wait mode. | 0 |
| Connection.PerfPreference.ConnectionTime | Socket performance preference for connection time. | 1 |
| Connection.PerfPreference.Latency | Socket performance preference for latency. | 0 |
| Connection.PerfPreference.Bandwidth | Socket performance preference for bandwidth. | 2 |
| ThreadPoolSize | The size of the shared thread pool that is used in the thread pool connection mode. | 1 |
| ThreadPoolSpinningTimeout | The thread pool spinning timeout (in nanoseconds) before a thread of thread pool enters into the blocking wait mode for an operation. | 0 |
| ReasonableTransmissionTime | The reasonable transmission time as % from heartBtInt field value. When either end of the connection has not received any data for (HeartBtInt * (100 + ReasonableTransmissionTime)/100) seconds, it will transmit a Test Request message. If there is still no heartbeat message received after (HeartBtInt * (100 + ReasonableTransmissionTime)/100) seconds then the connection should be considered lost and corrective action be initiated. | 20 |
| SessionListenerDelayWarningThreshold | The limit of time to spend in the session listener code. If the listener takes longer then the warning is generated. In milliseconds. | 1000 |
| Engine.DefaultHeartbeatInterval | The default heartbeat interval in seconds. | 30 |
| SessionEventArgReuse | Option to re-use session event argument objects in each session. | false |
| Session.InboundMessageReuse | Option to use the same message object for the incoming messages. In case of re-using mode the message object can be operated only until control is returned from the inbound message listener. | false |
| Session.SendLogoutOnException | Option to send the logout message before dropping the telecommunication link in case of exception during the processing of incoming FIX message. | true |
| CheckCompIdsMatch | Option to check that SenderCompId and TargetCompId of every inbound FIX message match session values. | false |
| ProcessLogonNextExpectedMsgSeqNum | Option to process NextExpectedMsgSeqNum(789) field in Logon(A). | false |
| Resending.QueueSize | Number of sent messages that are available for resending on counterparty's Resend Request(2) message. | 1000 |
| Session.UpdateSendingTime | If this option set to true then the SendingTime(52) field in the outgoing messages is always updated. If this option set to false then the SendingTime(52) field is set only if it is not set already. | true |
| Session.UpdateHeaderFields | If this option set to true then common header fields in the outgoing messages are always updated. If this option set to false then the common header fields are set only if they are not set already. The fields are SenderCompID(52), TargetCompID(56) and others. | true |
| Session.ResendRequestMaximumRange | The maximum number of messages to be requested in one Resend Request(2) message. For no limits use 0. | 0 |
| Session.SpecifyLastMsgSeqNumProcessed | Option to specify the LastMsgSeqNumProcessed(369) field on every message sent. | false |
| EngineShutdownHook | Option to shutdown engine automatically on JVM shutdown. | false |
| Connection.OutStreamDelayWarningThreshold | Option to specify the output stream write delay threshold after which the warning is logged. In milliseconds. | 1000 |
| UserSessionLevelMessageWarning | Option to specify whether to warn if user sends session level message. | true |
| Acceptor.SendLogoutOnInvalidLogon | Option to send a Logout(5) message, before dropping the telecommunication link in reply to an invalid logon attempt. | false |
| Session.ResendRequestLogic | Session resend request logic. Available values can be found here: biz.onixs.fix.engine.ResendRequestLogic | STANDARD_FIX |
| Session.MessageMode | Session message mode. Available values can be found here: biz.onixs.fix.engine.MessageMode | MESSAGE |
| IgnoreFileBasedStorageIntegrityErrors | behaviour when File Based Storage errors are detected. If ‘true’ creates new session storage when errors are detected. If ‘false’ throws exception and prevents session against creating. | false |
| SpecifyApplVerIdField | Option to specify ApplVerID(1128) tag in all FIX messages when a FIX protocol version FIX 5.0 and above is used. | true |
| SslNeedClientAuth | Option to specify whether client authentication should be required. | false |
| DaemonAcceptorThread | Option to specify whether acceptor thread is daemon. | true |
By default, any external resource is looked for at the following places and in the following order (if other is not set explicitly):
If it can't be found at the 1st location then the 2nd one is checked, etc.
For example, you can put a dialect file to the classpath or current directory and it will be located by the engine.
This section describes venue specific settings, which should be used in work with a particular venue specific counterparty.
CME Globex implemented secure authentication for iLink and Drop Copy sessions on Convenience Gateway (CGW) and Market Segment Gateway (MSGW). The new logon procedure secures the client system logon with:
Customer identity verification - a client system logon request will be signed with security credentials issued and validated by CME Group.
Message confidentiality and integrity - to credential the logon message, the client system sends a keyed-hash message authentication code (HMAC) generated from a combination of the logon FIX tag values. When CME Globex receives the logon message, it uses the identical inputs to calculate the HMAC value to validate against the logon request. If the values do not match, CME Globex rejects the logon.
Customers must create secure key pairs for iLink and Drop Copy Sessions in the CME Customer Center.
CME Secure Logon requires to add several specific tags to Logon message, so our FIX Engine is ready to implement Secure Logon procedure. We have updated our CME Trading Client sample to help to implement Secure Logon.
Implementing Secure Logon consist of two parts: creating canonical request string and calculating HMAC. Canonical request string is just a union of several tags from Logon message. Please note, that Session fills values of some fields like SendingTime right before message sent, so its values can be taken from biz.onixs.fix.engine.Session.OutboundSessionMessageListener.onOutboundSessionMessage(Object, OutboundSessionMessageArgs) callback. To calculate HMAC a standard Java cryptographic classes are used.
Example
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | public void onOutboundSessionMessage(Object o, Session.OutboundSessionMessageArgs args) { if (FIX42.MsgType.Logon.equals(args.getMsg().getType())) { Message customLogon = args.getMsg(); if (accessKeyID != null && secretKey != null) { customLogon.set(FIX42.Tag.EncodedTextLen, accessKeyID.length()) .set(FIX42.Tag.EncodedText, accessKeyID) .set(encryptedPasswordMethodTag, "CME-1-SHA-256") .remove(FIX42.Tag.EncryptMethod); String canonicalRequest = createCanonicalRequest(customLogon, session.getHeartBtInt()); String hash = calculateHmac(canonicalRequest, secretKey); customLogon.set(encryptedPasswordLenTag, hash.length()) .set(encryptedPasswordTag, hash); } } } private String createCanonicalRequest(Message customLogon, int heartbeatInterval) { StringBuilder sb = new StringBuilder(); char delimiter = '\n'; sb.append(session.getOutSeqNum()).append(delimiter); sb.append(session.getSenderCompID()).append(delimiter); sb.append(session.getSenderSubID()).append(delimiter); sb.append(customLogon.get(FIX42.Tag.SendingTime)).append(delimiter); sb.append(session.getTargetSubID()).append(delimiter); sb.append(heartbeatInterval).append(delimiter); sb.append(session.getSenderLocationID()).append(delimiter); sb.append(customLogon.get(FIX42.Tag.LastMsgSeqNumProcessed)).append(delimiter); sb.append(customLogon.get(tradingSystemNameTag)).append(delimiter); sb.append(customLogon.get(tradingSystemVersionTag)).append(delimiter); sb.append(customLogon.get(tradingSystemVendorTag)); return sb.toString(); } private String calculateHmac(String canonicalRequest, String userKey) { String hash = null; try { // Init HMAC instance Mac sha256HMAC; sha256HMAC = Mac.getInstance("HmacSHA256"); // Initialize HMAC instance with the key // Decode the key first, since it is base64url encoded byte[] decodedUserKey = Base64.getUrlDecoder().decode(userKey); SecretKeySpec secretKey = new SecretKeySpec(decodedUserKey, "HmacSHA256"); sha256HMAC.init(secretKey); // Calculate HMAC, base64url encode the result and strip padding hash = Base64.getUrlEncoder().withoutPadding().encodeToString((sha256HMAC.doFinal(canonicalRequest.getBytes("UTF-8")))); } catch (NoSuchAlgorithmException | InvalidKeyException | IllegalStateException | UnsupportedEncodingException e) { e.printStackTrace(); } return hash; } |
This section explains how logging services are organized in the engine. It depicts the structure of logs, as well as how logging can be customized.
By default, the engine uses file system directory specified by the “Log.Directory” configuration parameter for storing as well as the engine log file:
For each FIX session with file-based session storage the following files are also created:
Where:
When the biz.onixs.fix.engine.Session object is created anew after the engine restart, FIX session state (the sequence numbers of last received and sent messages, previously sent messages, etc) is restored from these files.
To start a FIX session as a new one (so-called “clean start”), it is necessary to remove the log files from the previous runs.
Connecting parties must bi-laterally agree as to when sessions are to be started/stopped and log files are backed up based upon individual system and time zone requirements.
When the FIX session is finished, the log files are not needed anymore. They can be backed up and later a new FIX session with the same SenderCompID and TargetCompID will start sequence numbers from 1. The usual practice is to back up the log files at the end of each business day (so-called “End Of Day procedure”) to start the sequence numbers from 1 at the beginning of the next day.
Note: To reset the local sequence numbers to 1 and backup the log files, the biz.onixs.fix.engine.Session.resetLocalSequenceNumbers() method must be used. This method can be called only when the session is disconnected.
By default, the engine logs all important aspects of its activity while working. The SLF4J (Simple Logging Facade for Java) is used by Handler internally. The SLF4J detects and uses the concrete logging implementation configured by the user. By default, the Logback logging implementation is recommended.
Please see Logging section for more details.
Sometimes fields of the same tag number can appear multiple times in the same message within a so-called repeating group.
The following diagram depicts the general structure of a repeating group in a FIX message:
Each repeating group starts from the field which identifies the number of repeating entries within the repeating group. Such a leading field must immediately precede the repeating group entries. In the referenced example, the leading field is defined by the NoRoutingIDs tag.
Each entry of a repeating group has a selected field which in turn identifies the beginning of a group new entry. This field must occur in a raw message before any other tag from a single repeating group entry. In other words, these fields separate repeating group entries from each other. RoutingType field is an example of such a separator.
Please also see the quote from FIX standard:
Fix 4.4 Specification vol 1 page 19
If the repeating group is used, the first field of the repeating group is required. This allows implementations of the protocol to use the first field as a “delimiter” indicating a new repeating group entry. The first field listed after the NoXXX then becomes conditionally required if the NoXXX field is greater than zero.
Important aspects of FIX repeating groups structure:
Well-defined FIX messages assume all fields of a repeating group that follow one another. The appearance of an arbitrary field, which doesn't belong to the repeating group (according to the FIX specification), automatically indicates the end of the repeating group. Therefore, the further appearance of fields from the repeating group is disallowed by the FIX Standard.
It is possible for repeating groups to appear inside of another repeating group. In such a case, all entries of the inner repeating group belong to the single entry of the outer repeating group.
The number of entries or instances, which are defined by the value of the leading field (tag), must correspond to the number of instances/entries, which actually appear in the message. In particular, the number of tags-separators must equal to the value of the leading field.
Group class
The FIX repeating group is represented with the biz.onixs.fix.parser.Group class.
Since a repeating group is identified by the leading field, which defines the number of repeating instances, the engine follows this approach in the handling of repeating groups. In particular, the biz.onixs.fix.parser.Group object is accessed using its leading (number of instances) field. Changing the value of this field affects the length of a repeating group. Removing this field from a message or other repeating group removes all entries of the repeating group.
Adding Group into Message
To create a new repeating group or modify the number of instances in the existing one the biz.onixs.fix.parser.Message.setGroup(int, int) method must be used.
Accessing Repeating Group Entries
To get the group object that represents the existing repeating group of the message the biz.onixs.fix.parser.Message.getGroup(int) method must be used.
Removing Repeating Group from Message
To remove a repeating group the biz.onixs.fix.parser.Message.remove(int) method must be used. The group class works with fields and embedded repeating groups in the same manner as the biz.onixs.fix.parser.Message class. But each method has an additional parameter that defines the index of the repeating group instance (starting from 0).
Ill-Formed Repeating Group
Declared number of instances is the number of repeating group instances, that are defined by the FIX field which identifies a repeating group size (“NoField”). The actual number of instances is the number of repeating group instances which actually exist in FIX message, and can be calculated by counting the presence of FIX fields (which make up repeating group) in the sequence of the fields.
By default, the engine strictly follows requirements of the FIX Standard. Thus, it requires the actual number of repeating group instances to match the declared number of instances. FIX message parsing services report an error by throwing an exception if FIX message doesn't meet the requirements.
The engine exposes configuration parameter Validate.NumberOfRepeatingGroupInstances, which allows changing the default behavior. By changing the parameter value as shown below, it's possible to have ill-formed FIX messages parsed successfully.
Note: In case a repeating group from a message is not parsed completely try to parse the message using message constructor with enabled validateDuplicatedField parameter.
Note: If there is a parse exception with message Duplicate field found … and it refers to a field from repeating group probably the field is not defined correctly in the dialect file.
Example
Setting Value
35 36 37 38 39 40 41 42 43 44 45 46 47 | final Message message = Message.create("V", Version.FIX42); message.set(Tag.MDReqID, "ABSD").set(Tag.SubscriptionRequestType, 1) .set(Tag.MarketDepth, 1).set(Tag.MDUpdateType, 1).set(Tag.AggregatedBook, "N"); // create a repeating group NoMDEntryTypes with two instances final Group groupMDEntryTypes = message.setGroup(Tag.NoMDEntryTypes, 2); groupMDEntryTypes.set(Tag.MDEntryType, 0, "EntryType_1"); // first instance groupMDEntryTypes.set(Tag.MDEntryType, 1, "EntryType_2"); // second instance // create a repeating group NoRelatedSym with two instances final Group groupRelatedSym = message.setGroup(Tag.NoRelatedSym, 2); groupRelatedSym.set(Tag.Symbol, 0, "EURUSD_0"); // first instance groupRelatedSym.set(Tag.Symbol, 1, "EURUSD_1"); // second instance message.validate(); LOG.info("Message: {}", message); |
Getting Value
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | final Message message = new Message("8=FIX.4.2\u00019=142\u000135=V\u000149=0\u000156=0\u000134=0\u0001" + "52=99990909-17:17:17\u0001262=ABSD\u0001263=1\u0001264=1\u0001265=1\u0001266=N\u0001267=2\u0001" + "269=EntryType_1\u0001269=EntryType_2\u0001146=2\u000155=EURUSD_0\u000155=EURUSD_1\u000110=163\u0001"); LOG.info("MDEntryTypes group"); final Group mdEntryTypes = message.getGroup(Tag.NoMDEntryTypes); if (mdEntryTypes != null) { for (int i = 0; i < mdEntryTypes.getNumberOfInstances(); i++) { LOG.info("Instance # {}", i); LOG.info("MDEntryType = '{}'", mdEntryTypes.get(Tag.MDEntryType, i)); } } LOG.info("RelatedSym group"); final Group relatedSym = message.getGroup(Tag.NoRelatedSym); if (relatedSym != null) { for (int i = 0; i < relatedSym.getNumberOfInstances(); i++) { LOG.info("Instance # {}", i); LOG.info("Symbol = '{}'", relatedSym.get(Tag.Symbol, i)); } } |
During the FIX session lifetime, state changes occur. The biz.onixs.fix.engine.Session class exposes the biz.onixs.fix.engine.Session.getState() method to determine in which state it currently resides.
The following tables describe all possible FIX session states that can occur during its lifetime, as well as what a specific state means for a session with a particular role.
| State | Acceptor |
|---|---|
| DISCONNECTED | The session is disconnected. |
| AWAIT_LOGON | The session is waiting for the initial Logon message. |
| ESTABLISHED | The session is fully established (after the successful logon exchange). |
| WAIT_FOR_RETRANSMISSION | The session is waiting for the message retransmission from the counterparty. |
| AWAIT_CONFIRMING_LOGOUT | The initial logout message was sent and the session is waiting for the acknowledgment logout message. |
| State | Initiator |
|---|---|
| DISCONNECTED | The session is disconnected. |
| AWAIT_CONFIRMING_LOGON | The initial logon message was sent and the session is waiting for the acknowledgment logon message. |
| ESTABLISHED | The session is fully established (after the successful logon exchange). |
| RECONNECTING | The session is trying to restore the telecommunication link. |
| WAIT_FOR_RETRANSMISSION | The session is waiting for the message retransmission from the counterparty. |
| AWAIT_CONFIRMING_LOGOUT | The initial logout message was sent and the session is waiting for the acknowledgment logout message. |
To be notified about the changes in the session state implement the biz.onixs.fix.engine.Session.StateChangeListener class and register it as event handler with the biz.onixs.fix.engine.Session.addStateChangeListener(StateChangeListener) method.
Example
26 27 28 29 30 31 32 33 34 35 36 37 38 39 | public class SessionStateChangeListener implements Session.StateChangeListener { public static void main(String[] args) { final Engine engine = Engine.init(); final Session sn = new Session("SenderCompId", "TargetCompId", Version.FIX42); sn.addStateChangeListener(new SessionStateChangeListener()); engine.shutdown(); } @Override public void onStateChange(Object sender, Session.StateChangeArgs e) { System.out.println("Prev session state: " + e.getPrevState()); System.out.println("New session state: " + e.getNewState()); } } |
The events and their listeners of the biz.onixs.fix.engine.Session class are listed below.
| Avoid time-consuming tasks and session management method calls (logon, logout, reset, etc.) in the session event listener calling thread. |
Session state is changed: biz.onixs.fix.engine.Session.StateChangeListener
Application-level message is received from the counterparty: biz.onixs.fix.engine.Session.InboundApplicationMessageListener
Application-level flat message is received from the counterparty: biz.onixs.fix.engine.Session.InboundApplicationFlatMessageListener
Session-level message is received from the counterparty: biz.onixs.fix.engine.Session.InboundSessionMessageListener
Session-level flat message is received from the counterparty: biz.onixs.fix.engine.Session.InboundSessionFlatMessageListener
Application-level message will be sent to the counterparty: biz.onixs.fix.engine.Session.OutboundApplicationMessageListener
Application-level flat message will be sent to the counterparty: biz.onixs.fix.engine.Session.OutboundApplicationFlatMessageListener
Session-level message will be sent to the counterparty: biz.onixs.fix.engine.Session.OutboundSessionMessageListener
Application-level message resending request: biz.onixs.fix.engine.Session.MessageResendingListener
Error condition is detected: biz.onixs.fix.engine.Session.ErrorListener
Warning condition is detected: biz.onixs.fix.engine.Session.WarningListener
Example
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | public class SessionEventListener implements Session.StateChangeListener, Session.MessageResendingListener, Session.InboundSessionMessageListener, Session.InboundApplicationMessageListener, Session.OutboundSessionMessageListener, Session.OutboundApplicationMessageListener, Session.InboundApplicationFlatMessageListener, Session.InboundSessionFlatMessageListener, Session.OutboundApplicationFlatMessageListener, Session.WarningListener, Session.ErrorListener { public static void main(String[] args) { final Engine engine = Engine.init(); final Session session = new Session("SenderCompId", "TargetCompId", Version.FIX42); final SessionEventListener listener = new SessionEventListener(); session.addStateChangeListener(listener) .setInboundSessionMessageListener(listener) .setInboundApplicationMessageListener(listener) .setOutboundSessionMessageListener(listener) .setOutboundApplicationMessageListener(listener) .setWarningListener(listener) .setErrorListener(listener) .setMessageResendingListener(listener); engine.shutdown(); } @Override public void onStateChange(Object sender, Session.StateChangeArgs args) { System.out.println("New session state: " + args.getNewState()); } @Override public void onInboundApplicationMessage(Object sender, Session.InboundApplicationMessageArgs args) { System.out.println("Incoming application-level message: " + args.getMsg()); } @Override public void onInboundApplicationFlatMessage(Object sender, Session.InboundApplicationFlatMessageArgs args) { System.out.println("Incoming application-level flat message: " + args.getMsg()); } @Override public void onInboundSessionMessage(Object sender, Session.InboundSessionMessageArgs args) { System.out.println("Incoming session-level message: " + args.getMsg()); } @Override public void onInboundSessionFlatMessage(Object sender, Session.InboundSessionFlatMessageArgs args) { System.out.println("Incoming session-level flat message: " + args.getMsg()); } @Override public void onOutboundApplicationMessage(Object sender, Session.OutboundApplicationMessageArgs args) { System.out.println("Outgoing application-level message: " + args.getMsg()); } @Override public void onOutboundApplicationFlatMessage(Object sender, Session.OutboundApplicationFlatMessageArgs args) { System.out.println("Outgoing application-level flat message: " + args.getMsg()); } @Override public void onOutboundSessionMessage(Object sender, Session.OutboundSessionMessageArgs args) { System.out.println("Outbound session-level message: " + args.getMsg()); } @Override public void onWarning(Object sender, Session.WarningArgs args) { System.out.println("Session warning: " + args.getReason()); } @Override public void onError(Object sender, Session.ErrorArgs args) { System.out.println("Session error: " + args.getReason()); } @Override public boolean onMessageResending(Object sender, Session.MessageResendingArgs args) { return false; } @Override public void onMessageResendingStarted(final Object sender, final long beginSeqNum, final long endSeqNum) { System.out.println("On Message Resending Started: " + beginSeqNum + " " + endSeqNum); } @Override public void onMessageResendingFinished(final Object sender, final long beginSeqNum, final long endSeqNum) { System.out.println("On Message Resending Finished: " + beginSeqNum + " " + endSeqNum); } } |
Sometimes there is a requirement to accept an incoming FIX session “on the fly”, without the prior creation of the session object. The engine exposes an ability to create a session object in response to the incoming FIX connection. To take advantage of this feature it is necessary to implement and subscribe to the biz.onixs.fix.engine.Engine.DynamicAcceptorListener listener in the engine object.
The event arguments object provides information to make a decision on whether to accept the connection:
Note: Please note that the listener will not trigger for already created sessions (a session with same identification information) even if they are in biz.onixs.fix.engine.SessionState.DISCONNECTED state.
To accept the incoming connection:
Otherwise, the incoming connection will be rejected. Optionally a reject reason can be passed to the event arguments object.
Note: There is no need to call biz.onixs.fix.engine.Session.logonAsAcceptor() method in biz.onixs.fix.engine.Engine.DynamicAcceptorListener interface implementation, it will be called internally after the session passed to createdSession property.
Example
28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | public class DynamicAcceptorSample { public static void main(String[] args) { final EngineSettings engineSettings = new EngineSettings(); engineSettings.addListenPort(4500); final Engine engine = Engine.init(engineSettings); engine.addDynamicAcceptorListener(new AcceptingLogic()); Utils.waitForEnterToTerminateApp(); engine.shutdown(); } static class AcceptingLogic implements Engine.DynamicAcceptorListener { @Override public void onDynamicAcceptor(Object sender, Engine.DynamicAcceptorArgs args) { final SessionId sessionId = args.getSessionId(); System.out.println("Dynamic acceptor: " + sessionId + ", logon message " + args.getIncomingLogonMessage()); final Session session = new Session(sessionId); System.out.println("Session created: " + session); args.setCreatedSession(session); } } private static class RejectingLogic implements Engine.DynamicAcceptorListener { @Override public void onDynamicAcceptor(Object sender, Engine.DynamicAcceptorArgs args) { final SessionId sessionId = args.getSessionId(); System.out.println("Dynamic acceptor: " + sessionId + ", logon message " + args.getIncomingLogonMessage()); args.setRejectReason("Face control failed"); } } } |
Sometime there is a need to set additional fields (e.g. Username(553), Password(554)) in the initiation logon message. In this case the biz.onixs.fix.engine.Session.logonAsInitiator(String, int, int, Message) method can be used.
Example
33 34 35 36 | final Session initiator = new Session("InitiatorCompId", "AcceptorCompId", Version.FIX43); final Message customLogonMsg = Message.create("A", Version.FIX43); customLogonMsg.set(Tag.Username, "USERNAME").set(Tag.Password, "PASSWORD"); initiator.logonAsInitiator("localhost", 11011, 30, customLogonMsg); |
Check also Samples :: Credential, the CredentialBuySide example.
If there is a need to set additional fields in the Logout(5) message then the custom logout message can be configured.
This custom logout message can be set on the session object via the biz.onixs.fix.engine.Session.setLogoutMessage(Message) method.
Example
33 34 35 36 37 38 | final Session initiator = new Session("InitiatorCompId", "AcceptorCompId", Version.FIX43); final Message customLogoutMessage = Message.create(SessionLevelMsgType.Logout, Version.FIX43); customLogoutMessage.set(2000, "Custom field"); initiator.setLogoutMessage(customLogoutMessage) .logonAsInitiator("localhost", 11011) .logout(); |
Or this custom logout message can be passed to the biz.onixs.fix.engine.Session.logout(Message) method.
Example
33 34 35 36 | final Session initiator = new Session("InitiatorCompId", "AcceptorCompId", Version.FIX43); final Message customLogoutMessage = Message.create(SessionLevelMsgType.Logout, Version.FIX43); customLogoutMessage.set(2000, "Custom field"); initiator.logonAsInitiator("localhost", 11011).logout(customLogoutMessage); |
When Accepting an incoming FIX connection additional authentication checks may be required. The typical checks are logon username/password and source IP address. Depending on the result of the logon verification check the decision can be made whether to accept the FIX connection or reject and close the connection. This is typically performed at the logon stage.
In order to achieve this, you should subscribe to the inbound session messages and add the required check logic. If the check fails and the FIX connection needs to be rejected (closed) then an exception can be thrown from the inbound session message listener.
In the following example, the password authentication check is demonstrated.
31 32 | final Session session = new Session("AcceptorCompId", "InitiatorCompId", Version.FIX42); session.setInboundSessionMessageListener(new LogonPasswordAuthentication()); |
The text from the exception goes to the FIX logout message text field (Text(58)).
Check also Samples :: Credential for paired live applications with authentication support.
Often a counterparty requires that clients connect via a proxy for security reasons. In such cases, one can use the
biz.onixs.fix.engine.Session.setProxySettings(ProxySettings) method to set the Socks or HTTP proxy parameters.These parameters will be used during the logon as initiator to set up the proxy tunnel before the Logon exchange.
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | final String senderCompId = "SenderCompID"; final String targetCompId = "TargetCompID"; final Version version = Version.FIX44; // Socks Proxy host. final String proxyHost = "192.22.33.1"; // Socks Proxy port. final int proxyPort = 5000; // HTTP Proxy user name if necessary. final String proxyUsername = "Username"; // HTTP Proxy password if necessary. final String proxyPassword = "Password"; // Counterparty host. final String counterpartyHost = "192.55.66.1"; // Counterparty port. final int counterpartyPort = 6000; final Session initiator = new Session(senderCompId, targetCompId, version); // Sets Socks proxy settings, which will be used in the `Session.logonAsInitiator(..)` method // to setup the proxy connection before the Logon exchange. initiator.setProxySettings(new ProxySettings(ProxySettings.Type.SOCKS, proxyHost, proxyPort, proxyUsername, proxyPassword)); // Establishes the proxy tunnel first and then performs the usual Logon exchange. initiator.logonAsInitiator(counterpartyHost, counterpartyPort); // Message exchanges and processing logic goes here as usual. initiator.logout(); |
Session class exposes the biz.onixs.fix.engine.Session.resetLocalSequenceNumbers() method to reset both incoming and outgoing message local sequence numbers to 1 and backup the previous log files.
Note: The biz.onixs.fix.engine.Session.resetLocalSequenceNumbers() method can be called only when the session is disconnected.
ResetSeqNumFlag(141) field in the Logon(A) message indicates that the both sides of the FIX session should reset sequence numbers.
To send a logon message with this field during Establishing FIX Connection, use the setResetSeqNumFlag parameter of the biz.onixs.fix.engine.Session.logonAsInitiator(String, int, boolean) method.
Note: Some FIX venues do not support this flag.
Note: Only the first Logon message will be sent with the ResetSeqNumFlag flag. If subsequent Logon messages are sent during the automatic attempts to restore the telecommunication link they will be sent without this flag.
It is also possible to send a Logon message with the ResetSeqNumFlag set when the FIX Session is established (e.g. to maintain 24 hour connectivity) using the biz.onixs.fix.engine.Session.resetSeqNumViaLogonExchange() method.
Some FIX venues respond to the initial Logon message without the ResetSeqNumFlag flag, by Logon message with the flag set. This case is not covered by the FIX protocol. Therefore, in such cases, if a Logon is initiated without ResetSeqNumFlag flag, then the FIX Engine will track it and if the confirmation Logon contains ResetSeqNumFlag flag, then the FIX Engine will send another confirmation Logon with ResetSeqNumFlag flag to the counterparty. This Logon should be considered as a confirmation Logon to reset sequence numbers. In order to work it properly, one needs to use only ways, described on this page, to set ResetSeqNumFlag flag. For example, you should not set this flag manually inside the outbound callback.
When the Resend Request(2) message is received from the counterparty (e.g. due to a sequence number mismatch), MessageResending event is raised if an application has previously subscribed to this event. Otherwise, the SequenceReset-GapFill(4) message will be sent instead of the requested message.
If the sent application-level message needs to be resent to the counterparty then the return value of the MessageResending event listener should be set to true while handling the event. Otherwise, the SequenceReset-GapFill(4) message will be sent instead (e.g. in case of an aged order).
The message that is about to be resent is available via Msg property of the biz.onixs.fix.engine.Session.MessageResendingArgs parameter.
Example
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | public class MessageResendingSample implements Session.MessageResendingListener { public static void main(String[] args) { final Engine engine = Engine.init(); final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX50); session.setMessageResendingListener(new MessageResendingSample()); engine.shutdown(); } @Override public boolean onMessageResending(Object sender, Session.MessageResendingArgs args) { System.out.println("Message is about to be resent to the counterpart: " + args.getMsg()); // Return false if it's necessary to skip this message (GapFill will be sent). return false; } @Override public void onMessageResendingStarted(final Object sender, final long beginSeqNum, final long endSeqNum) { System.out.println("ResendRequest(2) message is received and the session is about to " + "start to resend messages: beginSeqNum " + beginSeqNum + " endSeqNum " + endSeqNum); } @Override public void onMessageResendingFinished(final Object sender, final long beginSeqNum, final long endSeqNum) { System.out.println("Resending process is completed: beginSeqNum " + beginSeqNum + " endSeqNum " + endSeqNum); } } |
Memory-based session storage is typically used to maintain a high-performance FIX session when persisting of session state and messages in the file system is not required.
To create such a session storageType parameter of session constructor should be set to biz.onixs.fix.engine.storage.SessionStorageType.MemoryBasedStorage.
30 31 | final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42, SessionStorageType.MemoryBasedStorage); |
Asynchronous File-Based Session Storage combines the good-performance with the File-Based Session Storage functionality. This storage has the same abilities as File-Based Session Storage except for the asynchronous file operations. Each this storage creates the separated thread, which will be used to perform all operations with the file system.
To create such a session storageType parameter of session constructor should be set to biz.onixs.fix.engine.storage.SessionStorageType.AsyncFileBased.
30 31 | final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42, SessionStorageType.AsyncFileBased); |
Pluggable session storage is the storage implemented by the user. Such storage should implement the biz.onixs.fix.engine.storage.SessionStorage and biz.onixs.fix.engine.storage.StorageRepository interfaces. After implementing the instance of the above-mentioned interface the implemented storage repository should be registered in storage repository manager. Then a session with corresponding storage type can be created (see below for more details).
35 36 37 38 | final StorageRepositoryManager repositoryManager = engine.getStorageRepositoryManager(); final StorageRepository myStorageRepository = new MyStorageRepository(); SessionStorageType myStorageType = repositoryManager.register("MyStorageID", myStorageRepository); final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42, myStorageType); |
Example
Please, check the pluggable-storage sample, included into distribution package of FIX Engine.
For the security reasons the values of the following fields can be scrambled in the session storage:
In order to achieve this on the session level the biz.onixs.fix.engine.Session.setScramblePassword(boolean) option should be true.
There is an option to skip saving of some inbound FIX messages to session storage for a performance reason.
This can be configured on a per-session basis: biz.onixs.fix.engine.Session.setInboundMessageLogFilter(MessageFilter).
For instance, the biz.onixs.fix.filter.TypeMessageFilter implementation is used to filter specified message types:
31 32 33 | final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX40); final MessageFilter messageFilter = new TypeMessageFilter("W X"); session.setInboundMessageLogFilter(messageFilter); |
The space-delimited list of message types to filter is specified.
Each session instance starts two separate threads: for sending and receiving messages. The following table explains which listener could be called from each one thread.
| Listener | Application thread | Sending thread | Receiving thread |
|---|---|---|---|
| ErrorListener | No | No | Yes |
| WarningListener | Yes | No | Yes |
| InboundApplicationMessageListener | No | No | Yes |
| InboundSessionMessageListener | No | No | Yes |
| OutboundApplicationMessageListener | Yes | No | Yes |
| OutboundSessionMessageListener | Yes | No | Yes |
| MessageResendingListener | No | No | Yes |
| StateChangeListener | Yes | Yes | Yes |
The engine by default updates SendingTime of FIX messages to be sent to counterparty using internally implemented timestamp provider. Also, it uses a timestamp provider while writes timestamp of a FIX message written into a log file.
Custom timestamp provider can be implemented and passed to session in order to be used in the above-mentioned cases.
It is required to implement the biz.onixs.fix.engine.TimestampProvider interface and pass an instance to the session using the biz.onixs.fix.engine.Session.setTimestampProvider(TimestampProvider) method.
Some venues limit the number of messages that the venue can process during the time unit (so-called “message throttling”). If an application sends messages too often, then they can be rejected.
To not exceeds the limits, there is the biz.onixs.fix.engine.Session.throttle() method.
This method performs outgoing message throttling. It must be called before each of a biz.onixs.fix.engine.Session.send(Message) function call. If the count of messages per a time unit exceeds the throttling limit, the function will block until the given time interval is passed.
To set throttling limit parameters, use the biz.onixs.fix.engine.Session.setThrottlingLimit(int, long) or biz.onixs.fix.engine.Session.setThrottlingLimit(int, long, TimeUnit) method.
Example
34 35 36 37 38 39 40 41 42 43 44 | // Set throttling limit as 35 messages per 1200 milliseconds. session.setThrottlingLimit(35, 1200L, TimeUnit.MILLISECONDS); // Create a FIX message to send. Message fixMessage = createFixMessage(); // Throttle the sending. session.throttle(); // Send the message. session.send(fixMessage); |
Sometimes, it is more effective to perform the event dispatching in a loop in the main application thread. For this purpose, one can use the biz.onixs.fix.engine.ConnectionMode.EXTERNAL_THREAD mode.
Stack
Before an externally-managed session is created, the application must create an biz.onixs.fix.engine.SessionReactor instance. There is one reactor class, which can be used, currently: biz.onixs.fix.engine.TCPStandardStack.
36 | final SessionReactor stack = new TCPStandardStack(); |
The stack is provided in Session's constructor, for example:
39 | final Session initiator = new Session(stack, "InitCompId", "AccCompId", Version.FIX44); |
Asynchronous Logon and Logout
Applications must call asynchronous Session's methods to logon biz.onixs.fix.engine.Session.logonAsInitiatorAsync(String, int, int, boolean, Message) and logout biz.onixs.fix.engine.Session.logoutAsync().
42 43 44 45 | initiator.logonAsInitiatorAsync(CounterpartyHost, CounterpartyPort, HeartBtInt, false, null); // Dispatching events.. initiator.logoutAsync("End of connection"); |
Dispatching Network Events
Applications must call biz.onixs.fix.engine.SessionReactor.dispatchEvents() frequently for each stack that is in use.
48 49 50 51 | boolean isFinished = false; while (!isFinished) { stack.dispatchEvents(); } |
Shutdown
The stack requires finishing all outstanding work and handling all outstanding events. Before destroying the stack instance, the following code should be executed:
54 55 56 | while (!stack.isQuiescent()) { stack.dispatchEvents(); } |
Method biz.onixs.fix.engine.SessionReactor.isQuiescent() returns a boolean value indicating whether a stack is quiescent.
This can be used to ensure that all connections have been closed gracefully before destroying a stack (or exiting the application). Destroying a stack while it is not quiescent is permitted by the API, but when doing so there is no guarantee that sent data has been acknowledged by the peer or even transmitted, and there is the possibility that peers' connections will be reset.
External Thread Mode for Acceptor Sessions
There is an ability to use the ExternalThread mode for acceptor sessions too. However, to dispatch listening events and accept incoming TCP connections, one must pass the the stack to the biz.onixs.fix.engine.Engine.init(EngineSettings, SessionReactor) method. The same stack should also be passed to the acceptor session's constructor. In this case, the FIX Engine does not create an additional thread to listen for incoming connections and uses the corresponding stack to dispatch these events:
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | final SessionReactor stack = new TCPStandardStack(); final EngineSettings engineSettings = new EngineSettings(); // Invoke the Engine.init with the given stack instance. Engine.init(engineSettings, stack); final Session acceptor = new Session(stack, "AccCompId", "InitCompId", Version.FIX44); acceptor.logonAsAcceptor(); boolean isFinished = false; // Dispatching events.. while (!isFinished) { stack.dispatchEvents(); } acceptor.dispose(); Engine.getInstance().shutdown(); while(!stack.isQuiescent()) { stack.dispatchEvents(); } |
Split the Sessions Processing by Different Stacks in Separate Threads
When the amount of high-loaded sessions is large, splitting sessions into groups and performing the event dispatching in separate threads makes sense and can improve performance. In this case, considering that the external thread reactors are not thread safe, each stack with corresponding sessions should be created in its thread:
39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | static class SessionReactorThread extends Thread { private final int sessionNumber; private final int reactorThreadId; private final List<Session> sessions = new ArrayList<>(); private final CompletableFuture<Void> isCompleted = new CompletableFuture<>(); SessionReactorThread(final int sessionNumber, final int reactorThreadId) { this.sessionNumber = sessionNumber; this.reactorThreadId = reactorThreadId; } boolean isCompleted() { return isCompleted.isDone(); } @Override public void run() { try { // Initialize stack. final SessionReactor stack = new TCPStandardStack(); // Initialize sessions. for (int sessionCounter = 0; sessionCounter < sessionNumber; ++sessionCounter) { final Session session = new Session(stack, reactorThreadId + "-Init-" + sessionCounter, reactorThreadId + "-Acc-" + sessionCounter, Version.FIX44); sessions.add(session); final CompletableFuture<Void> logon = session.logonAsInitiatorAsync(CounterpartyHost, CounterpartyPort, HeartBtInt, true, null); while (!logon.isDone()) { stack.dispatchEvents(); } } boolean finished = false; // Useful work and dispatching events... while (!finished) { stack.dispatchEvents(); } // Shutdown sessions. for (int sessionCounter = 0; sessionCounter < sessionNumber; ++sessionCounter) { CompletableFuture<Void> logout = sessions.get(sessionCounter) .logoutAsync("The session is disconnected"); while (!logout.isDone()) { stack.dispatchEvents(); } sessions.get(sessionCounter).dispose(); } isCompleted.complete(null); while (!stack.isQuiescent()) { stack.dispatchEvents(); } } catch (final IOException e) { isCompleted.completeExceptionally(e); } } }; public static void main(String[] args) throws IOException, InterruptedException { // Initialize engine in the main thread. final EngineSettings engineSettings = new EngineSettings(); Engine.init(engineSettings); // Initialize and start the required amount of threads. final int SessionNumber = 25; final int SessionPerThread = 5; final int ThreadNumber = SessionNumber / SessionPerThread; final List<SessionReactorThread> reactorThreads = new ArrayList<>(); for(int reactorThreadCounter = 0; reactorThreadCounter < ThreadNumber; ++reactorThreadCounter) { final SessionReactorThread sessionReactorThread = new SessionReactorThread(SessionPerThread, reactorThreadCounter); reactorThreads.add(sessionReactorThread); sessionReactorThread.start(); } // Waiting for threads completion. for(int reactorThreadCounter = 0; reactorThreadCounter < ThreadNumber; ++reactorThreadCounter) { reactorThreads.get(reactorThreadCounter).join(); } // Shutdown the FIX Engine. Engine.getInstance().shutdown(); } |
Using Different Sessions with Different Threading Models
It is possible to use initiator sessions with different threading models. For example, if the Engine is configured to use the default threading model (biz.onixs.fix.engine.ConnectionMode.DEDICATED_THREADS), and the reactor instance is passed to the session's constructor, then this particular session will use the biz.onixs.fix.engine.ConnectionMode.EXTERNAL_THREAD threading model. All other sessions will use the threading model configured for the whole FIX Engine. However, acceptor sessions can be used in the same threading mode only. This is because an acceptor connection is created when no corresponding acceptor session is identified. Therefore, the threading model is determined by whether the reactor instance is passed or not to the biz.onixs.fix.engine.Engine.init(EngineSettings, SessionReactor) method. If you pass the reactor instance to the biz.onixs.fix.engine.Engine.init(EngineSettings, SessionReactor) method, all acceptor connections and sessions will use the biz.onixs.fix.engine.ConnectionMode.EXTERNAL_THREAD threading model. If you do not pass the reactor instance to the biz.onixs.fix.engine.Engine.init(EngineSettings, SessionReactor) method, all acceptor connections and sessions will use the configured for the whole FIX Engine threading model.
It is common for firms to implement slightly different interpretations of the FIX protocol (FIX dialects).
Within the FIX protocol, such deviations from the FIX specification must be described in the FIX Engine, to ensure that the parsing or creation of the corresponding FIX messages is correct.
The Dialect configuration parameter specifies one or several dialect definition files in XML format. The dialect definition file must conform to the dialect XML schema.
FIX dialect can be set on:
Please read the details below.
The engine level dialect is defined with the FIX element with no id attribute specified. The dialect defined for the engine level is set implicitly for all engine sessions with such FIX version.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.2"> <Message type="N"> <Group numberOfInstancesTag="73"> <Field tag="37"/> </Group> </Message> </FIX> </Dialect> |
In this example, the dialect is set for all FIX 4.2 sessions. I.e. it changes the base FIX version definitions in the engine.
You need no additional steps to start using this dialect in your code. It must be only set in the configuration.
30 31 32 33 34 35 | final EngineSettings engineSettings = new EngineSettings(); engineSettings.setDialects("site/dialect_no_id.xml"); final Engine engine = Engine.init(engineSettings); final Session session = new Session("SenderCompID", "TargetCompID", Version.FIX42); final Message message = Message.create("D", Version.FIX42); session.send(message); |
The session level dialect is defined with the FIX element with id attribute specified. The dialect defined for the session level needs to be set explicitly for the selected sessions.
1 2 3 4 5 6 7 8 9 10 11 12 13 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.2" id="MyFIX"> <Message type="N"> <Group numberOfInstancesTag="73"> <Field tag="37"/> </Group> </Message> </FIX> </Dialect> |
In this example, the dialect is defined over base FIX 4.2 version definition and has specified id. I.e. it doesn't change the base FIX version definitions in the engine.
Using FIX dialect in the code is easy.
30 31 32 33 34 35 36 | final EngineSettings engineSettings = new EngineSettings(); engineSettings.setDialects("site/dialect_with_id.xml"); final Engine engine = Engine.init(engineSettings); final Version version = Version.getById("MyFIX"); final Session session = new Session("SenderCompID", "TargetCompID", version); final Message message = Message.create("D", version); session.send(message); |
To show how to define a FIX variant Dialect using the XML-based description, there is a step-by-step explanation below. It is used by the engine. For a complete reference of the definition, capabilities refer to dialects definition XML schema.
To add a new field to a FIX Message add the corresponding <Field> entity to the description of FIX message in the dialect description file.
Note: Inclusion of a Field element for the already registered (e.g. standard) field or group causes the engine to override its attribute. In this way, the existing field attributes can be modified. This behavior allows you to make a mandatory field or an optional group, and vice versa. As a separate action, the same approach gives the opportunity to replace existing FIX Repeating Groups with regular fields, and vice versa.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.0"> <Message type="D"> <!-- Field #526 does not belong to the Standard FIX 4.0. --> <Field tag="526" name="SecondaryClOrdID" isRequired="true"/> <!-- Field #21 is required in Standard FIX 4.0, but will be optional in this FIX dialect. --> <Field tag="21" name="HandlInst" isRequired="false"/> </Message> </FIX> </Dialect> |
To add a new repeating group to a FIX message, add the corresponding <Group> entity to the FIX dialect description file. The same approach is used to add new fields into an existing repeating group.
Note: In the case of a new group definition, the first field (subgroup) will be used as a leading tag (i.e. the tag which separates repeating groups entries). If the current definition describes the changes to an existing group, then the original leading tag remains operative.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.0"> <Message type="D"> <!-- Number of repeating groups for pre-trade allocation, does not belong to Standard FIX 4.0 --> <Group numberOfInstancesTag="78" name="NoAllocs"> <!-- Does not belong to Standard FIX 4.0 --> <Field tag="79" name="AllocAccount"/> <!-- Does not belong to Standard FIX 4.0 --> <Field tag="661" name="AllocAcctIDSource"/> <!-- Does not belong to Standard FIX 4.0 --> <Field tag="736" name="AllocSettlCurrency"/> <!-- Other group fields --> </Group> </Message> </FIX> </Dialect> |
To make a field, which is required according to the FIX protocol specification, optional, and vice-versa, add the corresponding <Field> entity to the FIX dialect description and set the isRequired attribute accordingly.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.0"> <Message type="D"> <!-- Required in Standard FIX 4.0, but optional in this FIX dialect --> <Field tag="21" isRequired="false" name="HandlInst"/> <!-- Optional in Standard FIX 4.0, but required in this FIX dialect --> <Field tag="109" isRequired="true" name="ClientID"/> </Message> </FIX> </Dialect> |
To define a custom (user-defined) message, add the corresponding <Message> entity to the FIX dialect description.
Example
1 2 3 4 5 6 7 8 9 10 11 12 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.0"> <Message type="UserDefinedMessage_1"> <Field tag="100" isRequired="true"/> <Field tag="101"/> </Message> </FIX> </Dialect> |
Such a user-defined message can be used exactly the same way as the standard FIX message:
32 33 34 | final Message userDefinedMsg = Message.create("UserDefinedMessage_1", Version.FIX40); userDefinedMsg.set(100, "Field 100") .set(101, "Field 101"); |
As it was previously described, the presence of a field definition in a dialect description file overrides and substitutes any entity of the same tag number which is already available in the message or outer repeating group. In this way, repeating groups can be replaced with regular fields and vice versa. However, sometimes there is a requirement to exclude completely certain fields and/or repeating groups from messages or another repeating group. Moreover, sometimes it is necessary to exclude completely certain messages from use. To satisfy this requirement, the dialect description syntax offers the mode attribute, which allows the removal of a single field, repeating group or an entire message from the dialect. To achieve this result, the “remove” value must be specified for the attribute in the corresponding dialect entry for <Field>, <Group> or <Message>.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.2"> <Message type="8"> <!-- Partial fill not supported, order is either filled or canceled. Thus, CumQty not needed. --> <Field tag="14" mode="remove"/> <!-- No need in MiscFees group as well. --> <Group numberOfInstancesTag="136" mode="remove"/> </Message> <!-- "New Order - List" is not supported. --> <Message type="E" mode="remove"/> </FIX> </Dialect> |
Using the mode attribute FIX standard dictionary can be overridden. If override mode is defined for either FIX version, the engine will replace the entire definition of FIX version with the new one. In this case, the FIX version will contain only messages, repeating groups and files defined by the description. For example, the description of FIX version 5.0 below will be empty and not contain any definitions at all.
Example
1 2 3 4 5 6 7 8 9 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="5.0" mode="override" id="Custom_FIX_5.0"> </FIX> </Dialect> |
With the use of the mode attribute, fields can be removed from repeating groups and messages. It is also possible to redefine the structure of the message and/or repeating group. The dialect description language supports the “override” value for the mode attribute. When the override mode is defined for either a message or repeating group, the engine will replace the specified definition of the message or repeating group. In this case, the message and/or repeating group will consist only of the fields and inner repeating groups defined by the description.
Note: Once a repeating group is overridden, its leading tag will be changed in the same way as if it is a new group definition, i.e. the first field, defined within the overridden group, will be considered as the leading one.
Note: Message overriding impacts the basic message structure. Only 4 fields will always be present in the overridden message structure: BeginString (8), BodyLength (9), MsgType (35), CheckSum (10)
All other fields should be added to the overridden message structure manually.
If the append mode is used and if the declared element has been already presented in the description of the message, it will be removed from its current position and added to the end of the message. For example, if the standard definition of a message is as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="5.0" id="Custom_FIX_5.0"> <Message type="7"> <Field tag="1"/> <Field tag="2"/> <Field tag="3"/> </Message> </FIX> </Dialect> |
And the user has defined their custom definition of this message as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="5.0" id="Custom_FIX_5.0"> <Message type="7"> <Field tag="1"/> <Field tag="4"/> </Message> </FIX> </Dialect> |
The result message definition will be as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="5.0" id="Custom_FIX_5.0"> <Message type="7"> <Field tag="2"/> <Field tag="3"/> <Field tag="1"/> <Field tag="4"/> </Message> </FIX> </Dialect> |
Note: In order to control tags order, you need to create a custom dialect and describe all tags in a Message or Repeating group in the necessary order with the override mode.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.4"> <!-- Advertisement is now only text info. --> <Message type="7" mode="override"> <Field tag="58" name="Text" isRequired="true"/> </Message> <Message type="S"> <!-- Completely redefines group to do not use Legs component. --> <Group numberOfInstancesTag="555" name="NoLegs" mode="override"> <!-- Now tag #588 will be as leading tag in the group. --> <Field tag="588"/> <Field tag="8000"/> </Group> </Message> </FIX> </Dialect> |
You can use the <Component> entity to describe common components in order to use it in all necessary messages. There is need to add the name attribute by which you can identify a particular component and use it in all necessary messages. Component Block can contain <Field>, <Group> and even other <Component> entities. When a Component Block occurs in a message description, all described fields from this component are considered as fields directly described in this message.
Example
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | <?xml version="1.0" encoding="utf-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX version="4.0"> <!-- Header component description. --> <Component name="Header"> <Field tag="8" name="BeginString" isRequired="true" /> <Field tag="9" name="BodyLength" isRequired="true" /> <Field tag="35" name="MsgType" isRequired="true" /> <Field tag="49" name="SenderCompID" isRequired="true" /> <Field tag="56" name="TargetCompID" isRequired="true" /> <Field tag="115" name="OnBehalfOfCompID" /> <Field tag="128" name="DeliverToCompID" /> <Field tag="90" name="SecureDataLen" /> <Field tag="91" name="SecureData" /> <Field tag="34" name="MsgSeqNum" isRequired="true" /> <Field tag="50" name="SenderSubID" /> <Field tag="57" name="TargetSubID" /> <Field tag="116" name="OnBehalfOfSubID" /> <Field tag="129" name="DeliverToSubID" /> <Field tag="43" name="PossDupFlag" /> <Field tag="97" name="PossResend" /> <Field tag="52" name="SendingTime" isRequired="true" /> <Field tag="122" name="OrigSendingTime" /> </Component> <!-- Trailer component description. --> <Component name="Trailer"> <Field tag="93" name="SignatureLength" /> <Field tag="89" name="Signature" /> <Field tag="10" name="CheckSum" isRequired="true" /> </Component> <Message type="A"> <!-- Using fields from Header component. --> <Component name="Header" /> <Field tag="98" name="EncryptMethod" isRequired="true" /> <Field tag="108" name="HeartBtInt" isRequired="true" /> <Field tag="95" name="RawDataLength" /> <Field tag="96" name="RawData" /> <!-- Using fields from Trailer component. --> <Component name="Trailer" /> </Message> </FIX> </Dialect> |
The engine supports dialect in the QuickFIX XML format, so you can use it as it is. Identifier (id) will be generated when a QuickFIX dialect is used. This id is equal to the dialect file name without extension. Therefore, QuickFIX dialect becomes a session-level.
Check also Samples :: Dialect for paired live applications with dialect support.
The SSL (Secure Sockets Layer) support helps to make FIX communication secure.
The SSL support is implemented using Java Security / JSSE (Java Secure Socket Extension) implementation. Learn more here about Java Security and JSSE.
The SSL configuration procedure is slightly different for FIX acceptor and initiator sessions.
Before establishing an SSL-secured FIX session the SSL context needs to be created. One of the methods follows.
34 35 | final SSLContext sslContext = SslContextFactory.getInstance(keyStoreName, trustStoreName, keyPassword, keyStorePassword, trustStorePassword); |
Parameters:
The key and trust stores are loaded as described in the Unified Resource Loading.
The acceptor SSL configuration is performed on the engine level via the following method.
38 39 40 41 42 43 | final SSLContext sslContext = SslContextFactory.getInstance(keyStoreName, trustStoreName, keyPassword, keyStorePassword, trustStorePassword); Engine.setSSLContext(sslContext); final Engine engine = Engine.init(11011); final Session session = new Session("SellSide", "BuySide", Version.FIX40); session.logonAsAcceptor(); |
Make sure that you configure SSL before doing engine init.
The initiator SSL configuration is performed on the session level via the following method.
38 39 40 41 42 | final Session session = new Session("BuySide", "SellSide", Version.FIX40); final SSLContext sslContext = SslContextFactory.getInstance(keyStoreName, trustStoreName, keyPassword, keyStorePassword, trustStorePassword); session.setSSLContext(sslContext); session.logonAsInitiator("localhost", 11011); |
Make sure that you configure SSL before doing session logon.
The SSL encryption can be configured via system properties instead of the biz.onixs.fix.engine.SslContextFactory
In this case the default SSL context can be used:
36 37 38 39 40 41 42 43 | System.setProperty("javax.net.ssl.keyStore", keyStoreName); System.setProperty("javax.net.ssl.keyStorePassword", keyStorePassword); System.setProperty("javax.net.ssl.trustStore", trustStoreName); System.setProperty("javax.net.ssl.trustStorePassword", trustStorePassword); final Session session = new Session("BuySide", "SellSide", Version.FIX40); final SSLContext sslContext = SSLContext.getDefault(); session.setSSLContext(sslContext); session.logonAsInitiator("localhost", 11011); |
| System Property | Description |
|---|---|
| javax.net.ssl.keyStore | Key store file location. |
| javax.net.ssl.keyStorePassword | Password to access the private key from the key store file specified by “javax.net.ssl.keyStore”. This password is used twice: to unlock the key store file (store password), and to decrypt the private key stored in the key store (key password). |
| javax.net.ssl.trustStore | Trust store file location. |
| javax.net.ssl.trustStorePassword | Password to unlock the key store file (store password) specified by “javax.net.ssl.trustStore”. |
| javax.net.ssl.trustStoreType | For Java keystore file format, this property has the value jks (or JKS). You do not normally specify this property, because its default value is already jks. |
| javax.net.debug | To switch on logging for the SSL/TLS layer, set this property to SSL. |
| There is no way to pass the key password via a system property. It can be equal to the key store password though. |
To perform key and certificate management the keytool can be used. This tool is a part of JDK package and the complete documentation can be found in the JDK documentation → Tools & Tool APIs → Security.
Below you can find the simplified pre-steps to do in order to organize secure communication between two parties.
Generate the key pair for the “SellSide” and “BuySide”:
keytool -genkeypair -dname “cn=SellSide, ou=FixEngine, o=OnixS, c=US” -alias sellside -keypass pass11 -keystore sellside.keystore.bin -storepass pass11 -validity 360
keytool -genkeypair -dname “cn=BuySide, ou=FixEngine, o=OnixS, c=US” -alias buyside -keypass pass22 -keystore buyside.keystore.bin -storepass pass22 -validity 360
Export the certificate of “SellSide” and “BuySide”:
keytool -exportcert -alias sellside -file sellside.cer -keystore sellside.keystore.bin -storepass pass11
keytool -exportcert -alias buyside -file buyside.cer -keystore buyside.keystore.bin -storepass pass22
By default, the certificates are exported in the binary format. However, you can select BASE64-encoded printable format - Internet RFC 1421. Sometimes it is referred to as PEM - Privacy Enhanced Mail. Use the “-rfc” key then.
keytool -exportcert -rfc -alias sellside -file sellside.pem -keystore sellside.keystore.bin -storepass pass11
keytool -exportcert -rfc -alias buyside -file buyside.pem -keystore buyside.keystore.bin -storepass pass22
Import the certificate of “BuySide” into the “SellSide” trust store and vice versa:
keytool -importcert -noprompt -alias buyside -file buyside.cer -keystore sellside.truststore.bin -storepass pass33
keytool -importcert -noprompt -alias sellside -file sellside.cer -keystore buyside.truststore.bin -storepass pass44
By default, the certificates are imported in the binary format. However, you can select BASE64-encoded printable format - Internet RFC 1421. Sometimes it is referred to as PEM - Privacy Enhanced Mail. Use the “-rfc” key then.
keytool -importcert -noprompt -rfc -alias buyside -file buyside.pem -keystore sellside.truststore.bin -storepass pass33
keytool -importcert -noprompt -rfc -alias sellside -file sellside.pem -keystore buyside.truststore.bin -storepass pass44
As a result, we have two pairs of the key store and trust store for each party to establish secure communication:
This is what was used for the samples.
Check also Samples :: SSL for paired live applications with SSL support.
The FIX biz.onixs.fix.engine.Engine class is implemented as a singleton.
Thus you can't have more than a single instance of the engine in the same JVM. If you have several applications based on the engine and you want to run them in the same JVM then you need to share the engine instance between all of them. It means engine configuration must include everything to satisfy every application needs.
Assume 1st application uses the FIX dialect defined in the “sharing-engine-dialect-1.xml”.
1 2 3 4 5 6 7 8 | <?xml version="1.0" encoding="UTF-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX id="dialect-1" version="5.0_SP2"> </FIX> </Dialect> |
And assume 2nd application uses the FIX dialect defined in the “sharing-engine-dialect-2.xml”.
1 2 3 4 5 6 7 8 | <?xml version="1.0" encoding="UTF-8"?> xsi:schemaLocation="https://ref.onixs.biz/fix/dialects https://ref.onixs.biz/fix/dialects/dialects-2.18.xsd"> <FIX id="dialect-2" version="5.0_SP2"> </FIX> </Dialect> |
In order to run both applications in the same JVM at the same time, we need to pass both dialect files to the engine during initialization.
25 26 27 28 29 30 31 32 33 34 35 36 37 38 | public class SharingEngineMain { public static void main(String[] args) { final EngineSettings settings = new EngineSettings(); settings.setDialects("site/sharing-engine-dialect-1.xml | site/sharing-engine-dialect-2.xml"); final Engine engine = Engine.init(settings); // final SharingEngineApp1 app1 = new SharingEngineApp1(); final SharingEngineApp2 app2 = new SharingEngineApp2(); app1.work(); app2.work(); // engine.shutdown(); } } |
The 1st application will use the pre-initialized engine instance.
25 26 27 28 29 30 | public class SharingEngineApp1 { public void work() { final Version version = Version.getById("dialect-1"); final Session session = new Session("SenderCompId-1", "TargetCompId-1", version); } } |
And the 2nd application will use the same pre-initialized engine instance.
25 26 27 28 29 30 | public class SharingEngineApp2 { public void work() { final Version version = Version.getById("dialect-2"); final Session session = new Session("SenderCompId-2", "TargetCompId-2", version); } } |
This section summarizes our findings and recommends best practices to tune the different layers of Java FIX Engine.
Please note that the exact benefits and effects each of these configuration choices, that will be highly dependent upon the specific applications and workloads. In this way, so we strongly recommend experimenting with the different configuration options with your workload before deploying them in a production environment.
Using MemoryBasedStorage instead of FileBasedStorage boosts performance since FIX messages are stored directly in memory. Alternatively, it's possible to use custom pluggable storage (using PluggableStorage) which does nothing on FIX message-related operations as soon as no resend requests are to be supported.
It is also recommended to use AsyncFileBasedStorage which provides same functionality as FileBasedStorage with good-performance.
By default, threads are used by FIX session to send and receive FIX messages; they can be executed on any of available processors/cores. Specifying CPU affinity for each session thread may give a significant performance boost. It can be configured via the biz.onixs.fix.engine.Session.setReceivingThreadAffinity(int:A) and biz.onixs.fix.engine.Session.setSendingThreadAffinity(int:A) methods.
The biz.onixs.fix.engine.Session.setSendSpinningTimeout(long) method can be used to decrease the latency of the data sending.
If the value is zero (by default) and the outgoing message cannot be sent immediately it is placed to the outgoing queue. If the value greater than zero, the biz.onixs.fix.engine.Session.send(Message) method waits for the socket sending buffer availability in the spin loop mode before placing the message to the outgoing queue (to be sent later by the sending thread). The method parameter specifies the spin loop period in nanoseconds. biz.onixs.fix.engine.Session.setSendSpinningTimeout(long) method usage makes sense when your session sends FIX messages frequently, in this case, waiting in the loop is cheaper than the thread context switch. However please note that the spin wait increases the CPU usage so the spin wait period should not be too long.
The biz.onixs.fix.engine.Session.setReceiveSpinningTimeout(long) method can be used to decrease the latency of the data receiving.
If the value is zero (by default) and there is no data to read from the wire the receiving thread enters into the blocking wait mode. If the value greater than zero, the receiving thread waits for the data from wire to be available in the spin loop mode before entering into the blocking wait mode. The method parameter specifies the spin loop period in nanoseconds. biz.onixs.fix.engine.Session.setReceiveSpinningTimeout(long) method usage makes sense when your session receives FIX messages frequently, in this case, waiting in the loop is cheaper than the thread context switch. However please note that the spin wait increases the CPU usage so the spin wait period should not be too long.
Ideally, each spinning thread should run on a separate CPU core so that it will not stop other important threads from doing work if it blocks or is de-scheduled. If more than one spinning thread shares the same CPU core, it could significantly increase jitter.
By default, the logging of an outgoing message to the session storage is performed before sending it to the wire. This is more reliable because we guarantee that an outgoing message is stored before going to the counterparty and if the application is shut down after sending, for some reason, the sent message can be resent afterward.
However, this approach adds the logging latency to the FIX Engine sending latency. As a result, it increases the tick-to-trade latency. When the latency is more important, one can switch off the logging before sending, by setting the biz.onixs.fix.engine.Session.setLogBeforeSending(boolean) option to false. In this case, the logging of outgoing messages to the session storage will be performed after sending it to the wire. This way, one can exclude the logging latency from the FIX Engine sending latency and as a result, decrease the tick-to-trade latency.
The biz.onixs.fix.engine.Session.warmUp(Message) method can be used to warm up the sending path.
It makes sense if your session sends FIX messages infrequently, in this case, the sending path and associated data structures will not be in a cache and this can increase the send latency. You can periodically (a recommended period is 500 usec and less) call the biz.onixs.fix.engine.Session.warmUp(Message) to avoid cache misses and keep the sending path fast.
Object creation is an expensive operation in Java, with an impact on both performance and memory consumption. The cost varies depending on the amount of initialization that needs to be performed when the object is to be created. OnixS Java FIX Engine exposes an ability to reuse the incoming message by Session. We highly recommend to turn on SessionInboundMessageReuse to minimize the excess object creation and garbage collection overhead.
If no messages are re-sent on counterparty's Resend Request messages, then it is possible to set the Resending.QueueSize to zero to increase overall performance.
Validation significantly reduces performance due to miscellaneous checks, performed on each FIX message.
When FIX Message is constructed, the space for all the fields, that are defined for the message, is allocated. The latest FIX specifications define a lot of fields for each message type. Even when most of the fields are not used, FIX Engine reserves the space for them, and this has a negative effect on FIX Engine's performance.
Editing dictionaries descriptions and excluding messages and fields (especially repeating groups), which are not used by an application, have direct influence onto FIX messages processing speed and thus decrease general latency of FIX message related processing.
While constructing FIX messages, each time message, that has to be sent, is not efficient, because it involves memory allocation and subsequent destruction. The biz.onixs.fix.parser.Message class provides ability to reuse a message of a particular type. It exposes the biz.onixs.fix.parser.Message.reset() method.
It wipes out all the FIX fields, assigned to the given instance. This method does not deallocate all memory occupied, it just brings the message to the initial state as it was just constructed. However, after the previously allocated memory is reused, thus, the second use of the object allows to setup FIX fields faster.
A common strategy is to use the same message instance for sending multiple times. As soon as a message of a certain type is constructed, it is updated with common fields for the subsequent re-use. Afterward, before sending it to the counterparty, it is updated with mutable fields (like price or quantity) exactly for the case and sent to the counterparty.
As mutable fields are updated, each time message is about to be sent to a counterparty.
There are many settings in the Session class and this section covers the only subset that relates to high-throughput workloads.
Please see corresponding section in Low Latency Best Practices.
Please see the corresponding section in Manipulating Threads Affinity.
Please see corresponding section in Low Latency Best Practices.
Please see corresponding section in Low Latency Best Practices.
Please see corresponding section in Low Latency Best Practices.
Please see the corresponding section in Low Latency Best Practices.
There are many options in the protocol stack that can affect the efficiency of the data delivery. You must understand the characteristics of the version of the stacks you are running, and that they are compatible with the versions and options on the other stacks.
Nagle’s algorithm is very useful for minimizing network overhead by concatenating packets together. We strongly recommend enabling Nagle’s algorithm for high-throughput workloads. Please see Connection.TcpNoDelay setting in the Settings section for more details.
Please see corresponding section in Low Latency Best Practices.