Message encoding and decoding

General principles

One IMessage instance represents one SBE message at a time.
ByteDecoder creates IMessage objects and can be reused.
Both sides of communication must use the same schema, or compatible schema versions.
IMessage works directly with the byte buffer. Field updates are written to the buffer immediately.

Preparing ByteDecoder

Create MessageSchema from XML and then create ByteDecoder:

        ByteDecoder decoder = null;
        try {

            MessageSchema schema = new MessageSchema(ParseUtil.parse(new File("SbeXmlTemplate.xml")));
            decoder = new ByteDecoderFactory().create(schema);
        } catch (LicenseException e) {
            throw new RuntimeException(e);
        }

Some venues require explicit SOFH configuration. In this case, create schema with a specific framing header policy:

ByteDecoder decoder = null;
try {
    // Use one of well-known Simple Open Framing Header specifications.
    MessageSchema schema = new MessageSchema(ParseUtil.parse(new File("SbeXmlTemplate.xml")),
            FramingHeaderPolicy.getCustomPolicy(FramingHeaderSpecification.B3_FRAMING_HEADER));
    decoder = new ByteDecoderFactory().create(schema);
} catch (LicenseException e) {
    throw new RuntimeException(e);
}

Message encoding

Encoding means building a valid SBE packet in a byte buffer.

Main steps:

Allocate a buffer.
Select template ID.
Call ByteDecoder.encode(...).
Fill message fields.
Send or store only the used message part.

Preparing buffer

The buffer must contain the full message, including SOFH and message header. Usually a fixed reserve like 4 KB is enough, then the real message size is taken from IMessage.getMsgSize().

// Allocate buffer.
final int LargeBufferSize = 4096;
byte[] buffer = new byte[LargeBufferSize];

Bind the buffer to a message template:

// Choose template ID for the message encoded
int templateID = 101;
// Obtain IMessage instance that will handle the message with chosen template ID:
IMessage msgEncoded = decoder.encode(buffer, 0, buffer.length, templateID);

After initialization:

Headers are initialized.
Required fields are set to default values.
Optional fields are set to null representation.
Top-level groups and var-data fields are initialized as empty.
Message version is set to the latest version allowed by schema.

Fields filling

Use IFieldSet methods to write field values.

Fixed-length fields

Use typed setters:

setInt, setLong, setBoolean, and similar methods for scalar fields.
setString for fixed-length strings.
setDecimal, setMonthYear, setTimestamp for known composite wrappers.

Useful operations:

reset(tag) restores default value for required fields and null value for optional fields.
setBytes(tag, value) overwrites raw bytes and skips semantic validation. Use it only when needed.

Repeating groups

Flow for writing a group:

Get IGroup by tag.
Set group size.
Iterate elements with setPos, next, or rewind.
Fill fields for each element.

Sample group definition:

<group name="NoMDEntries" id="268">
    <field name="MDEntryType" id="269" type="char"/>
    <field name="MDEntryPx" id="270" type="decimal" presence="optional"/>
    <field name="Currency" id="15" type="char" length="3" presence="constant">USD</field>
</group>

Sample code:

        // Obtain list of bids from outside
        ScaledDecimal[] bids = getBids();

        // Choose group template ID
        int groupTag = 268;

        // Obtain group object
        IGroup group = msgEncoded.getGroup(groupTag);

        // Set group length
        int groupLength = bids.length;
        group.setLength(groupLength);

        // Iterate throuth the group elements to update them
        for(int i = 0; i < group.getLength(); ++i)
        {
            // Set position inside the group
            group.setPos(i);

            // All field operations will be performed on the current group element
            group.setChar(269, '0');
            group.setDecimal(270, bids[i]);

            // No need to update the constant field 'Currency'.
        }

Recommendation: fill groups in schema order. Re-sizing deep nested groups can be expensive.

Variable-length fields

Use setVarData(tag, bytes) for var-data fields. For predictable layout and lower overhead, fill var-data fields in schema order.

Message decoding

Decoding means binding existing bytes to IMessage and reading fields.

Preparing decoding

Read bytes from external source:

// Allocate buffer.
byte[] buffer;
// buffer = ... fill it somewhere else ...

Bind bytes to message object:

// Obtain IMessage instance that will handle the message:
IMessage msgDecoded = decoder.decode(buffer, 0, buffer.length);

decode(...) returns:

IMessage, when message is complete and recognized;
null, when there is not enough data in the buffer.

Invalid data causes an exception.

Reading message service parameters

Read message-level metadata (template ID, semantic type, size, and related values):

        // Number of bytes occupied by the SBE message.
        int bytesUsed = msgDecoded.getMsgSize();

        // Template ID of the message.
        int templateId = msgDecoded.getTemplateId();

        // Version of the message.
        int version = msgDecoded.getVersion();

Decoding multiple messages from one buffer

IMessage.getMsgSize() returns full packet size (including headers).
Use it to move offset to the next message:

IMessage msgNext = decoder.decode(buffer, bytesUsed, buffer.length - bytesUsed);

Loop example:

        int offset = 0;
        while(offset < buffer.length) {
            IMessage msg = decoder.decode(buffer, offset, buffer.length - offset);
            if (msg == null) {
                break;
            }
            offset += msg.getMsgSize();

            // ... Handle the message content ...
        }

Checked loop with explicit message-size validation:

        int checkedOffset = 0;
        while (checkedOffset < buffer.length) {
            IMessage msg = decoder.decode(buffer, checkedOffset, buffer.length - checkedOffset);
            if (msg == null) {
                // Partial packet at the end of the buffer.
                break;
            }

            int msgSize = msg.getMsgSize();
            if (msgSize <= 0) {
                throw new IllegalStateException("Decoded message size must be positive.");
            }

            // ... process decoded message ...
            checkedOffset += msgSize;
        }

Reading fixed-length fields

For required fields, call typed getters like getInt, getLong, getString.

For optional fields, use one of two approaches:

contains(tag) + regular getter;
tryGet* methods (tryGetInt, tryGetString, and others) to avoid exceptions and temporary null objects.

Example with tryGet* methods:

        AtomicInteger optInt = new AtomicInteger();
        if (msgDecoded.tryGetInt(9002, optInt)) {
            int optionalIntValue = optInt.get();
            // ... use optionalIntValue ...
        } else {
            // Field is missing, null, or cannot be converted to int.
        }

        String optionalText = msgDecoded.tryGetString(9003);
        if (optionalText != null) {
            // ... use optionalText ...
        }

Reading repeating groups

Use getGroup(tag) to obtain IGroup, then iterate its elements and read fields with standard getters. Group navigation methods are the same as in encoding flow: setPos, next, and rewind.

Reading variable-length fields

Use getVarData(tag) or tryGetVarData(tag) for var-data content. Use getBytes(tag) when raw field bytes are needed without type-specific conversion.