Composite data handling
A composite is an SBE type that contains several elementary fields. Composites are used for values that cannot be represented by one primitive field, for example:
- decimal value;
- date without time;
- timestamp with unit and optional timezone;
- service headers.
Composite layout is always defined in the XML template.
Predefined composites
The decoder recognizes several composite shapes as predefined:
- Framing header (SOFH);
- Message header;
- Repeating group header;
- Decimal;
- MonthYear;
- UtcTimestamp;
- TzTimestamp;
- VarData header.
Recognition is based on field composition, not on type name. If shape does not match predefined rules, the type is treated as custom composite.
Rules and semantics of wire format come from the SBE standard. This guide focuses on API behavior.
Framing header
Framing header is configured through API, not directly from template parsing flow. See Framing and message headers.
Message header
Message header is inserted after SOFH and is processed automatically. Standard fields are handled by decoder. Extra custom fields can be handled through virtual fields or custom composites.
Message header fields:
| Name | Type | Required | Description |
|---|---|---|---|
blockLength |
Integer 8-32 bits | Yes | Length of fixed-size block of the message body. |
version |
uint16 |
Yes | Message version. |
templateId |
uint8-uint32 |
Yes | Message template ID. |
schemaId |
Integer | Yes | Schema ID. |
numGroups |
Integer | No | Number of top-level repeating groups. |
numVarDataFields |
Integer | No | Number of top-level variable-length fields. |
Repeating group header
Each group has its own header. Decoder fills it automatically.
Group header fields:
| Name | Type | Required | Description |
|---|---|---|---|
blockLength |
Integer 8-32 bits | Yes | Length of fixed-size block for one group element. |
numInGroup |
uint16 |
Yes | Number of elements in the group. |
numGroups |
Integer | No | Number of nested groups on top level of group element. |
numVarDataFields |
Integer | No | Number of top-level variable-length fields in group element. |
Decimal
Decimal composite is accessed with:
Structure:
| Name | Type | Description |
|---|---|---|
mantissa |
Integer 8-64 bits | Mantissa value. |
exponent |
Integer | Exponent value. |
MonthYear
MonthYear composite is accessed with:
Structure (at least one field must be present):
| Name | Type | Required | Description |
|---|---|---|---|
year |
Integer | No | Year. |
month |
Integer | No | Month. |
day |
Integer | No | Day. |
week |
Integer | No | Week number. |
UTC Timestamp and TZ Timestamp
Timestamp composites are accessed with:
SbeTimestamp object is populated according to concrete composite layout. For UTC timestamp, timezone fields stay untouched.
UTCTimestamp structure:
| Name | Type | Description |
|---|---|---|
time |
Integer 8-64 bits | Number of time units. |
unit |
Integer | Time unit. |
TZTimestamp structure:
| Name | Type | Description |
|---|---|---|
time |
Integer 8-64 bits | Number of time units. |
unit |
Integer | Time unit. |
timezoneHour |
Integer | Timezone hour offset. |
timezoneMinute |
Integer | Timezone minute offset. |
VarData header
Var-data fields are encoded as header + payload and handled automatically.
Structure:
| Name | Type | Description |
|---|---|---|
length |
Integer 8-64 bits | Length of payload. |
data/varData |
Bytes | Payload bytes. |
Custom composites
Any composite that is not recognized as predefined is a custom composite. You can handle it in three ways:
| Approach | Strengths | Limitations |
|---|---|---|
| Stateful custom object | Detached Java object, straightforward usage | Reads/writes full object each time; not suitable for custom service headers |
| Stateless composite interface | Field-level access, no full copy | Bound to current message lifecycle |
| Virtual fields | Fast field-level access, no extra interface | No single aggregate object |
Creating custom composite in template
Each custom composite must be declared in XML template:
<composite name="NewType" description="New non-standard custom type.">
<type name="Number" type="int32" presense="optional"/>
</composite>
Stateful custom composite (using user class)
Stateful object is detached from fieldset. Data is copied between message buffer and object on read/write calls.
Required steps:
- Define custom interface and mark fields with required annotations.
- Pass this interface class to
ByteDecoderFactory.create(...).
Interface example:
@SbeCompositeType("NewType")
public interface CustomComposite {
@SbeCompositeField("Number")
void setN1(int value);
@SbeCompositeField("Number")
int getN1();
@SbeCompositeField("Number")
void setNullNumber();
@SbeCompositeField("Number")
boolean isNullNumber();
}
Create empty instance:
Error during retrieving content skip as ignoreDownloadError activated.
Read and write field value:
// The custom2Obj gets content of the composite field
msg1.getCustomComposite(9001, customObj);
customObj.setN1(12345);
msg1.getCustomComposite(9001, customObj);
Stateless custom composite
Stateless composite is bound to current fieldset and field. It gives direct field-level access without full object copy.
Setup steps are the same:
- Define interface.
- Pass class to decoder factory.
Template example:
<composite name="NewType" description="New non-standard custom type.">
<type name="Number" type="int32" presense="optional"/>
</composite>
Interface example:
@SbeCompositeType("NewType")
public interface CustomComposite {
@SbeCompositeField("Number")
void setN1(int value);
@SbeCompositeField("Number")
int getN1();
@SbeCompositeField("Number")
void setNullNumber();
@SbeCompositeField("Number")
boolean isNullNumber();
}
Get stateless instance:
CustomComposite customObject = msg1.getStatelessComposite(9001, CustomComposite.class);
Read/write through stateless interface:
int n1 = customObject.getN1(); // Reads data.
customObject.setN1(12345); // Immediately updates low-level data.
Stateful vs stateless on the same field
The following example shows both approaches for one composite field:
// Stateful composite: detached object, requires explicit write-back.
CustomComposite stateful = decoder.createEmptyCustomComposite(CustomComposite.class);
msg1.getCustomComposite(9001, stateful);
int statefulN1 = stateful.getN1();
stateful.setN1(statefulN1 + 1);
msg1.setCustomComposite(9001, stateful);
// Stateless composite: direct view over message bytes.
CustomComposite stateless = msg1.getStatelessComposite(9001, CustomComposite.class);
int statelessN1 = stateless.getN1();
stateless.setN1(statelessN1 + 1);
Access to custom composites using virtual fields
A virtual field is a composite member that can be addressed as a normal IFieldSet field by synthetic ID. No extra Java interface is needed.
Virtual IDs are generated during schema parsing. Use MessageSchema.queryVirtualFieldId to get an ID.
Field path rules:
- path is
String[]; - each element is name or numeric ID of schema element;
- last element is member of target composite;
- first element is message name/ID, or
"*"for message header.
Access field by virtual ID
Composite definition:
<composite name="Composite2" description="Custom composite type.">
<type name="CountOfSomething" type="int32"/>
<type name="SizeOfSomething" type="int64"/>
</composite>
Message definition:
<sbe:message name="CustomMessage" id="1" description="The message with custom composite field.">
<!-- The 35'th field required for FIX messages -->
<field name="messageType" type="NoMatter" id="35" presence="constant" value="NoMatter.Also"/>
<!-- The custom composite field -->
<field name="cust1" type="Composite2" id="9001"/>
</sbe:message>
Goal: access SizeOfSomething inside field cust1 of message CustomMessage.
Path components:
- Message name (or ID).
- Field name (or ID).
- Composite member name.
Path example:
String[] fieldPath = new String[] {
"CustomMessage", // Message name
"cust1", // Field name
"SizeOfSomething" // Composite field (element) name.
};
Query virtual ID:
Document xmlSchema = ParseUtil.parse(new File("MessageSchema.xml"));
MessageSchema schema = new MessageSchema(xmlSchema);
int virtualFieldId = schema.queryVirtualFieldId(fieldPath);
If you want pure virtual-field access, suppress custom composite recognition by passing placeholder class.
Placeholder class:
@SbeAutoRecognize
public class Composite2 {
}
Decoder creation:
Class<?>[] customComposites = {
Composite2.class
};
ByteDecoder decoder = new ByteDecoderFactory().create(schema, customComposites);
Use virtual ID:
byte[] data = new byte[SIZE_ENOUGH_TO_ENCODE];
IMessage msg = decoder.encode(data, 0, data.length, 1);
// Set the virtual field value
msg.setLong(virtualFieldId, 123L);
// Get the virtual field value
long v = msg.getLong(virtualFieldId);
Access custom message-header fields by virtual ID
Standard header fields are handled automatically. If the header has custom members, you can update them via virtual IDs.
Header example with extra SequenceNo field:
<composite name="messageHeader" description="Message header with non-standard field.">
<type name="blockLength" primitiveType="uint16"/>
<type name="templateId" primitiveType="uint16"/>
<type name="schemaId" primitiveType="uint16"/>
<type name="version" primitiveType="uint16"/>
<type name="SequenceNo" primitiveType="int32"/>
</composite>
Decoder creation (no custom header class required):
ByteDecoder decoder = new ByteDecoderFactory().create(schema);
Path for header field:
String[] headerSequenceNoFieldPath = new String[] {
// Message header composite pseudo-name
SpecialFieldIds.MESSAGE_HEADER_FIELD_NAME,
// The message header field name
"SequenceNo"
};
// Get virtual ID of the 'SequenceNo' message header field
int seqNumID = schema.queryVirtualFieldId(headerSequenceNoFieldPath);
Message creation:
byte[] data1 = new byte[SIZE_ENOUGH_TO_ENCODE];
IMessage msg1 = decoder.encode(data, 0, data.length, 1);
Header field access by virtual ID:
// Set the 'SequenceNo' message header field
msg1.setInt(seqNumID, 12345);
// Get the 'SequenceNo' message header field
int seqNum1 = msg1.getInt(seqNumID);
Java SBE Decoder