How long does it take to send One Million FIX messages using Java?

Version 1.0.0

Contents

• Introduction
• Test harness
• Test phases
• Results
• See also

Introduction

One of the ways to ensure the highest possible performance and quality of software products is to run stress/load tests. This article describes how we do the load testing of our Java FIX Engine.

Test harness

The test harness consists of two projects: Initiator and Acceptor.

The Initiator establishes the FIX Session and sends millions of FIX messages, and the Acceptor receives these messages. During that, we measured how long it took the Acceptor to receive them.

• Both of the participants are configured with the same numbers of warmup and test messages;
• We use the TcpNodelay=true configuration parameter to improve throughput;
• To get the maximum possible performance, we use Session.UpdateSendingTime = false and Session.UpdateHeaderFields = false configuration settings.

We send FIX messages between two machines that have the following configuration:

• Intel® Core® CPU i9-12900KS @ 3.40GHz, 32 GB DDR5 RAM
• Ubuntu 22.04
• Java 11.0.21
• Solarflare Network cards
• Onload 8.1.3

Both machines can be switched to a power-saving mode with a processor frequency of 800 MHz. This mode is called idle-mode, and the maximum performance mode is called perf-mode.

Test phases

Initialization

Initiator connects to Acceptor and establishes the FIX Session:

initiator.logonAsInitiator(host, port, heartBeatInterval);

Warmup stage and main loop

The Initiator sends a certain number of messages before performance measurement begins. This preliminary sending of messages is called the "warm-up phase", but technically all messages are sent in one big cycle.

The message used in the test is a New Order Single FIX message (78 bytes).

final long totalMessageNum = warmUpMsgNum + messageNum;

for (var i = 0; i < totalMessageNum; ++i)
    initiator.send(order);

The Acceptor receives all the messages and, after the warmup messages are received, begins to measure the time:

    public void onInboundApplicationMessage(final Object sender, final Session.InboundApplicationMessageArgs args) {
        if (warmUpCounter < getWarmUpMessageNum()) {
            warmUpCounter++;
            if (warmUpCounter == getWarmUpMessageNum()) {
                LOG.info("session #{}: warm-up: {} messages transferred", getNumber(), getWarmUpMessageNum());
                timer = new PrecisionTimer();
            }
        } else {
            benchmarkCounter++;
            if (0L == (benchmarkCounter % getLogStatNumber())) {
                logStat();
            }
            if (benchmarkCounter == getMessageNum()) {
                getCompleted().countDown();
            }
        }
    }

The Acceptor periodically logs the number of messages received and statistical data collected during the receiving.

    private void logStat() {
        timer.update();
        final long speed = (long) timer.getItemsPerSecond(benchmarkCounter);
        final long latency = (long) timer.getNanoSecondsPerItem(benchmarkCounter);
        LOG.info("session #{}: benchmark: {} / {} messages transferred, avg throughput {} msgs/sec, " +
                "avg latency {} nanos", getNumber(), benchmarkCounter, getMessageNum(), speed, latency);
    }

Finally, we see the numbers we are looking for.

Results

To get the maximum possible performance we use the Onload tool. Also, we have compared the Onload-managed performance with performance without onload and in the idle-mode.

Test conditions	Throughputting (msgs/sec)	Time to send 1M messages (seconds)
Onload + perf-mode	3,197,647	0.31
perf-mode only	1,876,835	0.53
idle-mode	224,796	4.45

See also

OnixS ultra-latency Java FIX Engine