Taking timestamps of the received data

On Linux, the handler can probe the timestamp of the received data and provide it to the user along with the data. This can be useful to monitor the TCP/IP stack performance:

class DataListener : public SessionListener
{
public:
    void onReceivedBytes(const char * /*bytes*/, size_t /*size*/, const ReceivedDataTimestamp& timestamp, Session * /*sn*/) override
    {
        std::clog << "Received data timestamp: " << timestamp.toString() << std::endl;

        if(timestamp.source_ != ReceivedDataTimestamp::None)
        {
            const TimeSpan latency = UtcWatch::now() - timestamp.time_;
            std::clog << "Received data latency: " << toStr(latency, TimeSpanFormat::HHMMSSpsec) << std::endl;
        }
    }
};

Depending on the Session configuration, hardware capabilities, and the operation system, ReceivedDataTimestamp::Source can have the following values:

• None: no timestamp is taken;
• Software: a timestamp provided by the OS kernel / user-level network stack is taken;
• Hardware: a timestamp provided by the network card is taken;

If accessible, the network interface card timestamp is used (ReceivedDataTimestamp::Source::Hardware); otherwise, the timestamp supplied by the OS kernel or the user-level network stack is used (ReceivedDataTimestamp::Source::Software).

To configure the session to take the timestamp, SessionSettings::enableRxTimestamp must be set:

SessionSettings settings;
settings.enableRxTimestamp(true);
Session session(settings, MarketSegmentId, &listener);

If the OnixS TcpDirectStack is used, the timestamping must be enabled on the stack level:

TcpDirectAttr attr;
attr.set("rx_timestamping", 1);
TcpDirectStack stack(attr);

If the Solarflare OpenOnload is used, The EF_RX_TIMESTAMPING environment variable must be set to 1:

EF_RX_TIMESTAMPING=1 onload ./onixs-ilink3-app

To compare hardware-based timestamps with the host time, it is necessary to synchronize the network interface card clock with the host clock, as described in IEEE 1588-2008.

For the Solarflare cards, the sfptpd utility can be used to perform this synchronization.

Note: According to our measurements, enabling this feature adds around 50-100 nanoseconds to the resulting latency.

Note: This functionality is available on Linux only.

This article describes how to detect TCP congestion using described functional.