The Solarflare Feed Engine provides kernel-bypass multicast reception through Solarflare's ef_vi API for latency-sensitive applications on Linux with supported Solarflare network adapters.
The OnixS::CME::MDH::SolarflareFeedEngine class implements this receive path. Construct an explicit instance and bind it to the Handler as described in Using an explicit Feed Engine.
The Solarflare ef_vi API uses the concept of virtual interfaces to receive network data. The corresponding configuration parameters are available via the OnixS::CME::MDH::SolarflareFeedEngineSettings class:
The Solarflare Feed Engine can back its ef_vi receive-buffer pools with explicit 2 MiB Linux HugeTLB pages. A 2 MiB page covers the same memory as 512 regular 4 KiB pages, so using HugeTLB pages can reduce CPU translation lookaside buffer (TLB) misses and page-table walks while packets are received and processed. This can improve latency and reduce latency variation, particularly when many links or large buffer pools are active. The benefit depends on the hardware and workload; measure it under representative market-data traffic.
HugeTLB allocation is opt-in. Select PreferHuge to use HugeTLB pages when available while retaining a native-memory fallback:
HugeTLB allocation can fail with ENOMEM when the pool has insufficient free 2 MiB pages or with EINVAL when the kernel does not support the requested 2 MiB page size. PreferHuge falls back to native page-aligned memory after such a failure; RequireHuge causes multicast-link setup to fail.
Check that the kernel exposes a 2 MiB HugeTLB pool and inspect its current capacity:
nr_hugepages is the configured pool size. free_hugepages counts pages that are not yet allocated, including pages already committed to existing mappings; resv_hugepages counts those commitments. Therefore, free_hugepages - resv_hugepages is the uncommitted capacity that a new mapping can obtain from the existing pool. The Feed Engine needs enough uncommitted pages when each ef_vi multicast link is created. The following command also shows the system default HugeTLB page size and summary counters:
Reserve pages at runtime by writing the required count to the 2 MiB pool. For example, to reserve 128 pages (256 MiB):
Repeat the availability check and confirm that nr_hugepages reached the requested value. Runtime allocation can fall short when physical memory is fragmented. If 2 MiB is the default HugeTLB page size reported by /proc/meminfo, the equivalent setting is:
To retain this setting after reboot, add vm.nr_hugepages = 128 to a file under /etc/sysctl.d/ and run sudo sysctl --system. If runtime allocation is unreliable, reserve the pages at boot with the kernel parameters hugepagesz=2M hugepages=128; boot-time reservation occurs before memory becomes fragmented. See the Linux HugeTLB documentation for kernel configuration, boot-loader, and NUMA-specific options.
Each ef_vi multicast link requires the following number of 2 MiB pages for its receive-buffer pool:
The HugeTLB mapping length is rounded up to whole 2 MiB pages; this does not increase cacheSize or the memory region registered with ef_vi. Because each receive buffer occupies 2048 bytes, use a cacheSize that is a multiple of 1024 to avoid unused space at the end of the final HugeTLB page.
Reserve at least the sum of this value for every ef_vi multicast link that the engine may allocate, plus capacity required by other HugeTLB users. A handler allocates links for all of its multicast feeds during Handler::start(), before any feed connects. A channel can require up to eight links: incremental A/B, MBP recovery A/B, MBO recovery A/B, and instrument-definition A/B. Include all handlers that can use the engine when sizing the pool, rather than counting only connected feeds.
By default, ef_vi uses a finite buffer table on the network adapter to map registered packet memory. Each ef_vi multicast link created by the Solarflare Feed Engine has its own registered receive-buffer pool. A deployment with many links or large OnixS::CME::MDH::SolarflareFeedEngineSettings::cacheSize values can exhaust the available buffer table entries and cause multicast-link initialization to fail while registering the buffer pool.
Physical addressing mode requires permission from the sfc_char driver. For a non-root deployment, create a dedicated group for applications that may use physical mode and add the application user to it. For example:
Use the numeric group ID reported by getent to create a modprobe configuration such as /etc/modprobe.d/sfc_char.conf:
Apply the new value at runtime. It takes effect for protection domains created afterwards:
Alternatively, stop all applications using ef_vi or Onload and reload the Onload driver stack so that the modprobe configuration is re-read:
The user must start a new login session after being added to the group. Confirm that the group is listed by id before starting the application.
After configuring the driver permission, set EF_VI_PD_FLAGS to phys to select physical addressing mode. The setting must be present in the application's environment before the Solarflare Feed Engine creates its protection domains. It can be applied to a single invocation:
Alternatively, export it before starting the application:
Without the matching phys_mode_gid permission, the driver rejects protection-domain allocation for a non-root process. The Feed Engine then fails to initialize in conventional mode; in mixed mode, affected links may fall back to ordinary sockets instead of using ef_vi.
Physical addressing mode lets the adapter access packet buffers by their physical addresses, avoiding the buffer-table address-translation limit. Use it when the default mode cannot register all receive-buffer memory required by the configured links. Leave the variable unset when the default buffer-table capacity is sufficient.
Physical addressing mode and HugeTLB pages are independent settings. EF_VI_PD_FLAGS=phys changes how the network adapter addresses registered memory and avoids NIC buffer-table limits. HugeTLB configuration changes how Linux backs that memory and can reduce CPU translation lookaside buffer (TLB) misses. Enabling either feature does not enable or require the other.
This setting affects only links backed by ef_vi; socket-based links in mixed mode and TCP recovery connections are unaffected. See the AMD ef_vi User Guide for further details about packet-buffer addressing modes and their security implications.
The mixed mode for the Solarflare Feed Engine combines the ef_vi SDK with standard network sockets. This mode allows taking advantage of using the ultra-low latency ef_vi API on Solarflare network adapters while receiving some data from non-Solarflare network interfaces using the socket API.
The following code shows how to configure data receiving when Incremental data comes via the Solarflare network adapter (eth0) and recovery data is available on a non-Solarflare network interface (eth2).
Configuring consists of two steps.
The first step is to tell the Solarflare Feed Engine to work in the mixed mode:
The second step is to bind Handler's feeds to proper network interfaces:
