(→Module parameters) |
(→Performance) |
||
(42 intermediate revisions by 2 users not shown) | |||
Line 5: | Line 5: | ||
At this moment BrandMeister provides several backends (options) to send outgoing UDP: | At this moment BrandMeister provides several backends (options) to send outgoing UDP: | ||
* socket | * socket | ||
− | * | + | * uring |
* AF_XDP | * AF_XDP | ||
* DPDK | * DPDK | ||
* Ixy | * Ixy | ||
+ | * verbs | ||
Line 15: | Line 16: | ||
== socket == | == socket == | ||
− | This is standard default backend that uses Berkley sockets for sending a traffic. It tries to send the data in non-blocking mode and has special transmission thread to re-send failed packets or offload main thread on high load (> 50% CPU core). | + | This is standard default backend that uses Berkley sockets for sending a traffic. It tries to send the data in non-blocking mode and has special transmission thread to re-send failed packets or offload main thread on high load (> 50% CPU core). '''Not available in builds for Debian 12 and higher.''' |
=== You have to use it if you have === | === You have to use it if you have === | ||
Line 30: | Line 31: | ||
''transmitter = "socket";'' | ''transmitter = "socket";'' | ||
− | == | + | == uring == |
− | This is | + | This is standard default backend in '''builds for Debian 12 and higher'''. Uses Berkley sockets and io_uing Kernel API for sending a traffic. |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
=== Performance === | === Performance === | ||
− | Passed performance tests to transmit to | + | Passed performance tests to transmit to 45K connections |
=== Configuration === | === Configuration === | ||
− | ''transmitter = " | + | ''transmitter = "uring <module parameters>";'' |
− | + | ||
− | + | ||
==== Module parameters ==== | ==== Module parameters ==== | ||
− | + | All these parameters are optional and override default settings | |
− | All these parameters are optional and override default settings | + | |
− | + | ||
<pre> | <pre> | ||
− | (- | + | (-c) --core-count <n> - set count of workers (default is 2) |
− | (- | + | (-l) --buffer-length <n> - set buffer length to <n> slots (default is 512) |
</pre> | </pre> | ||
− | |||
− | |||
== AF_XDP == | == AF_XDP == | ||
Line 65: | Line 55: | ||
=== Limitations === | === Limitations === | ||
− | * | + | * Requires to use a single ethernet interface for BrandMeister's traffic (IPv4 and IPv6, local site connectivity will not work) |
+ | * All traffic will be routed via default gateway (except loopback, see next bullet) | ||
+ | * Loopback addresses (127.0.0.1 and ::1) are handled by using Berkley sockets | ||
* Interface have to be configured to use the same count of TX and RX queues (please read man ''ethtool'') | * Interface have to be configured to use the same count of TX and RX queues (please read man ''ethtool'') | ||
* May have compatibility problems (NIC may have no support of XDP) | * May have compatibility problems (NIC may have no support of XDP) | ||
+ | * On intel 10g nics (ixgbe) xdp tx works only if xdp rx is also enabled | ||
=== Performance === | === Performance === | ||
Line 80: | Line 73: | ||
==== Module parameters ==== | ==== Module parameters ==== | ||
− | + | All these parameters are optional and override default settings | |
− | All these parameters are optional and override default settings | + | |
− | + | ||
<pre> | <pre> | ||
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) | (-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) | ||
Line 101: | Line 92: | ||
=== Limitations === | === Limitations === | ||
− | * | + | * All traffic will be routed via default gateway (except loopback, see next bullet) |
+ | * Loopback addresses (127.0.0.1 and ::1) are handled by using Berkley sockets | ||
* You have to have separate NIC or virtual detachable NIC port allowed to use for the DPDK transmission | * You have to have separate NIC or virtual detachable NIC port allowed to use for the DPDK transmission | ||
* Only DPDK port #0 will be used | * Only DPDK port #0 will be used | ||
Line 109: | Line 101: | ||
=== Performance === | === Performance === | ||
− | Passed performance tests to transmit to | + | Passed performance tests to transmit to 100K connections on '''ixgbe''' |
=== Configuration === | === Configuration === | ||
Line 129: | Line 121: | ||
==== Module parameters ==== | ==== Module parameters ==== | ||
− | + | All these parameters are optional and override default settings | |
− | All these parameters are optional and override default settings | + | |
− | + | ||
<pre> | <pre> | ||
(-c) --core-ratio <n> - ratio between NIC queues and DPDK cores (instead of default value of 4) | (-c) --core-ratio <n> - ratio between NIC queues and DPDK cores (instead of default value of 4) | ||
Line 162: | Line 152: | ||
<pre> | <pre> | ||
− | + | --core-ratio <n> | |
− | + | --queue-size <n> | |
− | + | --queue-count <n> | |
− | + | --buffer-length <n> | |
</pre> | </pre> | ||
''transmitter = "Modules/Dixie.so:eth0 0000:af:00.0 --queue-count 8";' | ''transmitter = "Modules/Dixie.so:eth0 0000:af:00.0 --queue-count 8";' | ||
+ | |||
+ | == verbs == | ||
+ | |||
+ | This is method is only suitable when you use [https://en.wikipedia.org/wiki/OpenFabrics_Alliance OFED/RDMA-enabled NIC], mostly [https://www.nvidia.com/en-us/networking/ Mellanox and its OEM derives]. Both IP over Ethernet and IP over InfiniBand are supported. | ||
+ | |||
+ | === Configuration === | ||
+ | ''transmitter = "verbs:<interface name> <module parameters>";'' | ||
+ | |||
+ | ''transmitter = "verbs:rocep175s0 --reference-interface vlan100";'' | ||
+ | |||
+ | ==== Module parameters ==== | ||
+ | All these parameters are optional and override default settings | ||
+ | <pre> | ||
+ | --core-count <n> - set count of workers (default is 1) | ||
+ | --queue-size <n> - set queue size manually (default is 512) | ||
+ | --device-port <n> - device port (default is 1) | ||
+ | --buffer-length <n> - set workers buffer length to <n> slots (default is 1024) | ||
+ | --reference-interface <ipoib0> - reference interface to monitor routes, gateways | ||
+ | </pre> | ||
= UDP Receiver = | = UDP Receiver = | ||
Line 174: | Line 183: | ||
In reception part UDPTX's driver works in parallel with socket receiver. All it does, is accelerate reception of UDP packets on particular interface. | In reception part UDPTX's driver works in parallel with socket receiver. All it does, is accelerate reception of UDP packets on particular interface. | ||
− | * eBPF + AF_XDP | + | * socket (in dedicated thread) |
+ | * uring | ||
+ | * eBPF + AF_XDP | ||
+ | * eBPF + AF_XDP + XDPHelper | ||
* verbs | * verbs | ||
+ | |||
+ | == socket == | ||
+ | |||
+ | This method allows Core to offload main thread on high-loaded servers by reducing amount of system calls to receive UDP messages. '''Not available in builds for Debian 12 and higher.''' | ||
+ | |||
+ | === Configuration === | ||
+ | ''receiver = "socket <module parameters>";'' | ||
+ | |||
+ | ==== Module parameters ==== | ||
+ | All these parameters are optional and override default settings | ||
+ | <pre> | ||
+ | (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) | ||
+ | </pre> | ||
+ | |||
+ | == uring == | ||
+ | |||
+ | This method allows Core to offload main thread on high-loaded servers by reducing amount of system calls to receive UDP messages. | ||
+ | '''Available in builds for Debian 12 and higher.''' | ||
+ | |||
+ | === Configuration === | ||
+ | ''receiver = "uring <module parameters>";'' | ||
+ | |||
+ | ==== Module parameters ==== | ||
+ | All these parameters are optional and override default settings | ||
+ | <pre> | ||
+ | (-l) --buffer-length <n> - set buffer length to <n> slots (default is 2048) | ||
+ | </pre> | ||
== eBPF + AF_XDP == | == eBPF + AF_XDP == | ||
Line 194: | Line 233: | ||
==== Module parameters ==== | ==== Module parameters ==== | ||
− | + | All these parameters are optional and override default settings | |
− | All these parameters are optional and override default settings | + | |
− | + | ||
<pre> | <pre> | ||
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) | (-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) | ||
Line 214: | Line 251: | ||
* The same list of limitations as in case of eBPF + AF_XDP except support of multiple instances and eBPF preloading | * The same list of limitations as in case of eBPF + AF_XDP except support of multiple instances and eBPF preloading | ||
* All instances of BrandMeister Core should use the same network interface | * All instances of BrandMeister Core should use the same network interface | ||
− | * Multiple instances only work on NICs with ''' | + | * Multiple instances only work on NICs with '''N-tuples''' or '''flow-control''' features |
=== Configuration === | === Configuration === | ||
Line 222: | Line 259: | ||
==== Module parameters ==== | ==== Module parameters ==== | ||
− | All these parameters are optional and override default settings | + | All these parameters are optional and override default settings |
<pre> | <pre> | ||
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) | (-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) | ||
Line 236: | Line 273: | ||
This is method is only suitable when you use [https://en.wikipedia.org/wiki/OpenFabrics_Alliance OFED/RDMA-enabled NIC], mostly [https://www.nvidia.com/en-us/networking/ Mellanox and its OEM derives]. NIC should have support of Flow Steering. Both IP over Ethernet and IP over InfiniBand are supported. | This is method is only suitable when you use [https://en.wikipedia.org/wiki/OpenFabrics_Alliance OFED/RDMA-enabled NIC], mostly [https://www.nvidia.com/en-us/networking/ Mellanox and its OEM derives]. NIC should have support of Flow Steering. Both IP over Ethernet and IP over InfiniBand are supported. | ||
− | In case of Ethernet card it should have N-tuples enabled ('''ethtool -K eth0 ntuple on'''). | + | In case of Ethernet card it should have N-tuples enabled ('''ethtool -K eth0 ntuple on'''). Mellanox Connectx cards need device managed flow steering (options mlx4_core log_num_mgm_entry_size=-1) |
=== Configuration === | === Configuration === | ||
''receiver = "verbs:<interface name> <module parameters>";'' | ''receiver = "verbs:<interface name> <module parameters>";'' | ||
+ | |||
+ | ''receiver = "verbs:rocep175s0 --reference-interface vlan100";'' | ||
==== Module parameters ==== | ==== Module parameters ==== | ||
− | All these parameters are optional and override default settings | + | All these parameters are optional and override default settings |
<pre> | <pre> | ||
--queue-size <n> - set queue size manually (default is 512) | --queue-size <n> - set queue size manually (default is 512) | ||
− | |||
--reference-interface <eth0.1> - reference interface to monitor routes, gateways | --reference-interface <eth0.1> - reference interface to monitor routes, gateways | ||
</pre> | </pre> |
UDPTX is BrandMeister-own UDP communication library, used to transmit and receive UDP traffic fast. It is very important for BrandMeister to spend less time to send and receive packets, it makes transmission (and finally sound) more smooth.
At this moment BrandMeister provides several backends (options) to send outgoing UDP:
This is standard default backend that uses Berkley sockets for sending a traffic. It tries to send the data in non-blocking mode and has special transmission thread to re-send failed packets or offload main thread on high load (> 50% CPU core). Not available in builds for Debian 12 and higher.
Passed performance tests to transmit to 20K connections
transmitter = "socket";
This is standard default backend in builds for Debian 12 and higher. Uses Berkley sockets and io_uing Kernel API for sending a traffic.
Passed performance tests to transmit to 45K connections
transmitter = "uring <module parameters>";
All these parameters are optional and override default settings
(-c) --core-count <n> - set count of workers (default is 2) (-l) --buffer-length <n> - set buffer length to <n> slots (default is 512)
This is faster forwarding backend that uses AF_XDP socket of Ethernet interface for sending a traffic and in most cases communicates directly with Linux network interface driver. Due to monopolise NIC queues any configuration with multiple cores has to use ranges of queues per instance (for example: instance 1 uses 8 queues staring from 0 and instance 2 uses 8 queues starting from 8).
Passed performance tests to transmit to 55K connections
transmitter = "xdp:<interface name> <module parameters>";
transmitter = "xdp:eth0";
All these parameters are optional and override default settings
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) (-i) --in-flight <n> - set number of packets in flight (default value is calculated) (-c) --core-ratio <n> - ratio between NIC queues and transmitting cores (instead of default value of 2) (-f) --first-queue <n> - set number of first transmitting queue manually (default is 0) (-q) --queue-count <n> - set transmission queue count manually (instead of all available queues) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) (-r) --reference-interface <s> - reference interface to monitor routes, gateways
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is fastest forwarding backend that uses kernel-bypass NIC driver for sending a traffic. It allows to save much more CPU time due to direct poll communications to the NIC and CRC offload features of some NIC models. In some tests we got up to 75% acceleration. List of supported NIC models can be found here.
Passed performance tests to transmit to 100K connections on ixgbe
transmitter = "Modules/DPDK-edge.so:<reference interface> <EAL parameters> [--] <module parameters>";
transmitter = "Modules/DPDK-edge.so:eth0 -a 0000:af:00.0 --file-prefix bm --lcores '(0-8)@1,3,5,7,9' -- -c 1 -q 2048 -b 1 -l 4096";
# /etc/systemd/system/brandmeister@.service.d/override.conf [Service] User=root
All these parameters are optional and override default settings
(-c) --core-ratio <n> - ratio between NIC queues and DPDK cores (instead of default value of 4) (-q) --queue-size <n> - set PMD queue size to <n> slots (instead of automatically generated) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 512) (-p) --pthresh <n> | (-h) --hthresh <n> | PMD specific threshold values: (-w) --wthresh <n> | https://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html#configuration-of-transmit-queues (-r) --rs-thresh <n> | (-f) --free-thresh <n> | (-s) --software-crc - force software CRC calculation
Ixy is very experimental and light user-space network driver. At this moment it supports Intel 82599ES family (aka Intel X520) and virtio. Please read Ixy documentation.
Passed performance tests to transmit 120K connections
transmitter = "Modules/Dixie.so:<reference interface> <PCI address> <module parameters>";
--core-ratio <n> --queue-size <n> --queue-count <n> --buffer-length <n>
transmitter = "Modules/Dixie.so:eth0 0000:af:00.0 --queue-count 8";'
This is method is only suitable when you use OFED/RDMA-enabled NIC, mostly Mellanox and its OEM derives. Both IP over Ethernet and IP over InfiniBand are supported.
transmitter = "verbs:<interface name> <module parameters>";
transmitter = "verbs:rocep175s0 --reference-interface vlan100";
All these parameters are optional and override default settings
--core-count <n> - set count of workers (default is 1) --queue-size <n> - set queue size manually (default is 512) --device-port <n> - device port (default is 1) --buffer-length <n> - set workers buffer length to <n> slots (default is 1024) --reference-interface <ipoib0> - reference interface to monitor routes, gateways
In reception part UDPTX's driver works in parallel with socket receiver. All it does, is accelerate reception of UDP packets on particular interface.
This method allows Core to offload main thread on high-loaded servers by reducing amount of system calls to receive UDP messages. Not available in builds for Debian 12 and higher.
receiver = "socket <module parameters>";
All these parameters are optional and override default settings
(-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024)
This method allows Core to offload main thread on high-loaded servers by reducing amount of system calls to receive UDP messages. Available in builds for Debian 12 and higher.
receiver = "uring <module parameters>";
All these parameters are optional and override default settings
(-l) --buffer-length <n> - set buffer length to <n> slots (default is 2048)
This is modern method to accelerate UDP reception in BrandMeister Core. It allows to save up to 30% CPU time.
receiver = "Modules/ExpressFilter.o:<interface name> <module parameters>";
receiver = "Modules/ExpressFilter.o:eth0 --reference-interface eth0.1";
All these parameters are optional and override default settings
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) (-f) --first-queue <n> - set number of first receiving queue manually (default is 0) (-q) --queue-count <n> - set receiving queue count manually (instead of all available queues) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) (-r) --reference-interface <s> - reference interface to monitor routes, gateways
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is method is fully the same as eBPF + AF_XDP but uses small additional daemon XDPHelper to load and share eBPF program between several BrandMeister Core instances. This method also helps when NIC resets on eBPF load such as isgbe. XDPHelper is supplied with BrandMeister Core and starts automatically only when required (thanks to systemd and D-BUS activation). By default XDPHelper uses eBPF program ExpressFilter.o (see xdphelper.service).
receiver = "xdp:<interface name> <module parameters>";
receiver = "xdp:eth0 --reference-interface eth0.1";
All these parameters are optional and override default settings
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) (-f) --first-queue <n> - set number of first receiving queue manually (default is 0) (-q) --queue-count <n> - set receiving queue count manually (instead of all available queues) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) (-r) --reference-interface <s> - reference interface to monitor routes, gateways
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is method is only suitable when you use OFED/RDMA-enabled NIC, mostly Mellanox and its OEM derives. NIC should have support of Flow Steering. Both IP over Ethernet and IP over InfiniBand are supported. In case of Ethernet card it should have N-tuples enabled (ethtool -K eth0 ntuple on). Mellanox Connectx cards need device managed flow steering (options mlx4_core log_num_mgm_entry_size=-1)
receiver = "verbs:<interface name> <module parameters>";
receiver = "verbs:rocep175s0 --reference-interface vlan100";
All these parameters are optional and override default settings
--queue-size <n> - set queue size manually (default is 512) --reference-interface <eth0.1> - reference interface to monitor routes, gateways
UDPTX is BrandMeister-own UDP communication library, used to transmit and receive UDP traffic fast. It is very important for BrandMeister to spend less time to send and receive packets, it makes transmission (and finally sound) more smooth.
At this moment BrandMeister provides several backends (options) to send outgoing UDP:
This is standard default backend that uses Berkley sockets for sending a traffic. It tries to send the data in non-blocking mode and has special transmission thread to re-send failed packets or offload main thread on high load (> 50% CPU core).
Passed performance tests to transmit to 20K connections
transmitter = "socket";
This is fast forwarding backend that uses RAW (PACKET_MMAP) socket of Ethernet interface for sending a traffic. It allows to save up to 50% CPU time and has great compatibility.
Passed performance tests to transmit to 40K connections
transmitter = "raw:<interface name> <module parameters>";
transmitter = "raw:eth0 --count 2";
All these parameters are optional and override default settings of XDP module
(-b) --bypass - bypass QDisc traffic control (PACKET_QDISC_BYPASS) (-c) --count <n> - count of transmitter cores (default is 1)
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is faster forwarding backend that uses AF_XDP socket of Ethernet interface for sending a traffic and in most cases communicates directly with Linux network interface driver. Due to monopolise NIC queues any configuration with multiple cores has to use ranges of queues per instance (for example: instance 1 uses 8 queues staring from 0 and instance 2 uses 8 queues starting from 8).
Passed performance tests to transmit to 55K connections
transmitter = "xdp:<interface name> <module parameters>";
transmitter = "xdp:eth0";
All these parameters are optional and override default settings of XDP module
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) (-i) --in-flight <n> - set number of packets in flight (default value is calculated) (-c) --core-ratio <n> - ratio between NIC queues and transmitting cores (instead of default value of 2) (-f) --first-queue <n> - set number of first transmitting queue manually (default is 0) (-q) --queue-count <n> - set transmission queue count manually (instead of all available queues) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) (-r) --reference-interface <s> - reference interface to monitor routes, gateways
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is fastest forwarding backend that uses kernel-bypass NIC driver for sending a traffic. It allows to save much more CPU time due to direct poll communications to the NIC and CRC offload features of some NIC models. In some tests we got up to 75% acceleration. List of supported NIC models can be found here.
Passed performance tests to transmit to 60K connections on ixgbe
transmitter = "Modules/DPDK-edge.so:<reference interface> <EAL parameters> [--] <module parameters>";
transmitter = "Modules/DPDK-edge.so:eth0 -a 0000:af:00.0 --file-prefix bm --lcores '(0-8)@1,3,5,7,9' -- -c 1 -q 2048 -b 1 -l 4096";
# /etc/systemd/system/brandmeister@.service.d/override.conf [Service] User=root
All these parameters are optional and override default settings of PMD or DPDK module
(-c) --core-ratio <n> - ratio between NIC queues and DPDK cores (instead of default value of 4) (-q) --queue-size <n> - set PMD queue size to <n> slots (instead of automatically generated) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 512) (-p) --pthresh <n> | (-h) --hthresh <n> | PMD specific threshold values: (-w) --wthresh <n> | https://doc.dpdk.org/guides/prog_guide/poll_mode_drv.html#configuration-of-transmit-queues (-r) --rs-thresh <n> | (-f) --free-thresh <n> | (-s) --software-crc - force software CRC calculation
Ixy is very experimental and light user-space network driver. At this moment it supports Intel 82599ES family (aka Intel X520) and virtio. Please read Ixy documentation.
Passed performance tests to transmit 120K connections
transmitter = "Modules/Dixie.so:<reference interface> <PCI address> <module parameters>";
--core-ratio <n> --queue-size <n> --queue-count <n> --buffer-length <n>
transmitter = "Modules/Dixie.so:eth0 0000:af:00.0 --queue-count 8";'
In reception part UDPTX's driver works in parallel with socket receiver. All it does, is accelerate reception of UDP packets on particular interface.
This is modern method to accelerate UDP reception in BrandMeister Core. It allows to save up to 30% CPU time.
receiver = "Modules/ExpressFilter.o:<interface name> <module parameters>";
receiver = "Modules/ExpressFilter.o:eth0 --reference-interface eth0.1";
All these parameters are optional and override default settings of XDP module
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) (-f) --first-queue <n> - set number of first receiving queue manually (default is 0) (-q) --queue-count <n> - set receiving queue count manually (instead of all available queues) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) (-r) --reference-interface <s> - reference interface to monitor routes, gateways
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is method is fully the same as eBPF + AF_XDP but uses small additional daemon XDPHelper to load and share eBPF program between several BrandMeister Core instances. This method also helps when NIC resets on eBPF load such as isgbe. XDPHelper is supplied with BrandMeister Core and starts automatically only when required (thanks to systemd and D-BUS activation). By default XDPHelper uses eBPF program ExpressFilter.o (see xdphelper.service).
receiver = "xdp:<interface name> <module parameters>";
receiver = "xdp:eth0 --reference-interface eth0.1";
All these parameters are optional and override default settings.
(-w) --wakeup - use XDP in wakeup mode (XDP_USE_NEED_WAKEUP) (-f) --first-queue <n> - set number of first receiving queue manually (default is 0) (-q) --queue-count <n> - set receiving queue count manually (instead of all available queues) (-l) --buffer-length <n> - set workers buffer length to <n> slots (instead of default value of 1024) (-r) --reference-interface <s> - reference interface to monitor routes, gateways
Note: in case when you need to use VLANs, you have pass a physical interface (eth0 for example) as interface to bind and use --reference-interface key to pass the name of VLAN interface (eth0.1 for example)
This is method is only suitable when you use OFED/RDMA-enabled NIC, mostly Mellanox and its OEM derives. NIC should have support of Flow Steering. Both IP over Ethernet and IP over InfiniBand are supported. In case of Ethernet card it should have N-tuples enabled (ethtool -K eth0 ntuple on).
receiver = "verbs:<interface name> <module parameters>";
All these parameters are optional and override default settings.
--queue-size <n> - set queue size manually (default is 512) --device-port <n> - device port (default is 1) --reference-interface <eth0.1> - reference interface to monitor routes, gateways