(Modules and versions)
(Modules and versions)
Line 61: Line 61:
 
By-default package '''brandmeister-core''' contains module DPDK.so built with DPDK supplied via Debian distribution. So, for Debian 9 it is DPDK 16.11 and DPDK 18.11 for Debian 10.
 
By-default package '''brandmeister-core''' contains module DPDK.so built with DPDK supplied via Debian distribution. So, for Debian 9 it is DPDK 16.11 and DPDK 18.11 for Debian 10.
 
To be on the top of edge we supply separate package '''brandmeister-dpdk-edge''' with most modern stable statically-linked version of DPDK, at this moment it is 19.11.
 
To be on the top of edge we supply separate package '''brandmeister-dpdk-edge''' with most modern stable statically-linked version of DPDK, at this moment it is 19.11.
In this packade the module has name DPDK-edge.so
+
In this packade the module has name DPDK-edge.so and is supported on Debian 10 only.
  
 
= UDP Receiver =
 
= UDP Receiver =

Revision as of 15:45, 21 January 2020

UDPTX is BrandMeister-own UDP communication library, used to transmit and receive UDP traffic fast.

UDP Transmitter

At this moment BrandMeister provides several backends (options) to send outgoing UDP:

  • socket
  • raw
  • DPDK

socket

This is standard default backend that uses Berkley sockets for sending a traffic. It tries to send the data in non-blocking mode and has special transmission thread to re-send failed packets.

You have to use it if you have

  • non-ethernet interfaces
  • more then one interface for outgoing traffic (such as public + AMPR, or one for IPv4 and another one for IPv6)
  • huge routing tables

Configuration

transmitter = "socket";

raw

This is fast forwarding backend that uses RAW (PACKET_MMAP) socket of Ethernet interface for sending a traffic. It allows to save up to 50% CPU time.

Limitations

  • Requires to use a single ethernet interface for BrandMeister's traffic (IPv4 and IPv6, local site connectivity will not work)
  • All traffic will be routed via default gateway (except loopback, see next bullet)
  • Loopback addresses (127.0.0.1 and ::1) are handled by using Berkley sockets

Configuration

transmitter = "raw:<interface name>";

transmitter = "raw:eth0";

DPDK

This is fastest forwarding backend that uses kernel-bypass NIC driver for sending a traffic. It allows to save much more CPU time due to direct poll communications to the NIC and CRC offload features of some NIC models. In some tests we got up to 75% acceleration. List of supported models can be found here.

Limitations

  • The same set of limitations as raw
  • You have to have separate NIC or virtual detachable NIC port allowed to use for the DPDK transmission
  • Only DPDK port #0 will be in use

Configuration

transmitter = "Modules/DPDK.so:<reference interface> <EAL parameters>";

transmitter = "Modules/DPDK.so:eth0 -w 0000:af:00.0 --file-prefix brandmeister --huge-unlink --lcores 0@0,1@0,2@0 --master-lcore 0";

  • Reference interface is a kernel attached interface used for normal communications (please read about raw mode). DPDK will reuse its IPs and default gateway
  • For EAL parameters please read this documentation
  • It will use so many NIC queues as you define slave logical cores in startup parameters. It might help to solve issue with packet drops at TX on some NICs such as Intel's
  • The best performance on NUMA machines could be reached by using the same CPU as NIC connected by pinning logical cores via EAL's parameter lcores and BrandMeister's parameter affinity
  • Also in most cases it requires to run BrandMeister with root privileges, you can do this by overriding systemd configuration (beandmeister@.service.d)

Modules and versions

By-default package brandmeister-core contains module DPDK.so built with DPDK supplied via Debian distribution. So, for Debian 9 it is DPDK 16.11 and DPDK 18.11 for Debian 10. To be on the top of edge we supply separate package brandmeister-dpdk-edge with most modern stable statically-linked version of DPDK, at this moment it is 19.11. In this packade the module has name DPDK-edge.so and is supported on Debian 10 only.

UDP Receiver

In reception part UDPTX's driver works in parallel with socket receiver. All it does, is accelerate reception of UDP packets on particular interface.

eBPF / AF_XDP

This is modern and only method to accelerate UDP reception in BrandMeister. It allows to save up to 30% CPU time. Requires Linux kernel >= 4.18.

Configuration

receiver = "Modules/ExpressFilter.o:<reference interface>";

receiver = "Modules/ExpressFilter.o:eth0";

Options

  • Modules/ExpressFilter.o - basic eBPF module, does filtering for the AF_XDP receiver
  • Modules/ExpressFilterPlus.o - extended eBPF module, in addition passes pre-parsed data (requires to have modern libbpf and kernel)