An introduction to RSS (Receive Side Scaling) in DPDK - Achieving Higher Receive Data Rates

Introduction

In this article, I have explained about a basic DPDK application which reads the packets from NIC interface. In this article we will learn about the concept of RSS (Receive Side Scaling) in DPDK. If you are passionate about fast packet processing using DPDK then this article is for you.

Setting up DPDK environment

The DPDK environment setup includes binding the NIC to the specific kernel modules to access them directly and setting up huge pages. All of this is explained in detail in the last article. Kindly go through it in case you need to setup your DPDK environment.

Before we write our application we need to learn about the concept of RSS (Receive Side Scaling) in DPDK.

Introduction to RSS (Receive Side Scaling)

Modern NICs (Network Interface Cards) can have one or more receive and transmit queues. Whenever a packet is received by NIC, it is enqueued into one of the receive queue. If a NIC has only one queue then all the received packets are enqueued into the only queue. If NIC has multiple queues, the NIC can distribute the packets to multiple receive queues.

The benefit of multiple queues is that we can start dedicated threads for each queue in our DPDK application which would be reading the packets from their respective queues. Reading multiple receive queues in parallel by using dedicated threads increases the packet processing rate and thus increasing the receive data rate.

Now the question arises: Under which rules, a NIC decides the destination queue for each received packet? We will discuss RSS (Receive Side Scaling) to answer this question.

RSS (Receive Side Scaling) is a technique by which NIC can distribute the received packets to the multiple receive queues. RSS works as follows:

1. A packet is received by the NIC. NIC will check whether the received packet belongs to configured RSS target packet type. While configuring RSS, we always specify the RSS target packet type i.e. on which kind of packets RSS will be applied. Some examples of RSS target packet types are: IPv4 UDP packet, IPv6 TCP packet, IP UDP packet, IPv4 non-fragmented UDP packet, etc.
   We can specify one or more RSS target packet types. For example: RSS will be applied on IPv4 UDP non-fragmented packets and IPv6 UDP non-fragmented packets.
   Note: The specified RSS target packet types must also be supported by the target NIC.

2. If the packet does not belong to the configured RSS target packet type(s), it will always be enqueue to receive queue 0. For Example: An IPv4 TCP non-fragmented packet is received and the configured RSS type is IPv4 UDP non-fragmented packet. In this case the received packet is queued to queue 0.

3. If the packet belongs to the configured RSS target packet type(s), RSS will be applied to the packet to find the target queue number as follows:
   a. NIC will fetch the Source IP address, Destination IP address, Source port, Destination port from the packet (if available).
   b. NIC will calculate the RSS Hash by using the fetched data from the packet and a RSS hash key. RSS hash key is supplied by the user while configuring RSS. The most famous RSS hash algorithm is Toeplitz hash algorithm.

   c. The result of the hash algorithm is a 32 bit value. NIC will take the least significant 7 bits of the hash value to find the target queue number from an indirection table.
   d. An indirection table is a table consisting of 128 entries where each entry contains the queue number. For a 7 bit lookup in the table, we can get a target queue number which would be used to enqueue the received packet.
   e. The indirection table is configured by the user i.e. for each row a target queue number is defined. A simple indirection table will look as below.

For the indirection table shown in the above image, If the 7-LSBs value of the hash value is 3 then the target receive queue would be 1. A user can configure the indirection table according to its own requirement by using the DPDK APIs.

Writing DPDK application

Finally, we can write our RSS based DPDK application in C++. In this DPDK application, we will configure RSS in with multiple receive queues of the NIC. The NIC will distribute the received packets according to the configured RSS parameters.

The written DPDK application is present at: github.com/awaiskhalidawan/dpdk-tutorials

Below are the steps to build and run the application:

git clone https://github.com/awaiskhalidawan/dpdk-tutorials.git
cd dpdk-tutorials/
mkdir build
cd build
cmake ..
make

The application binaries will be generated in dpdk-tutorials/bin folder after the successful execution of above steps.

To finally run the DPDK application:

cd dpdk-tutorials/bin
sudo ./receive-side-scaling --lcores=0-1 -n 4 --

Note: This application will successfully execute only on the NICs which supports RSS hashing. Not all the NICs support RSS hashing. The virtual NIC attached to the virtual machine in Virtual Box does not support RSS hashing so this application will not work with virtual NIC.

Congratulations! You have run a DPDK application which configures the RSS (Receive Side Scaling) with multiple receive queues. The NIC will distribute the received packets to multiple receive queues. There are two dedicated threads running in our DPDK application which reads the packets from their respective receive queues. In this way we delivers the receive packets to multiple cores without any lock and thus achieving higher packet processing rates.

There is an extensive explanations of DPDK APIs as well as the code flow in the code of this application. Don't forget to go through the code while you run this DPDK application.

Note: A DPDK application must be run with sudo for now as DPDK abstraction layer requires sudo privileges. I will write a detailed article later on about how to run DPDK applications without sudo privileges.

Summary

In this article, a simple DPDK application is written and executed. This application configures RSS with multiple receive queues of the NIC to distribute the received packets which are then read by the dedicated threads running by our DPDK application to achieve higher packet processing rates. We also discussed in detail about the working of RSS (Receive Side Scaling).

Feel free to write me in the comments if you are unable to perform the steps successfully, mentioned in this article.