2. How to Use

SPP consists of primary process for managing resources, secondary processes for forwarding packet, and SPP controller to accept user commands and sent it to SPP processes.

You must keep in mind the order of launching processes. Primary process must be launched before secondary. In addition, controller need to be launched before primary and secondary because it prepares TCP connections for communicating primary and secondary.

  1. SPP Controller
  2. SPP Primary
  3. SPP Secondary

2.1. SPP Controller

SPP controller is implemented as a python script spp.py.

$ python spp.py
primary port : 5555
secondary port : 6666
Welcome to the spp.   Type help or ? to list commands.

spp >

Controller communicate with primary via TCP port 5555 and with secondary processes via 6666 in defalt. You can change port number by using options. Please refer help message for options.

$ ./src/spp.py -h
usage: spp.py [-h] [-p PRI_PORT] [-s SEC_PORT] [-m MNG_PORT] [-ip IPADDR]

SPP Controller

optional arguments:
  -h, --help            show this help message and exit
  -p PRI_PORT, --pri-port PRI_PORT
                        primary port number
  -s SEC_PORT, --sec-port SEC_PORT
                        secondary port number
  -m MNG_PORT, --mng-port MNG_PORT
                        management port number
  -ip IPADDR, --ipaddr IPADDR
                        IP address

SPP commands describes how to manage SPP processes from SPP controller.

2.2. SPP Primary

SPP primary is a resource manager and initializing EAL for secondary processes.

To launch primary, run spp_primary with options.

$ sudo ./src/primary/src/primary/x86_64-native-linuxapp-gcc/spp_primary \
    -c 0x02 -n 4 \
    --socket-mem 512,512 \
    --huge-dir=/dev/hugepages \
    --proc-type=primary \
    -- \
    -p 0x03 \
    -n 4 \
    -s 192.168.122.1:5555

SPP primary is a DPDK application and it takes EAL options before application specific options. Briefly describe about supported options.

  • EAL options:
    • -c: core mask (one or two cores required)
    • –socket-mem: memory size on each of NUMA nodes
    • –huge-dir: path of hugepage dir
    • –proc-type: process type
  • Application options:
    • -p: port mask
    • -n: number of ring PMD
    • -s: IP address of controller and port prepared for primary

Note

You do not need to give two cores if you are not interested in statistics. SPP primary is able to run with only one core and use second one to show the statistics.

2.3. SPP Secondary

Secondary process behaves as a client of primary process and a worker for doing tasks.

This section describes about spp_nfv and spp_vm, which just simply forward packets similar to l2fwd. The difference between them is running on host or VM. spp_vm runs inside a VM as described in name.

2.3.1. Launch on Host

Run spp_nfv with options.

$ cd /path/to/spp
$ sudo ./src/nfv/src/nfv/x86_64-native-linuxapp-gcc/spp_nfv \
    -c 0x06 -n 4 \
    --proc-type=secondary \
    -- \
    -n 1 \
    -s 192.168.122.1:6666
  • EAL options:
    • -c: core mask (two cores required)
    • –proc-type: process type
  • Application options:
    • -n: secondary ID
    • -s: IP address of controller and port prepared for secondary

Secondary ID is used to identify for sending messages and must be unique among all of secondaries. If you attempt to launch a secondary process with the same ID, SPP controller does not accept it and assign unused number.

2.3.2. Launch on VM

To communicate DPDK application running on a VM, it is required to create a virtual device for the VM. In this instruction, launch a VM with qemu command and create vhost-user and virtio-net-pci devices on the VM.

Before launching VM, you need to prepare a socket file for creating vhost-user device. Socket file is created from SPP secondary as following.

spp > sec 1;add vhost 0

In this example, create socket file with index 0 from secondary of ID 1. Socket file is created as /tmp/sock0. It is used as a qemu option to add vhost interface.

Launch VM with qemu-system-x86_64 for x86 64bit architecture. Qemu takes many options for defining resources including virtual devices.

$ sudo qemu-system-x86_64 \
    -cpu host \
    -enable-kvm \
    -numa node,memdev=mem \
    -mem-prealloc \
    -hda /path/to/image.qcow2 \
    -m 4096 \
    -smp cores=4,threads=1,sockets=1 \
    -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \
    -device e1000,netdev=net0,mac=00:AD:BE:B3:11:00 \
    -netdev tap,id=net0,ifname=net0,script=/path/to/qemu-ifup \
    -nographic \
    -chardev socket,id=chr0,path=/tmp/sock0 \                   # /tmp/sock0
    -netdev vhost-user,id=net1,chardev=chr0,vhostforce \        # netdev for vhost-user
    -device virtio-net-pci,netdev=net1,mac=00:AD:BE:B4:11:00 \  # device for virtio-net-pci
    -monitor telnet::44911,server,nowait

This VM has two network interfaces. -device e1000 is a management network port which requires qemu-ifup to activate while launching. Management network port is used for login and setup the VM. -device virtio-net-pci is created for SPP or DPDK application running on the VM.

vhost-user is a backend of virtio-net-pci which requires a socket file /tmp/sock0 created from secondary with -chardev option.

For other options, please refer to QEMU User Documentation.

Note

To launch several VMs, you have to prepare qemu images for the VMs. You shortcut installing and setting up DPDK and SPP for each of VMs by creating a tmeplate image and copy it to the VMs.

After booted, you install DPDK and SPP in the VM as in the host.

Run spp_vm with options.

$ cd /path/to/spp
$ sudo ./src/vm/src/vm/x86_64-native-linuxapp-gcc/spp_vm \
    -c 0x03 -n 4 \
    --proc-type=primary \
    -- \
    -p 0x01 \
    -n 1 \
    -s 192.168.122.1:6666
  • EAL options:
    • -c: core mask (two cores required)
    • –proc-type: process type
  • Application options:
    • -p: port mask
    • -n: secondary ID
    • -s: IP address of controller and port prepared for secondary

spp_vm is also managed from SPP controller as same as on host. Secondary ID is used to identify for sending messages and must be unique among all of secondaries. If you attempt to launch a secondary process with the same ID, SPP controller does not accept it and assign unused number.

In this case, port mask option is -p 0x01 (using one port) because the VM is launched with just one vhost interface. You can use two or more ports if you launch VM with several vhost-user and virtio-net-pci interfaces.

Notice that spp_vm takes options similar to spp_primary, not spp_nfv. It means that spp_vm has responsibilities for initializing EAL and forwarding packets in the VM.

Note

spp_vm is actually running as primary process on a VM, but managed as secondary process from SPP controller. SPP does not support running resource manager as primary inside a VM. Client behaves as secondary, but actually a primary, running on the VM to communicate with other SPP procesess on host.

spp_vm must be launched with --proc-type=primary and -p [PORTMASK] options similar to primary to initialize EAL.