Skip to content

Ixia-c ARP, BGP and traffic with FRR as a DUT


This lab demonstrates validation of an FRR DUT for basic BGP peering, prefix announcements and passing of traffic between announced subnets. To run OTG protocols and flows, Ixia-c Traffic and Protocol Engine are used.

The same setup can be brought up using one of two methods:

Also, the same OTG test logic can be executed using one of two OTG clients:

Each method has its own benefits. With curl, you can try each individual OTG API call needed to complete the test. On the other hand, otgen demonstrates how all these steps could be easily executed with a single command. By comparing with curl get_metrics or get_state requests output, you can better understands the logic otgen is using to wait for the results of previous API calls to converge or complete. Finally, you can use otgen run source code as a starting point for custom test logic you could develop using gosnappi library.

Lab configuration



Layer 3 topology and generated traffic flows

IP Diagram


The lab uses otg.json configuration file with the following properties:

OTG Diagram

To request Ixia-c to use ARP to determine destination MAC address for a flow f1, the following flow properties are used. The dst parameter in the packet section uses auto mode. In addition, tx_rx section has to use names of emulated devices' IP interfaces, as in "tx_names": ["otg1.eth[0].ipv4[0]"].

  "flows":  [
      "tx_rx":  {
        "choice":  "device",
        "device":  {
          "mode":  "mesh",
          "tx_names":  [
          "rx_names":  [
      "packet":  [
          "choice":  "ethernet",
          "ethernet":  {
            "dst":  {
              "choice":  "auto",
              "auto":  "00:00:00:00:00:00"
            "src":  {
              "choice":  "value",
              "value":  "02:00:00:00:01:aa"

Quick start

  1. Clone this repository

    git clone --recursive
    cd otg-examples/docker-compose/cpdp-frr
  2. To run all the steps below at once using Docker Compose, execute:

    make all
    make clean
  3. To use Containerlab option, run:

    make all-clab
    make clean


  • Linux host or VM with sudo permissions and Docker support
  • Docker
  • curl command
  • watch command (optional)
  • jq command (optional)

Install components

  1. Install docker-compose and add yourself to docker group. Logout for group changes to take effect.

    sudo curl -L "$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
    sudo chmod +x /usr/local/bin/docker-compose
    sudo usermod -aG docker $USER
  2. For Containerlab use case, install the latest release. For more installation options see here.

    bash -c "$(curl -sL"
  3. Install otgen tool, version 0.6.2 or later.

    bash -c "$(curl -sL" -- -v 0.6.2
  4. Make sure /usr/local/bin is in your $PATH variable (by default this is not the case on CentOS 7)

    if ! command -v ${cmd} &> /dev/null && [ -x ${dir}/${cmd} ]; then
      echo "${cmd} exists in ${dir} but not in the PATH, updating PATH to:"
      echo $PATH
  5. Clone this repository

    git clone --recursive

Docker Compose option to deploy the lab

  1. Launch the deployment using Docker Compose

    cd otg-examples/docker-compose/cpdp-frr
    docker-compose up -d
    sudo docker ps
  2. Make sure you have all five containers running. The result should look like this

    CONTAINER ID   IMAGE                                                              COMMAND                  CREATED         STATUS         PORTS                                                                                      NAMES
    0ea1e56720ac   "/docker_im/opt/Ixia…"   3 seconds ago   Up 3 seconds                                                                                              cpdp-frr_protocol_engine_1_1
    44f4c9fb8b3e   "/docker_im/opt/Ixia…"   3 seconds ago   Up 3 seconds                                                                                              cpdp-frr_protocol_engine_2_1
    6e50d4cad6a6             "./bin/controller --…"   4 seconds ago   Up 4 seconds                                                                                              cpdp-frr_controller_1
    7fe400f12196                                        "/sbin/tini -- /usr/…"   4 seconds ago   Up 3 seconds                                                                                              cpdp-frr_frr_1
    2a7e1c124cbd      "./"        4 seconds ago   Up 3 seconds>5556/tcp, :::5556->5556/tcp,>50071/tcp, :::50072->50071/tcp   cpdp-frr_traffic_engine_2_1
    cbc0a64278cc      "./"        4 seconds ago   Up 3 seconds>5555/tcp, :::5555->5555/tcp,>50071/tcp, :::50071->50071/tcp   cpdp-frr_traffic_engine_1_1
  3. Interconnect traffic engine containers via a veth pair

    sudo ../../utils/ cpdp-frr_traffic_engine_1_1 cpdp-frr_frr_1 veth0 veth1
    sudo ../../utils/ cpdp-frr_traffic_engine_2_1 cpdp-frr_frr_1 veth2 veth3
  4. Check traffic and protocol engine logs to see if they picked up veth interfaces

    sudo docker logs cpdp-frr_traffic_engine_1_1
    sudo docker logs cpdp-frr_traffic_engine_2_1
    sudo docker logs cpdp-frr_protocol_engine_1_1
    sudo docker logs cpdp-frr_protocol_engine_2_1

Containerlab option to deploy the lab

  1. Launch the deployment using Containerlab

    cd otg-examples/docker-compose/cpdp-frr
    sudo -E containerlab deploy

Run tests, curl option

  1. Apply config

    curl -k "${OTG_HOST}/config" \
        -H "Content-Type: application/json" \
        -d @otg.json
  2. Start protocols

    curl -k "${OTG_HOST}/control/state" \
        -H  "Content-Type: application/json" \
        -d '{"choice": "protocol","protocol": {"choice": "all","all": {"state": "start"}}}'
  3. Fetch ARP table

    curl -sk "${OTG_HOST}/monitor/states" \
        -X POST \
        -H  'Content-Type: application/json' \
        -d '{ "choice": "ipv4_neighbors" }'
  4. Fetch BGP metrics (stop with Ctrl-c)

    watch -n 1 "curl -sk \"${OTG_HOST}/monitor/metrics\" \
        -X POST \
        -H  'Content-Type: application/json' \
        -d '{ \"choice\": \"bgpv4\" }'"
  5. Fetch BGP prefix announcements - TODO this doesn't show the actual announcements

    curl -sk "${OTG_HOST}/monitor/states" \
        -X POST \
        -H  'Content-Type: application/json' \
        -d '{ "choice": "bgp_prefixes" }'
  6. Start transmitting flows

    curl -sk "${OTG_HOST}/control/state" \
        -H  "Content-Type: application/json" \
        -d '{"choice": "traffic", "traffic": {"choice": "flow_transmit", "flow_transmit": {"state": "start"}}}'
  7. Fetch flow metrics (stop with Ctrl-c)

    watch -n 1 "curl -sk \"${OTG_HOST}/monitor/metrics\" \
        -X POST \
        -H  'Content-Type: application/json' \
        -d '{ \"choice\": \"flow\" }'"
  8. Fetch port metrics

    curl -sk "${OTG_HOST}/monitor/metrics" \
        -X POST \
        -H  'Content-Type: application/json' \
        -d '{ "choice": "port" }'

Run tests, otgen option

  1. Use one otgen run command to repeat all of the steps above. Note --rxbgp 2x parameter. We use it to tell otgen it should wait, after starting the protocols, until twice as many routes were received from the DUT, than were advertised by KENG. For our lab configuration it would be the signal that BGP protocol has converged. In other setups this parameter might be different.

    export OTG_API="https://localhost:8443"
    otgen --log info run --insecure --file otg.json --json --rxbgp 2x --metrics flow | jq
  2. To format output as a table, use the modified command below. Note, there will be no log output in this case, so be patient to wait for the table output to appear.

    otgen run --insecure --file otg.json --json --rxbgp 2x --metrics flow | otgen transform --metrics flow | otgen display --mode table

Destroy the lab

  • To destroy the lab brought up via Docker Compose, including veth pairs, use:

    docker-compose down
  • To destroy the lab brought up via Containerlab, use:

    sudo containerlab destroy -c


  • copyright of Levente Csikor, with modifications to replace ifconfig with ip link.