Overview
Test includes the Non-RT RIC components: Policy Management Service (aka Policy Agent) , A1-Controller and A1 simulators
Tested on one Cloud VM - all components and simulators running on the same machine using docker or kubernetes - with the following spec: 84CPU, 16GB RAM and and Ubuntu 18.04 LTS
Docker runs all components a single node in the cluster
Kubernetes uses the whole cluster (6 worker noedes, 1 control node)
(Note: this is a benchmark report running docker on google a Xerces cloud instance with kubernetes running ONAP )
Each test is using 20 A1-Simulator. 7 processes are running in parallel to create/delete a total of 10000 policies (500 in each of the 20 A1-Simulator)
Images
Tested with the latest available images: Sat Jun 19 2021
Result - docker
Note that all simulators are as well as the test engine is running on the same machine.
"REST" in the table below indicate the NB REST interface of the Policy Management Service.
"HTTP" and "HTTPS" indicate which protocol has been used between the components.
Tested with the latest available images: Thu Oct 9 2020
| Test case setup | Protocol | Operation | Time (seconds) |
|---|---|---|---|
| REST towards Policy Management Service connecting to the simulators | HTTP | Create policy - 10000 | 3686 |
| REST towards Policy Management Service connecting to the simulators | HTTPS | Create policy - 10000 | 45143 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTP | Create policy - 10000 | 61305 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTPS | Create policy - 10000 | 63468 |
| REST towards Policy Management Service connecting to the simulators | HTTP | Delete policy - 10000 | 35 67 |
| REST towards Policy Management Service connecting to the simulators | HTTPS | Delete policy - 10000 | 39133 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTP | Delete policy - 10000 | 74269 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTPS | Delete policy - 10000 | 69399 |
Timer measurement in the test script
====================================
Activity Duration
Create 10000 polices over REST using HTTP 36 86 seconds
Delete 10000 polices over REST using HTTP 35 67 seconds
Create 10000 polices over REST+SDNC using HTTP 45 305 seconds
Delete 10000 polices over REST+SDNC using HTTP 39 269 seconds
Create 10000 polices over REST using HTTPS 61 143 seconds
Delete 10000 polices over REST using HTTPS 63 133 seconds
Create 10000 polices over REST+SDNC using HTTPS 74 468 seconds
Delete 10000 polices over REST+SDNC using HTTPS 399 seconds
Result - kubernetes
Note that the testengine runs on one node while the componentes/simulators are running on the worker nodes.
"REST" in the table below indicate the NB REST interface of the Policy Management Service.
"HTTP" and "HTTPS" indicate which protocol has been used between the components.
| Test case setup | Protocol | Operation | Time (seconds) |
|---|---|---|---|
| REST towards Policy Management Service connecting to the simulators | HTTP | Create policy - 10000 | 96 |
| REST towards Policy Management Service connecting to the simulators | HTTPS | Create policy - 10000 | 190 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTP | Create policy - 10000 | 321 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTPS | Create policy - 10000 | 508 |
| REST towards Policy Management Service connecting to the simulators | HTTP | Delete policy - 10000 | 84 |
| REST towards Policy Management Service connecting to the simulators | HTTPS | Delete policy - 10000 | 186 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTP | Delete policy - 10000 | 302 |
| REST towards Policy Management Service connecting via the A1-Controller to the simulators | HTTPS | Delete policy - 10000 | 499 |
Timer measurement in the test script
====================================
Activity Duration
Create 10000 polices over REST using HTTP 96 seconds
Delete 10000 polices over REST using HTTP 84 seconds
Create 10000 polices over REST+SDNC using HTTP 321 seconds
Delete 10000 polices over REST+SDNC using
...
HTTP 302 seconds
Create 10000 polices over REST using HTTPS 190 seconds
Delete 10000 polices over REST using HTTPS 186 seconds
Create 10000 polices over REST+SDNC using HTTPS 508 seconds
Delete 10000 polices over REST+SDNC using HTTPS 499 seconds