Test includes the Non-RT RIC components: Policy Management Service (aka Policy Agent), A1-Controller and A1 simulators

Tested on one Cloud VM - all components and simulators running on the same machine using docker - with the following spec: 8CPU, 16GB RAM and Ubuntu 18.04

(Note: this is a benchmark report running on google cloud instance)

Each test is using 20 A1-Simulator. 7 processes are running in parallel to create/delete a total of 10000 policies (500 in each of the 20 A1-Simulator)

"REST" in the table below indicate the NB REST interface of the Policy Management Service.

"HTTP" and "HTTPS" indicate which protocol has been used between the components.

Tested with the latest available images: Thu Oct 9 2020

Test case setup	Protocol	Operation	Time (seconds)
REST towards Policy Management Service connecting to the simulators	HTTP	Create policy	36
REST towards Policy Management Service connecting to the simulators	HTTPS	Create policy	45
REST towards Policy Management Service connecting via the A1-Controller to the simulators	HTTP	Create policy	61
REST towards Policy Management Service connecting via the A1-Controller to the simulators	HTTPS	Create policy	63
REST towards Policy Management Service connecting to the simulators	HTTP	Delete policy	35
REST towards Policy Management Service connecting to the simulators	HTTPS	Delete policy	39
REST towards Policy Management Service connecting via the A1-Controller to the simulators	HTTP	Delete policy	74
REST towards Policy Management Service connecting via the A1-Controller to the simulators	HTTPS	Delete policy	69

Timer measurement in the test script
====================================
Activity Duration
Create 10000 polices over REST using HTTP 36 seconds
Delete 10000 polices over REST using HTTP 35 seconds
Create 10000 polices over REST+SDNC using HTTP 45 seconds
Delete 10000 polices over REST+SDNC using HTTP 39 seconds
Create 10000 polices over REST using HTTPS 61 seconds
Delete 10000 polices over REST using HTTPS 63 seconds
Create 10000 polices over REST+SDNC using HTTPS 74 seconds
Delete 10000 polices over REST+SDNC using HTTPS 69 seconds

Benchmark tests - 20201009

benchmark log: