Note: This page is a work in progress (WIP).

Introduction

This tutorial outlines the steps to deploy a QOE prediction model on KServe using RAppManager. It includes instructions for setting up the environment, installing necessary components, and deploying the model.

For the UAV path prediction use case: TBD.

Case 1: Deploying the QoE Prediction Model

Prerequisites

Ensure you have the following setup:

• Operating System: Ubuntu 22.04 Server

• CPU: 16 cores

• RAM: 32 GB

• Disk Space: 100 GB

• Root Access: Required

Installation Steps

1. Install Required Components:

   • AIML Framework Manager (aimlfmw)

   • RAN PM (ranpm)

   • KServe Inference Service

2. Create Feature Group, Training Function, and Start Training Job:

   • Set up the required feature group.

   • Create the training function.

   • Initiate the training job.

3. Patch KServe:

   • Apply necessary patches to KServe after installation.

4. Install RAppManager, DMEParticipant, and ACM:

   • Install the required modules for RAppManager.

   • Onboard, prime, and deploy the QoE RApp via RAppManager.

Cloning Repositories

Install Training Host

1. Switch to root and navigate to the directory:

   sudo su cd aimlfw-dep

2. Edit the configuration file:

   nano RECIPE_EXAMPLE/example_recipe_latest_stable.yaml

3. Run the installation script:

   bin/install_traininghost.sh

Postgres Fix (If Needed)

Note: This issue with bitnami postgres might arise if previous resources were not completely cleaned. In that case the password for user: postgres will not be the one set by the secret)

If Postgres issues arise during setup (for example TM pod in traininghost namespace is in crashbackloop because unable to reach the database), follow these steps:

1. Delete and reinstall the Postgres database with a custom password:

   helm delete tm-db -n traininghost kubectl delete pvc data-tm-db-postgresql-0 -n traininghost helm install tm-db bitnami/postgresql --namespace traininghost \ --set auth.postgresPassword=mypwd \ --set volumePermissions.enabled=true \ --set auth.username=aa --set auth.password=aa \ --set auth.usePasswordFiles=true kubectl wait --for=condition=Ready -n traininghost pod/tm-db-postgresql-0 --timeout=200s

2. Check psql -U postgres and if you cannot login with password: mypwd : Edit the pg_hba.conf file to disable password authentication:

   kubectl exec -it tm-db-postgresql-0 -n traininghost -c postgresql -- /opt/bitnami/scripts/postgresql/entrypoint.sh /bin/bash cat /opt/bitnami/postgresql/conf/pg_hba.conf sed -ibak 's/^\([^#]*\)md5/\1trust/g' /opt/bitnami/postgresql/conf/pg_hba.conf pg_ctl reload

3. Check again psql -U postgres . In rare cases where the Postgres database isn’t found after step 2:

   psql --username=postgres --dbname=template1 CREATE DATABASE postgres; exit

4. Set the Postgres password:

   psql -U postgres alter user postgres with password 'mypwd'; exit

5. Revert the configuration back to md5 and reload:

   sed -i 's/^\([^#]*\)trust/\1md5/g' /opt/bitnami/postgresql/conf/pg_hba.conf pg_ctl reload exit

Restart Kubeflow Pods

To ensure smooth operation, restart all Kubeflow pods:

Deploy KServe Inference Service

1. Install and patch KServe:

   bin/install_kserve_inference.sh -f RECIPE_EXAMPLE/example_recipe_latest_stable.yaml kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.11.1/kserve.yaml kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.11.1/kserve-runtimes.yaml kubectl patch configmap/inferenceservice-config -n kserve --type=strategic -p '{"data": {"deploy": "{\"defaultDeploymentMode\": \"RawDeployment\"}"}}'

Install RAN PM

1. Modify the image prefix in global-values.yaml:

   echo "Ranpm change image prefix in helm/global_values" cd ../ranpm/install nano helm/global-values.yaml sed -i 's/"pmCounterNumber101"/"pmCounterNumber101", "pdcpBytesUl", "pdcpBytesDl"/' \ NRT_PM_LOG_JOB_FILE="ranpm/install/helm/nrt-pm-log/config/jobDefinition.json" \ NRT_PM_RAPP_JOB_FILE="ranpm/install/helm/nrt-pm-rapp/config/jobDefinition.json"

2. Run the installation script:

   bash install-ranpm.sh

If ISTIO is not installed, you can install it manually or use the the script installation via this step Up: Release K: Creating & running an ML-based rApp | Deploy KServe Inference Service

Model Creation via DME

Follow the AIML Framework Installation Guide to create the model via DME.

1. Reinstall AIML:
   Update the example_recipe_latest_stable.yaml with your machine’s IP and Datalake information, then reinstall:

   cd aimlfw-dep nano RECIPE_EXAMPLE/example_recipe_latest_stable.yaml bin/uninstall.sh bin/install.sh -f RECIPE_EXAMPLE/example_recipe_latest_stable.yaml

2. Get InfluxDB Token:
   Run the script to retrieve access tokens:

   cd aimlfw-dep/demos/hrelease/scripts ./get_access_tokens.sh

Feature Group and Training Job Creation

1. SSH into the environment with port forwarding:

   ssh -L 32005:localhost:32005 -L 32002:localhost:32002 -L 32088:localhost:32088 -L 18888:localhost:18888 -L 8080:localhost:8080 USER@IP

2. Run the QoE Pipeline Jupyter Notebook:
   Open qoe-pipeline.ipynb in Jupyter and run all cells. Ensure the response code is 200.
   Visit: http://localhost:32088/tree

3. Create Feature Group:
   Visit: http://localhost:32005/TrainingJob/CreateFeatureGroup

4. Push QoE Data:

   ./prepare_env_aimlfw_access.sh ./push_qoe_data.sh gnb300505 30 c4/B2

5. Verify Data in InfluxDB:

   kubectl exec -it influxdb2-0 -n nonrtric -- bash -c "influx query 'from(bucket: \"pm-logg-bucket\") |> range(start: -1000000000000000000d)' |grep pdcpBytesDl"

6. Create Training Job:
   Visit: http://localhost:32005/TrainingJob/CreateTrainingJob

^{NO UI STEPS (IF YOU WISH TO CURL DIRECTLY AIML DASHBOARD)
1b. CREATE A FEATURE GROUP (CHANGE THE "Host":"IP" AND THE "token":"TOKEN" IN THE BODY)}

^{2b.CREATE A TRAINING JOB}

^{3b.START A TRAINING JOB (Make sure you have the training function if not use UI steps)}

4b. GET INFOS

Final Step: Deploying the QoE Prediction Service Using Postman

1. Install Only the Required Services (ACM, RAppManager, DMEParticipant, Capifcore, ServiceManager, Kong)

To deploy the QoE prediction service, you'll need to install only a subset of the services, such as ACM, RAppManager, and others. Follow these steps:

1. Modify the install-nonrtric.sh Script:

   Open the script to enable only the services you need:

   nano rappmanager/scripts/install/install-nonrtric.sh

Replace the first lines with:

Comment Out Unnecessary KServe Steps:

In the install-all.sh script, comment out any steps related to KServe installation or patching since you’ve already handled them.

Comment the following lines:

Run the Installation Script:
Now, run the installation script for RAppManager and the other enabled services:

Wait until the scripts exits successfully (In some cases it can take more than 30 minutes)

2. Create a Namespace for KServe Testing

Create a new Kubernetes namespace specifically for testing the KServe inference service:

3. Deploy the QoE Prediction Service via Postman or cURL

Once the services are up and running, you can now deploy the QoE prediction service using Postman or cURL.

Watch the video for the steps using the postman collection to Onboard Prime an Instantiate the rApp.

• Check the Status of the Inference Service:

1. Verify that the inference service for QoE is deployed in the kserve-test namespace:

   kubectl get inferenceservices.serving.kserve.io -n kserve-test

   The output should display the QoE model's status and endpoint.

2. Prepare the Input for QoE Prediction:

   Create a JSON file (input_qoe.json) with test data to send to the prediction model:

   echo '{"signature_name": "serving_default", "instances": [[[2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56], [2.56, 2.56]]]}'> input_qoe.json

3. Send a Prediction Request Using Postman or cURL:

   To send a prediction request, you can either use Postman or the curl command. Here’s the curl command example:

   curl -v -H "Host: qoe-model.kserve-test.example.com" <http://<KONG_IP>>:<KONG_PORT>/v1/models/qoe-model:predict -d @./input_qoe.json #Example of calling directly the inference service curl -v -H "Host: qoe-model.kserve-test.example.com" http://10.99.159.42:80/v1/models/qoe-model:predict -d @./input_qoe.json

   Replace <KONG_IP> with the IP address of your Kong proxy and <KONG_PORT> with the configured NodePort (e.g., 32083).

4. Validate the Response:

   You should receive a JSON response with the prediction results. This confirms that the QoE model has been successfully deployed and is functioning correctly.

Useful Files

Note: This csar package is generated via rappmanger/sample-rapp-generator/generate.sh

Video of the prediction

Open

Deploying the rApp

Open

Undeploying the rApp

2 rApps folders and their Postman Collection:

• rapp-kserve-qoe will deploy the model-predictor

• rapp-kserve-caller will query sme discovery endpoint, parse the information to query kong proxy and obtain a prediction.

This .zip files are meant to be unpackaged into rappmanager/sample-rapp-generator before the rappmanger installation OR before their use they need to be:

1. Initialized by “rappmanager/scripts/install/patch-sample-rapps.sh“ and then

2. Packaged by “rappmanager/sample-rapp-generator/generate.sh“

After obtaining the .csar rapp package, this can be used to onboard the rapp via rappmanager postman collection.

Open

Logs of the caller after a successful installation of both rApps