ETSI-DMS on O-Cloud, Tacker installation
This document describes how to deploy an openstack-helm environment using Ceph.
1. Preparation
* Setup StarlingX.
Install StarlingX R8.0 AIO-SX by following procedure.
* https://docs.starlingx.io/r/stx.8.0/deploy_install_guides/release/virtual/aio_simplex.html
* Install required packages on StarlingX environment.
```
$ sudo ostree admin unlock --hotfix
$ wget https://mirror.ufam.edu.br/debian/pool/main/libe/liberror-perl/liberror-perl_0.17029-1_all.deb
$ wget https://mirror.ufam.edu.br/debian/pool/main/g/git/git-man_2.30.2-1_all.deb
$ wget https://mirror.ufam.edu.br/debian/pool/main/g/git/git_2.30.2-1_amd64.deb
$ wget https://mirror.ufam.edu.br/debian/pool/main/m/make-dfsg/make_4.3-4.1_amd64.deb
$ sudo dpkg -i liberror-perl_0.17029-1_all.deb
$ sudo dpkg -i git-man_2.30.2-1_all.deb
$ sudo dpkg -i git_2.30.2-1_amd64.deb
$ sudo dpkg -i make_4.3-4.1_amd64.deb
```
* Clone openstack-helm and openstack-helm-infra code.
```
$ git clone https://opendev.org/openstack/openstack-helm.git
$ git clone https://opendev.org/openstack/openstack-helm-infra.git
```
* Add labels to controller-0 node.
```
$ kubectl label node controller-0 ceph-mgr=enabled
$ kubectl label node controller-0 ceph-mon=enabled
$ kubectl label node controller-0 ceph-mds=enabled
$ kubectl label node controller-0 ceph-rgw=enabled
$ kubectl label node controller-0 ceph-osd=enabled
$ kubectl label node controller-0 openstack-control-plane=enabled
```
* Create namespaces.
```
$ kubectl create namespace openstack
$ kubectl create namespace ceph
```
2. Deploy Ingress Controller
Original procedure: https://docs.openstack.org/openstack-helm/latest/install/developer/kubernetes-and-common-setup.html#deploy-the-ingress-controller
* Modify `openstack-helm/tools/deployment/component/common/ingress.sh` file as follows:
```
diff --git a/tools/deployment/component/common/ingress.sh b/tools/deployment/component/common/ingress.sh
index 9ae03719..884f93fc 100755
--- a/tools/deployment/component/common/ingress.sh
+++ b/tools/deployment/component/common/ingress.sh
@@ -29,6 +29,23 @@ deployment:
type: DaemonSet
network:
host_namespace: true
+endpoints:
+ ingress:
+ port:
+ http:
+ default: 10080
+ https:
+ default: 10443
+ healthz:
+ default: 11254
+ status:
+ default: 11246
+ stream:
+ default: 11247
+ profiler:
+ default: 11245
+ server:
+ default: 18181
EOF
touch /tmp/ingress-component.yaml
@@ -48,21 +65,21 @@ pod:
EOF
fi
-helm upgrade --install ingress-kube-system ${HELM_CHART_ROOT_PATH}/ingress \
- --namespace=kube-system \
- --values=/tmp/ingress-kube-system.yaml \
- ${OSH_EXTRA_HELM_ARGS} \
- ${OSH_EXTRA_HELM_ARGS_INGRESS} \
- ${OSH_EXTRA_HELM_ARGS_INGRESS_KUBE_SYSTEM}
-
-#NOTE: Wait for deploy
-./tools/deployment/common/wait-for-pods.sh kube-system
+#helm upgrade --install ingress-kube-system ${HELM_CHART_ROOT_PATH}/ingress \
+# --namespace=kube-system \
+# --values=/tmp/ingress-kube-system.yaml \
+# ${OSH_EXTRA_HELM_ARGS} \
+# ${OSH_EXTRA_HELM_ARGS_INGRESS} \
+# ${OSH_EXTRA_HELM_ARGS_INGRESS_KUBE_SYSTEM}
+#
+##NOTE: Wait for deploy
+#./tools/deployment/common/wait-for-pods.sh kube-system
#NOTE: Deploy namespace ingress
helm upgrade --install ingress-openstack ${HELM_CHART_ROOT_PATH}/ingress \
--namespace=openstack \
--values=/tmp/ingress-component.yaml \
- --set deployment.cluster.class=nginx \
+ --set deployment.cluster.class=nginx-openstack \
${OSH_EXTRA_HELM_ARGS} \
${OSH_EXTRA_HELM_ARGS_INGRESS} \
${OSH_EXTRA_HELM_ARGS_INGRESS_OPENSTACK}
```
* Execute `ingress.sh`.
```
$ cd $HOME/openstack-helm/
$ ./tools/deployment/component/common/ingress.sh
3. Deploy Ceph
Original Procedure: https://docs.openstack.org/openstack-helm/latest/install/developer/deploy-with-ceph.html#deploy-ceph
* Modify `openstack-helm/toolsdeployment/component/ceph/ceph.sh` file as follows:
```
diff --git a/tools/deployment/component/ceph/ceph.sh b/tools/deployment/component/ceph/ceph.sh
index 7d2550cd..518df21d 100755
--- a/tools/deployment/component/ceph/ceph.sh
+++ b/tools/deployment/component/ceph/ceph.sh
@@ -47,8 +47,10 @@ endpoints:
ceph_mgr:
namespace: ceph
network:
- public: 172.17.0.1/16
- cluster: 172.17.0.1/16
+ #public: 172.17.0.1/16
+ #cluster: 172.17.0.1/16
+ public: 192.168.206.1/24
+ cluster: 192.168.206.1/24
deployment:
storage_secrets: true
ceph: true
```
* Modify `openstack-helm/tools/deployment/component/ceph/ceph-ns-activate.sh` file as follows:
```
diff --git a/tools/deployment/component/ceph/ceph-ns-activate.sh b/tools/deployment/component/ceph/ceph-ns-activate.sh
index 9574d4f0..acadfc3f 100755
--- a/tools/deployment/component/ceph/ceph-ns-activate.sh
+++ b/tools/deployment/component/ceph/ceph-ns-activate.sh
@@ -27,8 +27,10 @@ endpoints:
ceph_mon:
namespace: ceph
network:
- public: 172.17.0.1/16
- cluster: 172.17.0.1/16
+ #public: 172.17.0.1/16
+ #cluster: 172.17.0.1/16
+ public: 192.168.206.1/24
+ cluster: 192.168.206.1/24
deployment:
ceph: false
rbd_provisioner: false
```
* Execute `ceph.sh` and `ceph-ns-activate.sh`.
```
$ cd $HOME/openstack-helm/
$ ./tools/deployment/component/ceph/ceph.sh
$ ./tools/deployment/component/ceph/ceph-ns-activate.sh
```
4. Deploy Other Component
Original Procedure: https://docs.openstack.org/openstack-helm/latest/install/developer/deploy-with-ceph.html#deploy-mariadb ~
Install the following components to use Tacker:
* MariaDB
* RabbitMQ
* Memcached
* Keystone
* Glance
* Modify `openstack-helm/tools/deployment/component/glance/glance.sh` file as follows:
```
diff --git a/tools/deployment/component/glance/glance.sh b/tools/deployment/component/glance/glance.sh
index e3b45078..56ab0907 100755
--- a/tools/deployment/component/glance/glance.sh
+++ b/tools/deployment/component/glance/glance.sh
@@ -27,7 +27,8 @@ make glance
tee /tmp/glance.yaml <<EOF
storage: ${GLANCE_BACKEND}
volume:
- class_name: standard
+ class_name: general
EOF
helm upgrade --install glance ./glance \
--namespace=openstack \
```
* Execute script files.
```
$ ./tools/deployment/developer/ceph/050-mariadb.sh
$ ./tools/deployment/developer/ceph/060-rabbitmq.sh
$ ./tools/deployment/developer/ceph/070-memcached.sh
$ ./tools/deployment/developer/ceph/080-keystone.sh
$ ./tools/deployment/component/glance/glance.sh
```
5. Deploy Barbican and Tacker
* Modify `openstack-helm/tacker/templates/pvc.yaml` file as follows:
```
diff --git a/tacker/templates/pvc.yaml b/tacker/templates/pvc.yaml
index 8b1678b3..c0599b45 100644
--- a/tacker/templates/pvc.yaml
+++ b/tacker/templates/pvc.yaml
@@ -23,7 +23,7 @@ metadata:
name: {{ $name }}
spec:
accessModes:
- - "ReadWriteMany"
+ - "ReadWriteOnce"
resources:
requests:
storage: {{ $size }}
```
* Modify `openstack-helm/tacker/values.yaml` file as follows:
```
diff --git a/tacker/values.yaml b/tacker/values.yaml
index 90702f95..a3f02a62 100644
--- a/tacker/values.yaml
+++ b/tacker/values.yaml
@@ -105,12 +105,16 @@ pod:
security_context:
server:
pod:
- runAsUser: 42424
- runAsNonRoot: true
+ runAsUser: 0
+ runAsNonRoot: false
conductor:
pod:
- runAsUser: 42424
- runAsNonRoot: true
+ runAsUser: 0
+ runAsNonRoot: false
lifecycle:
termination_grace_period:
server:
```
* Execute script files.
```
$ ./tools/deployment/developer/common/085-barbican.sh
$ ./tools/deployment/component/tacker/tacker.sh
```
6. Verify successful deployment
* The helm releases are deployed as follows:
```
sysadmin@controller-0:~$ helm list -n ceph
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
ceph-client ceph 1 2023-10-04 13:28:04.214446853 +0000 UTC deployed ceph-client-0.1.47 v1.0.0
ceph-mon ceph 1 2023-10-04 13:23:25.009836684 +0000 UTC deployed ceph-mon-0.1.30 v1.0.0
ceph-osd ceph 1 2023-10-04 13:26:07.829373478 +0000 UTC deployed ceph-osd-0.1.47 v1.0.0
ceph-provisioners ceph 1 2023-10-04 13:30:04.478204441 +0000 UTC deployed ceph-provisioners-0.1.26 v1.0.0
ingress-ceph ceph 1 2023-10-02 07:59:46.619657229 +0000 UTC deployed ingress-0.2.17 v1.5.1
sysadmin@controller-0:~$ helm list -n openstack
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
barbican openstack 1 2023-10-04 14:11:54.122228604 +0000 UTC deployed barbican-0.3.5 v1.0.0
ceph-openstack-config openstack 1 2023-10-04 13:35:39.737074964 +0000 UTC deployed ceph-provisioners-0.1.26 v1.0.0
glance openstack 1 2023-10-05 01:14:44.18606719 +0000 UTC deployed glance-0.4.13 v1.0.0
ingress-openstack openstack 1 2023-10-02 07:59:30.823441021 +0000 UTC deployed ingress-0.2.17 v1.5.1
keystone openstack 2 2023-10-04 13:58:36.81624535 +0000 UTC deployed keystone-0.3.4 v1.0.0
mariadb openstack 1 2023-10-04 13:36:33.178219784 +0000 UTC deployed mariadb-0.2.33 v10.6.7
memcached openstack 1 2023-10-04 13:44:40.7788406 +0000 UTC deployed memcached-0.1.13 v1.5.5
rabbitmq openstack 1 2023-10-04 13:39:44.683045128 +0000 UTC deployed rabbitmq-0.1.29 v3.9.0
tacker openstack 1 2023-10-05 10:03:19.033603307 +0000 UTC deployed tacker-0.1.1 v1.0.0
```
* The pods are read as follows (Check all pods are "Completed" or "Running" status):
```
sysadmin@controller-0:~/openstack-helm$ kubectl get pod -n ceph
NAME READY STATUS RESTARTS AGE
ceph-bootstrap-zx84v 0/1 Completed 0 45h
ceph-cephfs-client-key-generator-25d8b 0/1 Completed 0 44h
ceph-checkdns-79d7d8bb68-qq8jb 1/1 Running 0 44h
ceph-mds-7b84466549-9cqlg 1/1 Running 0 44h
ceph-mds-keyring-generator-b4ggp 0/1 Completed 0 45h
ceph-mgr-664d8b66cb-rr7bz 1/1 Running 0 45h
ceph-mgr-keyring-generator-sqv4q 0/1 Completed 0 45h
ceph-mon-check-9dc8fd588-c5m4z 1/1 Running 0 45h
ceph-mon-default-37207810-cc466 1/1 Running 0 45h
ceph-mon-keyring-generator-mwxpl 0/1 Completed 0 45h
ceph-osd-default-83945928-zhpsf 2/2 Running 0 45h
ceph-osd-keyring-generator-s4c6p 0/1 Completed 0 45h
ceph-pool-checkpgs-28276455-6lgl5 0/1 Completed 0 12m
ceph-rbd-csi-provisioner-cc45b976c-clv8n 5/5 Running 0 44h
ceph-rbd-csi-provisioner-cc45b976c-wjtfb 5/5 Running 0 44h
ceph-rbd-plugin-4m4kq 2/2 Running 0 44h
ceph-rbd-pool-sqscp 0/1 Completed 0 44h
ceph-storage-keys-generator-mvcpf 0/1 Completed 0 45h
ingress-5955fbfb76-n9td7 1/1 Running 1 (45h ago) 4d2h
ingress-error-pages-6c49c5ff74-7nl2q 1/1 Running 1 (45h ago) 4d2h
sysadmin@controller-0:~/openstack-helm$ kubectl get pod -n openstack
NAME READY STATUS RESTARTS AGE
barbican-api-75fd4d79d7-ncz2c 1/1 Running 0 46h
barbican-db-init-mvhs4 0/1 Completed 0 46h
barbican-db-sync-2hn96 0/1 Completed 0 46h
barbican-ks-endpoints-57rm2 0/3 Completed 0 46h
barbican-ks-service-x2jqn 0/1 Completed 0 46h
barbican-ks-user-ds9h6 0/1 Completed 0 46h
barbican-rabbit-init-gz647 0/1 Completed 0 46h
barbican-test 0/1 Completed 0 46h
ceph-openstack-config-ceph-ns-ceph-config-generator-wqv7c 0/1 Completed 0 46h
ceph-openstack-config-ceph-ns-key-cleaner-lkbpk 0/1 Completed 0 3d9h
ceph-openstack-config-ceph-ns-key-generator-g55dn 0/1 Completed 0 46h
glance-api-97df56ddb-pr598 1/1 Running 0 35h
glance-bootstrap-fbmpq 0/1 Completed 0 35h
glance-db-init-gtmdc 0/1 Completed 0 35h
glance-db-sync-9jkb8 0/1 Completed 0 35h
glance-ks-endpoints-dkb6m 0/3 Completed 0 35h
glance-ks-service-xdhfk 0/1 Completed 0 35h
glance-ks-user-9xhvf 0/1 Completed 0 35h
glance-metadefs-load-rw2kc 0/1 Completed 0 35h
glance-rabbit-init-c4wvr 0/1 Completed 0 35h
glance-storage-init-lzn72 0/1 Completed 0 35h
ingress-5448bbd7d-7rz99 1/1 Running 1 (47h ago) 4d4h
ingress-error-pages-54c8fdfb4d-wgktt 1/1 Running 1 (47h ago) 4d4h
keystone-api-6cb7d765ff-srpwg 1/1 Running 0 46h
keystone-bootstrap-f9s5n 0/1 Completed 0 46h
keystone-credential-setup-27qkx 0/1 Completed 0 46h
keystone-db-init-sr9dj 0/1 Completed 0 46h
keystone-db-sync-7hnj8 0/1 Completed 0 46h
keystone-domain-manage-2n6sf 0/1 Completed 0 46h
keystone-fernet-rotate-28275120-djbg7 0/1 Completed 0 24h
keystone-fernet-rotate-28275840-z2wnq 0/1 Completed 0 12h
keystone-fernet-rotate-28276560-z6rmr 0/1 Completed 0 30m
keystone-fernet-setup-x8px7 0/1 Completed 0 46h
keystone-rabbit-init-w5h9q 0/1 Completed 0 46h
mariadb-ingress-7f9bcfd79b-6flfw 1/1 Running 0 46h
mariadb-ingress-7f9bcfd79b-tlwkc 1/1 Running 0 46h
mariadb-ingress-error-pages-557b55c45f-tw8sw 1/1 Running 0 46h
mariadb-server-0 1/1 Running 0 46h
memcached-memcached-785bbdd4d8-zxh76 1/1 Running 0 46h
rabbitmq-cluster-wait-49khp 0/1 Completed 0 46h
rabbitmq-rabbitmq-0 1/1 Running 0 46h
rabbitmq-rabbitmq-1 1/1 Running 0 46h
tacker-conductor-9f977f5b4-tx58c 1/1 Running 0 26h
tacker-db-init-4d7xz 0/1 Completed 0 26h
tacker-db-sync-vwzg2 0/1 Completed 0 26h
tacker-ks-endpoints-426wd 0/3 Completed 0 26h
tacker-ks-service-lltsv 0/1 Completed 0 26h
tacker-ks-user-5vpws 0/1 Completed 0 26h
tacker-rabbit-init-2jkgb 0/1 Completed 0 26h
tacker-server-76d9bbf6c8-skk8h 1/1 Running 0 26h
```
* Test if Tacker is working properly
```
$ TACKER_SERVER_POD=tacker-server-76d9bbf6c8-skk8h
$ TACKER_ENDPOINT=tacker.openstack.svc.cluster.local
# Issue token from keystone
$ kubectl exec -n openstack -it $TACKER_SERVER_POD \
-- curl -i -X POST -H "Content-Type: application/json" \
-d '{"auth":{"identity":{"methods":["password"],"password":{"user":{"domain":{"name":"default"},"name":"admin","password":"password"}}},"scope":{"project":{"domain":{"name":"default"},"name":"admin"}}}}' \
http://keystone.openstack.svc.cluster.local/v3/auth/tokens
HTTP/1.1 201 CREATED
Date: Fri, 06 Oct 2023 12:46:40 GMT
Content-Type: application/json
Content-Length: 3175
Connection: keep-alive
X-Subject-Token: gAAAAABlIAGv1RqxqMJ7rt_VyAtPTxF0XjMG19zp-0zaZmHdFkKmEjLfUus09GkPUdcbCeVuR8ZfmMjqg9C2kRCWWX4Llfdwld1lKM-beqQ7s127kjhpilf28e1oXh351CmBFy97PaZ9D5WBoe3fRrDkhhB_cEsB76Pyj6P2KQuNeMIhGmb1fKA
Vary: X-Auth-Token
x-openstack-request-id: req-408ef1f6-2b61-4a8d-89b0-0d987878cbbb
# Set `X-Subject-Token` retrieved as TOKEN
$ TOKEN=gAAAAABlIAGv1RqxqMJ7rt_VyAtPTxF0XjMG19zp-0zaZmHdFkKmEjLfUus09GkPUdcbCeVuR8ZfmMjqg9C2kRCWWX4Llfdwld1lKM-beqQ7s127kjhpilf28e1oXh351CmBFy97PaZ9D5WBoe3fRrDkhhB_cEsB76Pyj6P2KQuNeMIhGmb1fKA
$ kubectl exec -n openstack -it $TACKER_SERVER_POD \
-- curl -X GET ${TACKER_ENDPOINT}/vnflcm/v2/vnf_instances \
-H "X-Auth-Token:$TOKEN" -H "Version: 2.0.0"
[] *** Success if you can get an empty list ***