[O2] SSLCertVerificationError when apply O2 application on starlingx with SSL enabled
Description
2024-04-02 07:17:21,591:[o2common.service.watcher.worker]:[worker.py]-[43] [DEBUG]:_repeat started
2024-04-02 07:17:21,591:[o2common.service.watcher.base]:[base.py]-[92] [DEBUG]:probe resources with watcher: ocloud
2024-04-02 07:17:21,634:[o2common.service.watcher.base]:[base.py]-[48] [WARNING]:Failed to probe ocloud watcher due to: <class 'keystoneauth1.exceptions.connection.SSLError'> - SSL exception connecting to https://147.11.89.0:6385/v1/isystems: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/keystoneauth1/session.py", line 1021, in _send_request
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 517, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/src/o2common/service/watcher/base.py", line 40, in probe
cmds = self._probe(
^^^^^^^^^^^^
File "/src/o2ims/service/watcher/ocloud_watcher.py", line 36, in _probe
newmodel = self._client.get(None)
^^^^^^^^^^^^^^^^^^^^^^
File "/src/o2common/service/client/base_client.py", line 28, in get
return self._get(id)
^^^^^^^^^^^^^
File "/src/o2ims/adapter/clients/ocloud_client.py", line 45, in _get
return self.driver.getInstanceInfo()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/src/o2ims/adapter/clients/ocloud_client.py", line 269, in getInstanceInfo
systems = self.stxclient.isystem.list()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/v1/isystem.py", line 30, in list
return self._list(self._path(), "isystems")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/common/base.py", line 71, in _list
_, body = self.api.json_request('GET', url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/common/http.py", line 204, in json_request
resp = self._http_request(url, method, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/common/http.py", line 181, in _http_request
resp = self.session.request(url, method,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/keystoneauth1/session.py", line 930, in request
resp = send(**kwargs)
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/keystoneauth1/session.py", line 1025, in _send_request
raise exceptions.SSLError(msg)
keystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://147.11.89.0:6385/v1/isystems: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
Medium
2024-04-02 07:17:21,591:[o2common.service.watcher.worker]:[worker.py]-[43] [DEBUG]:_repeat started
2024-04-02 07:17:21,591:[o2common.service.watcher.base]:[base.py]-[92] [DEBUG]:probe resources with watcher: ocloud
2024-04-02 07:17:21,634:[o2common.service.watcher.base]:[base.py]-[48] [WARNING]:Failed to probe ocloud watcher due to: <class 'keystoneauth1.exceptions.connection.SSLError'> - SSL exception connecting to https://147.11.89.0:6385/v1/isystems: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
2024-04-02 07:17:21,636:[o2common.service.watcher.base]:[base.py]-[50] [DEBUG]:Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 653, in connect
sock_and_verified = _ssl_wrap_socket_and_match_hostname(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 806, in _ssl_wrap_socket_and_match_hostname
ssl_sock = ssl_wrap_socket(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 465, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 509, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/ssl.py", line 1075, in _create
self.do_handshake()
File "/usr/local/lib/python3.11/ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
raise new_e
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/keystoneauth1/session.py", line 1021, in _send_request
resp = self.session.request(method, url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 517, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/src/o2common/service/watcher/base.py", line 40, in probe
cmds = self._probe(
^^^^^^^^^^^^
File "/src/o2ims/service/watcher/ocloud_watcher.py", line 36, in _probe
newmodel = self._client.get(None)
^^^^^^^^^^^^^^^^^^^^^^
File "/src/o2common/service/client/base_client.py", line 28, in get
return self._get(id)
^^^^^^^^^^^^^
File "/src/o2ims/adapter/clients/ocloud_client.py", line 45, in _get
return self.driver.getInstanceInfo()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/src/o2ims/adapter/clients/ocloud_client.py", line 269, in getInstanceInfo
systems = self.stxclient.isystem.list()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/v1/isystem.py", line 30, in list
return self._list(self._path(), "isystems")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/common/base.py", line 71, in _list
_, body = self.api.json_request('GET', url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/common/http.py", line 204, in json_request
resp = self._http_request(url, method, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/src/cgtsclient/sysinv/cgts-client/cgts-client/cgtsclient/common/http.py", line 181, in _http_request
resp = self.session.request(url, method,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/keystoneauth1/session.py", line 930, in request
resp = send(**kwargs)
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/keystoneauth1/session.py", line 1025, in _send_request
raise exceptions.SSLError(msg)
keystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://147.11.89.0:6385/v1/isystems: HTTPSConnectionPool(host='147.11.89.0', port=6385): Max retries exceeded with url: /v1/isystems (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)')))
2024-04-02 07:17:21,636:[o2common.service.watcher.base]:[base.py]-[96] [DEBUG]:probe returns 0 resources