Symptoms


The following error is shown during hotfix installation on action related to packages upgrade/installation:


2020-06-12 06:41:21.680 [WARNING] Retrying action in "Installing/upgrading packages on billing nodes" ...     
2020-06-12 06:41:21.681 [DEBUG] installing package (3561) to host 7            
2020-06-12 06:41:21.689 [DEBUG] SQL: set standard_conforming_strings = off (())
2020-06-12 06:41:21.693 [DEBUG] SQL: SELECT 1 (())      
2020-06-12 06:41:21.694 [DEBUG] Connection <connection object at 0x7f07385cf050; dsn: 'dbname=oss user=oss password=xxxxxxxxxxxxxx host=192.168.0.5 port=5432 options='-c standard_conforming_strin
gs=off'', closed: 1> was closed                 
2020-06-12 06:41:21.694 [DEBUG] Connection <connection object at 0x7f07385cf050; dsn: 'dbname=oss user=oss password=xxxxxxxxxxxxxx host=192.168.0.5 port=5432 options='-c standard_conforming_strin
gs=off'', closed: 1> was removed from pool      
2020-06-12 06:41:21.694 [DEBUG] call method pem.packaging.installPackageSync({'host_id': 7, 'package_id': 3561, 'properties': []})    
2020-06-12 07:41:23.155 [DEBUG] return {'status': -1, 'error_message': "Internal error: HTTP POST to 'https://192.168.0.10:8352/process' failed: timeout expired..", 'extype_id': 1, 'module_id': 'Common', 'error_code': 0, 'properties': {'host_id': 7, 'package_id': 3561, 'properties': []}}                  
2020-06-12 07:41:23.155 [DEBUG] Error trace:    
2020-06-12 07:41:23.159 [DEBUG] Traceback (most recent call last):             
  File "/usr/local/pem/var/updates/oa-8.3-990_KB201021-1731/poaupdater/uAction.py", line 127, in fn           
    return fun(*args, **kwds)                   
  File "/usr/local/pem/var/updates/oa-8.3-990_KB201021-1731/poaupdater/uPackaging.py", line 166, in installPackageToHostAPI           
    return api.pem.packaging.installPackageSync(host_id=host_id, package_id=pkg_id, properties=proplist)["component_id"]              
  File "/usr/local/pem/var/updates/oa-8.3-990_KB201021-1731/poaupdater/openapi.py", line 239, in __call__     
    raise OpenAPIError(resp)                    
OpenAPIError: Internal error: HTTP POST to 'https://192.168.0.10:8352/process' failed: timeout expired..




Cause


  • pa-agent is not running on target host
  • Communication between target node and Management node is not possible on port 8352
  • Management node is not able to update packages through yum utility on target node



Resolution


  • Check current state of pa-agent service on the target node:
service pa-agent status

Start service if it is not running:

service pa-agent start

In case of errors, check log file /var/log/pa/core.log in order to obtain additional diagnostic information about the error.


  • Verify that Management node is able to communicate with target host on port 8352:


telnet 192.168.0.10 8352


  • Check if yum utility is working properly and required repositories are available:
yum repolist
yum list updates

In case if there are some issues, resolve them based on the error description.