Wei Zhou 87284f03f0
Upgrade to JRE17 and Upgrade System VMs/VRs to Python3 and Debian 12 (#8497)
* Update to 4.20.0

* Update to python3

* Upgrade to JRE 17

* Upgrade to Debian 12.4.0

* VR: upgrade to python3

for f in `find systemvm/ -name *.py`;do
    if grep "print " $f >/dev/null;then
        2to3-2.7 -w $f
    else
        2to3-2.7 -p -w $f
    fi
done

* java: Use JRE17 in cloudstack packages and systemvmtemplate

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Add --add-opens to JAVA_OPTS in systemd config

* Add --add-opens to JAVA_OPTS in systemd config for usage

* python3: fix "TypeError: a bytes-like object is required, not 'str'"

* python3: fix "ValueError: must have exactly one of create/read/write/append mode"

* Add --add-exports=java.base/sun.security.x509=ALL-UNNAMED for management server

* Use pip3 instead of pip for centos8

* python3: fix "TypeError: write() argument must be str, not bytes"

```
root@r-1037-VM:~# /opt/cloud/bin/passwd_server_ip.py 10.1.1.1
Traceback (most recent call last):
  File "/opt/cloud/bin/passwd_server_ip.py", line 201, in <module>
    serve()
  File "/opt/cloud/bin/passwd_server_ip.py", line 187, in serve
    initToken()
  File "/opt/cloud/bin/passwd_server_ip.py", line 60, in initToken
    f.write(secureToken)
TypeError: write() argument must be str, not bytes
root@r-1037-VM:~#
```

* Python3: fix "name 'file' is not defined"

```
root@r-1037-VM:~# /opt/cloud/bin/passwd_server_ip.py 10.1.1.1
Traceback (most recent call last):
  File "/opt/cloud/bin/passwd_server_ip.py", line 201, in <module>
    serve()
  File "/opt/cloud/bin/passwd_server_ip.py", line 188, in serve
    loadPasswordFile()
  File "/opt/cloud/bin/passwd_server_ip.py", line 67, in loadPasswordFile
    with file(getPasswordFile()) as f:
NameError: name 'file' is not defined
```

* python3: fix "TypeError: write() argument must be str, not bytes" (two more files)

* Upgrade jaxb version

* python3: fix more "TypeError: a bytes-like object is required, not str"

* python3: fix "Failed to update password server"

Failed to update password server due to: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

* python3: fix "bad duration value: ikelifetime=24.0h"

Jan 15 13:57:20 systemvm ipsec[3080]: # bad duration value: ikelifetime=24.0h

* python3: fix password server "invalid save_password token"

* test: incease retries in test_vpc_vpn.py

* python3: fix passwd_server_ip.py

see error below
```
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: ----------------------------------------
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: Exception occurred during processing of request from ('10.1.1.129', 32782)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: Traceback (most recent call last):
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 650, in process_request_thread
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.finish_request(request, client_address)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.RequestHandlerClass(request, client_address, self)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 720, in __init__
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.handle()
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/http/server.py", line 427, in handle
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.handle_one_request()
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/http/server.py", line 415, in handle_one_request
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     method()
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/opt/cloud/bin/passwd_server_ip.py", line 120, in do_GET
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.wfile.write(password)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 799, in write
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self._sock.sendall(b)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: TypeError: a bytes-like object is required, not 'str'
```

* python3: fix self.cl.get_router_password in Redundant VRs

```
File "/opt/cloud/bin/cs/CsDatabag.py", line 154, in get_router_password
    md5.update(passwd)
TypeError: Unicode-objects must be encoded before hashing"]
```

* scripts: mark multipath scripts as executable

* systemvm template: remove hyperv packages and do not export

* VR: update default RAM size of System VMs/VRs to 512MiB

Before
```
mysql> select id,name,cpu,speed,ram_size,unique_name,system_use from service_offering where name like "System%";
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
| id | name                                                     | cpu  | speed | ram_size | unique_name                      | system_use |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
|  3 | System Offering For Software Router                      |    1 |   500 |      256 | Cloud.Com-SoftwareRouter         |          1 |
|  4 | System Offering For Software Router - Local Storage      |    1 |   500 |      256 | Cloud.Com-SoftwareRouter-Local   |          1 |
|  5 | System Offering For Internal LB VM                       |    1 |   256 |      256 | Cloud.Com-InternalLBVm           |          1 |
|  6 | System Offering For Internal LB VM - Local Storage       |    1 |   256 |      256 | Cloud.Com-InternalLBVm-Local     |          1 |
|  7 | System Offering For Console Proxy                        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy           |          1 |
|  8 | System Offering For Console Proxy - Local Storage        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy-Local     |          1 |
|  9 | System Offering For Secondary Storage VM                 |    1 |   500 |      512 | Cloud.com-SecondaryStorage       |          1 |
| 10 | System Offering For Secondary Storage VM - Local Storage |    1 |   500 |      512 | Cloud.com-SecondaryStorage-Local |          1 |
| 11 | System Offering For Elastic LB VM                        |    1 |   128 |      128 | Cloud.Com-ElasticLBVm            |          1 |
| 12 | System Offering For Elastic LB VM - Local Storage        |    1 |   128 |      128 | Cloud.Com-ElasticLBVm-Local      |          1 |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
10 rows in set (0.00 sec)
```

New value
```
mysql> select id,name,cpu,speed,ram_size,unique_name,system_use from service_offering where name like "System%";
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
| id | name                                                     | cpu  | speed | ram_size | unique_name                      | system_use |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
|  3 | System Offering For Software Router                      |    1 |   500 |      512 | Cloud.Com-SoftwareRouter         |          1 |
|  4 | System Offering For Software Router - Local Storage      |    1 |   500 |      512 | Cloud.Com-SoftwareRouter-Local   |          1 |
|  5 | System Offering For Internal LB VM                       |    1 |   256 |      512 | Cloud.Com-InternalLBVm           |          1 |
|  6 | System Offering For Internal LB VM - Local Storage       |    1 |   256 |      512 | Cloud.Com-InternalLBVm-Local     |          1 |
|  7 | System Offering For Console Proxy                        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy           |          1 |
|  8 | System Offering For Console Proxy - Local Storage        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy-Local     |          1 |
|  9 | System Offering For Secondary Storage VM                 |    1 |   500 |      512 | Cloud.com-SecondaryStorage       |          1 |
| 10 | System Offering For Secondary Storage VM - Local Storage |    1 |   500 |      512 | Cloud.com-SecondaryStorage-Local |          1 |
| 11 | System Offering For Elastic LB VM                        |    1 |   128 |      512 | Cloud.Com-ElasticLBVm            |          1 |
| 12 | System Offering For Elastic LB VM - Local Storage        |    1 |   128 |      512 | Cloud.Com-ElasticLBVm-Local      |          1 |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
10 rows in set (0.01 sec)
```

* debian12: fix test_network_ipv6 and test_vpc_ipv6

* python3: remove duplicated imports

* debian12: failed to start Apache2 server (SSLCipherSuite @SECLEVEL=0)

error message
```
[Sat Jan 20 22:51:14.595143 2024] [ssl:emerg] [pid 10200:tid 140417063888768] AH02562: Failed to configure certificate cloudinternal.com:443:0 (with chain), check /etc/ssl/certs/cert_apache.crt
[Sat Jan 20 22:51:14.595234 2024] [ssl:emerg] [pid 10200:tid 140417063888768] SSL Library Error: error:0A00018E:SSL routines::ca md too weak
AH00016: Configuration Failed
```

openssl version
```
root@s-167-VM:~# openssl version -a
OpenSSL 3.0.11 19 Sep 2023 (Library: OpenSSL 3.0.11 19 Sep 2023)
built on: Mon Oct 23 17:52:22 2023 UTC
platform: debian-amd64
options:  bn(64,64)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/reproducible-path/openssl-3.0.11=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-3"
MODULESDIR: "/usr/lib/x86_64-linux-gnu/ossl-modules"
Seeding source: os-specific
CPUINFO: OPENSSL_ia32cap=0x80202001478bfffd:0x0
```

certificate
```
root@s-167-VM:~# keytool -printcert -rfc -file /usr/local/cloud/systemvm/certs/realhostip.crt
-----BEGIN CERTIFICATE-----
MIIFZTCCBE2gAwIBAgIHKBCduBUoKDANBgkqhkiG9w0BAQUFADCByjELMAkGA1UE
BhMCVVMxEDAOBgNVBAgTB0FyaXpvbmExEzARBgNVBAcTClNjb3R0c2RhbGUxGjAY
BgNVBAoTEUdvRGFkZHkuY29tLCBJbmMuMTMwMQYDVQQLEypodHRwOi8vY2VydGlm
aWNhdGVzLmdvZGFkZHkuY29tL3JlcG9zaXRvcnkxMDAuBgNVBAMTJ0dvIERhZGR5
IFNlY3VyZSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTERMA8GA1UEBRMIMDc5Njky
ODcwHhcNMTIwMjAzMDMzMDQwWhcNMTcwMjA3MDUxMTIzWjBZMRkwFwYDVQQKDBAq
LnJlYWxob3N0aXAuY29tMSEwHwYDVQQLDBhEb21haW4gQ29udHJvbCBWYWxpZGF0
ZWQxGTAXBgNVBAMMECoucmVhbGhvc3RpcC5jb20wggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQCDT9AtEfs+s/I8QXp6rrCw0iNJ0+GgsybNHheU+JpL39LM
TZykCrZhZnyDvwdxCoOfE38Sa32baHKNds+y2SHnMNsOkw8OcNucHEBX1FIpOBGp
h9D6xC+umx9od6xMWETUv7j6h2u+WC3OhBM8fHCBqIiAol31/IkcqDxxsHlQ8S/o
CfTlXJUY6Yn628OA1XijKdRnadV0hZ829cv/PZKljjwQUTyrd0KHQeksBH+YAYSo
2JUl8ekNLsOi8/cPtfojnltzRI1GXi0ZONs8VnDzJ0a2gqZY+uxlz+CGbLnGnlN4
j9cBpE+MfUE+35Dq121sTpsSgF85Mz+pVhn2S633AgMBAAGjggG+MIIBujAPBgNV
HRMBAf8EBTADAQEAMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAOBgNV
HQ8BAf8EBAMCBaAwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL2NybC5nb2RhZGR5
LmNvbS9nZHMxLTY0LmNybDBTBgNVHSAETDBKMEgGC2CGSAGG/W0BBxcBMDkwNwYI
KwYBBQUHAgEWK2h0dHA6Ly9jZXJ0aWZpY2F0ZXMuZ29kYWRkeS5jb20vcmVwb3Np
dG9yeS8wgYAGCCsGAQUFBwEBBHQwcjAkBggrBgEFBQcwAYYYaHR0cDovL29jc3Au
Z29kYWRkeS5jb20vMEoGCCsGAQUFBzAChj5odHRwOi8vY2VydGlmaWNhdGVzLmdv
ZGFkZHkuY29tL3JlcG9zaXRvcnkvZ2RfaW50ZXJtZWRpYXRlLmNydDAfBgNVHSME
GDAWgBT9rGEyk2xF1uLuhV+auud2mWjM5zArBgNVHREEJDAighAqLnJlYWxob3N0
aXAuY29tgg5yZWFsaG9zdGlwLmNvbTAdBgNVHQ4EFgQUZyJz9/QLy5TWIIscTXID
E8Xk47YwDQYJKoZIhvcNAQEFBQADggEBAKiUV3KK16mP0NpS92fmQkCLqm+qUWyN
BfBVgf9/M5pcT8EiTZlS5nAtzAE/eRpBeR3ubLlaAogj4rdH7YYVJcDDLLoB2qM3
qeCHu8LFoblkb93UuFDWqRaVPmMlJRnhsRkL1oa2gM2hwQTkBDkP7w5FG1BELCgl
gZI2ij2yxjge6pOEwSyZCzzbCcg9pN+dNrYyGEtB4k+BBnPA3N4r14CWbk+uxjrQ
6j2Ip+b7wOc5IuMEMl8xwTyjuX3lsLbAZyFI9RCyofwA9NqIZ1GeB6Zd196rubQp
93cmBqGGjZUs3wMrGlm7xdjlX6GQ9UvmvkMub9+lL99A5W50QgCmFeI=
-----END CERTIFICATE-----

Warning:
The certificate uses the SHA1withRSA signature algorithm which is considered a security risk. This algorithm will be disabled in a future update.
```

it comes from
```
$ openssl x509 -in ./systemvm/agent/certs/realhostip.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 11277268652730408 (0x28109db8152828)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certificates.godaddy.com/repository, CN = Go Daddy Secure Certification Authority, serialNumber = 07969287
        Validity
            Not Before: Feb  3 03:30:40 2012 GMT
            Not After : Feb  7 05:11:23 2017 GMT
        Subject: O = *.realhostip.com, OU = Domain Control Validated, CN = *.realhostip.com
```

* debian12: use ed25519 instead of rsa as ssh-rsa has been deprecated in OpenSSH

on xenserver
```
[root@pr8497-t8906-xenserver-71-xs2 ~]# ssh -i .ssh/id_rsa.cloud -p 3922 169.254.214.153
Warning: Permanently added '[169.254.214.153]:3922' (ECDSA) to the list of known hosts.
Permission denied (publickey).
```
in the CPVM
Jan 22 19:31:09 v-1-VM sshd[2869]: userauth_pubkey: signature algorithm ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]
Jan 22 19:31:09 v-1-VM sshd[2869]: Connection closed by authenticating user root 169.254.0.1 port 54704 [preauth]
```

ssh-dss (DSA) is not supported either

* debian12: add PubkeyAcceptedAlgorithms=+ssh-rsa to sshd_config

* VR: install python3 packages in case of Debian 11

* pom.xml: exclude systemvm/agent/packages/* in license check

* systemvm: do not patch router/systemvm during startup

this will cause 4.19 SYSTEM template not work, but may be expected
- python3 VS python2 (default)
- openSSL 3.0.1 VS 1.1.1w
- openssh-server 9.1 VS 8.4

* VR: patch router/systemvm if template is debian11

This supports debian 11 template by
- revert change in systemvm/debian/etc/ssh/sshd_config
- patch VR/systemvms during startup
- install packages during patching system vm/routers

* python3 flake: fix E502 the backslash is redundant between brackets

```
../debian/root/health_checks/router_version_check.py:55:70: E502 the backslash is redundant between brackets
../debian/root/health_checks/router_version_check.py:58:61: E502 the backslash is redundant between brackets
../debian/root/health_checks/router_version_check.py:67:71: E502 the backslash is redundant between brackets
../debian/root/health_checks/router_version_check.py:70:60: E502 the backslash is redundant between brackets
../debian/root/health_checks/haproxy_check.py:47:71: E502 the backslash is redundant between brackets
../debian/root/health_checks/haproxy_check.py:48:64: E502 the backslash is redundant between brackets
../debian/root/health_checks/cpu_usage_check.py:43:54: E502 the backslash is redundant between brackets
../debian/root/health_checks/cpu_usage_check.py:46:58: E502 the backslash is redundant between brackets
../debian/root/health_checks/memory_usage_check.py:31:65: E502 the backslash is redundant between brackets
../debian/root/health_checks/memory_usage_check.py:42:57: E502 the backslash is redundant between brackets
../debian/root/health_checks/memory_usage_check.py:45:63: E502 the backslash is redundant between brackets
```

* python3 flake: fix E275 missing whitespace after keyword

```
../debian/opt/cloud/bin/cs_firewallrules.py:29:20: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_dhcp.py:27:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_dhcp.py:36:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_guestnetwork.py:33:20: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_guestnetwork.py:35:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_vpnusers.py:37:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/merge.py:230:11: E275 missing whitespace after keyword
../debian/opt/cloud/bin/merge.py:239:19: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_remoteaccessvpn.py:24:12: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_site2sitevpn.py:24:12: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs/CsHelper.py:90:15: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs/CsAddress.py:367:15: E275 missing whitespace after keyword
```

* python3 flake: fix configure.py

```
../debian/opt/cloud/bin/configure.py:24:22: E401 multiple imports on one line
../debian/opt/cloud/bin/configure.py:43:180: E501 line too long (294 > 179 characters)
../debian/opt/cloud/bin/configure.py:46:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/configure.py:63:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/configure.py:65:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/configure.py:72:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/configure.py:310:25: E711 comparison to None should be 'if cond is not None:'
../debian/opt/cloud/bin/configure.py:312:29: E711 comparison to None should be 'if cond is None:'
../debian/opt/cloud/bin/configure.py:378:25: E711 comparison to None should be 'if cond is not None:'
../debian/opt/cloud/bin/configure.py:380:29: E711 comparison to None should be 'if cond is None:'
../debian/opt/cloud/bin/configure.py:490:29: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
../debian/opt/cloud/bin/configure.py:642:16: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/configure.py:644:18: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/configure.py:1416:1: E305 expected 2 blank lines after class or function definition, found 1
```

* python3 flake: fix other python files

```
../debian/opt/cloud/bin/vmdata.py:97:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/vmdata.py:99:14: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`

../debian/opt/cloud/bin/cs/CsRedundant.py:438:53: E203 whitespace before ':'
../debian/opt/cloud/bin/cs/CsRedundant.py:461:53: E203 whitespace before ':'
../debian/opt/cloud/bin/cs/CsRedundant.py:499:5: E303 too many blank lines (2)

../debian/opt/cloud/bin/cs/CsDatabag.py:189:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/cs/CsDatabag.py:193:37: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`

../debian/opt/cloud/bin/cs/CsHelper.py:118:30: E231 missing whitespace after ','
../debian/opt/cloud/bin/cs/CsHelper.py:119:15: E225 missing whitespace around operator
../debian/opt/cloud/bin/cs/CsHelper.py:127:19: E225 missing whitespace around operator

../debian/opt/cloud/bin/cs/CsAddress.py:324:43: E221 multiple spaces before operator

../debian/opt/cloud/bin/cs/CsVpcGuestNetwork.py:28:1: E302 expected 2 blank lines, found 1
```

* python3 flake: fix CsNetfilter.py

```
../debian/opt/cloud/bin/cs/CsNetfilter.py:226:13: E117 over-indented
../debian/opt/cloud/bin/cs/CsNetfilter.py:233:180: E501 line too long (197 > 179 characters)
../debian/opt/cloud/bin/cs/CsNetfilter.py:241:14: E201 whitespace after '{'
../debian/opt/cloud/bin/cs/CsNetfilter.py:242:14: E201 whitespace after '{'
../debian/opt/cloud/bin/cs/CsNetfilter.py:247:18: E201 whitespace after '{'
../debian/opt/cloud/bin/cs/CsNetfilter.py:247:74: E202 whitespace before '}'
../debian/opt/cloud/bin/cs/CsNetfilter.py:248:18: E201 whitespace after '{'
```

* systemvm/test: fix sys.path

```
$ bash runtests.sh
/usr/bin/python
Python 3.10.12
Running pycodestyle to check systemvm/python code for errors
Running pylint to check systemvm/python code for errors
Python 3.10.12
pylint 2.12.2
astroid 2.9.3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Running systemvm/python unit tests
....Device "eth0" does not exist.
.....................
----------------------------------------------------------------------
Ran 25 tests in 0.008s

OK
```

* Revert "systemvm template: remove hyperv packages and do not export"

This reverts commit 4383d59d031bde6eae7ebba261ff641ca0a66cd5.

* debian12: move SQL change to schema-41900to42000.sql

* debian12: update systemvm template version to 4.20 in pom.xml

* pom.xml: fix NPE if templates do not exist on download.cloudstack.org

* debian12: increase default system offering for routers to 384MiB RAM

* CKS: fix addkubernetessupportedversion failed with JRE17

```
marvin.cloudstackException.CloudstackAPIException: Execute cmd: addkubernetessupportedversion failed, due to: errorCode: 530, errorText:Cannot invoke "org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine$State.toString()" because the return value of "com.cloud.api.query.vo.TemplateJoinVO.getState()" is null
```

* python3: revert changes by 2to3 with systemvm/debian/root/health_checks/*.py

* debian12: use ISO/packages on download.cloudstack.org

* VR: Update default ram size to 384

* debian12: fix router_version_check.py after VR live-patch and add health check in test_routers.py

* debian12: fix build error after log4j 2.x merge

* VR: Update default ram size to 512MB (again)

This reverts commit 578dd2b73f380e8231ae1eb59827230757cac5e8 and efafa8c4d63775653a2cd406fca10784fbcec3e3.

* systemvmtemplate: Upgrade to Debian 12.5.0

* systemvm template: increase swap to 512MB

* VR: fix health check error due to deprecated SafeConfigParser

warning below
```
root@r-20-VM:~# /opt/cloud/bin/getRouterMonitorResults.sh true
/root/monitorServices.py:59: DeprecationWarning: The SafeConfigParser class has been renamed to ConfigParser in Python 3.2. This alias will be removed in Python 3.12. Use ConfigParser directly instead.
  parser = SafeConfigParser()
```

* test: fix wget does not work in macchinina vms on vmware80u1

fixes error below
```
{Cmd: wget -t 1 -T 1 www.google.com via Host: 10.0.55.186} {returns: ["wget: '/usr/lib/libpcre.so.1' is not an ELF file", "wget: can't load library 'libpcre.so.1'"]}
```

* packaging: add message for VR memory upgrade after packages installation

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-02-26 18:07:50 +05:30

699 lines
34 KiB
RPMSpec

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
%define __os_install_post %{nil}
%global debug_package %{nil}
# DISABLE the post-percentinstall java repacking and line number stripping
# we need to find a way to just disable the java repacking and line number stripping, but not the autodeps
Name: cloudstack
Summary: CloudStack IaaS Platform
#http://fedoraproject.org/wiki/PackageNamingGuidelines#Pre-Release_packages
%define _maventag %{_fullver}
Release: %{_rel}
Version: %{_ver}
License: ASL 2.0
Vendor: Apache CloudStack <dev@cloudstack.apache.org>
Packager: Apache CloudStack <dev@cloudstack.apache.org>
Group: System Environment/Libraries
# FIXME do groups for every single one of the subpackages
Source0: %{name}-%{_maventag}.tgz
BuildRoot: %{_tmppath}/%{name}-%{_maventag}-%{release}-build
BuildRequires: java-11-openjdk-devel
#BuildRequires: ws-commons-util
BuildRequires: jpackage-utils
BuildRequires: gcc
BuildRequires: glibc-devel
BuildRequires: /usr/bin/mkisofs
BuildRequires: python3-setuptools
BuildRequires: wget
BuildRequires: nodejs
%description
CloudStack is a highly-scalable elastic, open source,
intelligent IaaS cloud implementation.
%package management
Summary: CloudStack management server UI
Requires: java-17-openjdk
Requires: (tzdata-java or timezone-java)
Requires: python3
Requires: bash
Requires: gawk
Requires: which
Requires: file
Requires: tar
Requires: bzip2
Requires: gzip
Requires: unzip
Requires: /sbin/mount.nfs
Requires: (openssh-clients or openssh)
Requires: (nfs-utils or nfs-client)
Requires: iproute
Requires: wget
Requires: mysql
Requires: sudo
Requires: /sbin/service
Requires: /sbin/chkconfig
Requires: /usr/bin/ssh-keygen
Requires: (genisoimage or mkisofs)
Requires: ipmitool
Requires: %{name}-common = %{_ver}
Requires: (iptables-services or iptables)
Requires: rng-tools
Requires: (qemu-img or qemu-tools)
Requires: python3-pip
Requires: python3-setuptools
Requires: (libgcrypt > 1.8.3 or libgcrypt20)
Group: System Environment/Libraries
%description management
The CloudStack management server is the central point of coordination,
management, and intelligence in CloudStack.
%package common
Summary: Apache CloudStack common files and scripts
Requires: python3
Group: System Environment/Libraries
%description common
The Apache CloudStack files shared between agent and management server
%global __requires_exclude ^(libuuid\\.so\\.1|/usr/bin/python)$
%package agent
Summary: CloudStack Agent for KVM hypervisors
Requires: (openssh-clients or openssh)
Requires: java-17-openjdk
Requires: tzdata-java
Requires: %{name}-common = %{_ver}
Requires: libvirt
Requires: ebtables
Requires: iptables
Requires: ethtool
Requires: (net-tools or net-tools-deprecated)
Requires: iproute
Requires: ipset
Requires: perl
Requires: (python3-libvirt or python3-libvirt-python)
Requires: (qemu-img or qemu-tools)
Requires: qemu-kvm
Requires: cryptsetup
Requires: rng-tools
Requires: (libgcrypt > 1.8.3 or libgcrypt20)
Requires: (selinux-tools if qemu-tools)
Provides: cloud-agent
Group: System Environment/Libraries
%description agent
The CloudStack agent for KVM hypervisors
%package baremetal-agent
Summary: CloudStack baremetal agent
Requires: tftp-server
Requires: xinetd
Requires: syslinux
Requires: chkconfig
Requires: dhcp
Requires: httpd
Group: System Environment/Libraries
%description baremetal-agent
The CloudStack baremetal agent
%package usage
Summary: CloudStack Usage calculation server
Requires: java-17-openjdk
Requires: tzdata-java
Group: System Environment/Libraries
%description usage
The CloudStack usage calculation service
%package ui
Summary: CloudStack UI
Group: System Environment/Libraries
%description ui
The CloudStack UI
%package marvin
Summary: Apache CloudStack Marvin library
Requires: python3-pip
Requires: gcc
Requires: python3-devel
Requires: libffi-devel
Requires: openssl-devel
Group: System Environment/Libraries
%description marvin
Apache CloudStack Marvin library
%package integration-tests
Summary: Apache CloudStack Marvin integration tests
Requires: %{name}-marvin = %{_ver}
Group: System Environment/Libraries
%description integration-tests
Apache CloudStack Marvin integration tests
%if "%{_ossnoss}" == "noredist"
%package mysql-ha
Summary: Apache CloudStack Balancing Strategy for MySQL
Group: System Environmnet/Libraries
%description mysql-ha
Apache CloudStack Balancing Strategy for MySQL
%endif
%prep
echo Doing CloudStack build
%setup -q -n %{name}-%{_maventag}
%build
cp packaging/centos8/replace.properties build/replace.properties
echo VERSION=%{_maventag} >> build/replace.properties
echo PACKAGE=%{name} >> build/replace.properties
touch build/gitrev.txt
echo $(git rev-parse HEAD) > build/gitrev.txt
if [ "%{_ossnoss}" == "NOREDIST" -o "%{_ossnoss}" == "noredist" ] ; then
echo "Adding noredist flag to the maven build"
FLAGS="$FLAGS -Dnoredist"
fi
if [ "%{_sim}" == "SIMULATOR" -o "%{_sim}" == "simulator" ] ; then
echo "Adding simulator flag to the maven build"
FLAGS="$FLAGS -Dsimulator"
fi
if [ \"%{_temp}\" != "" ]; then
echo "Adding flags to package requested templates"
FLAGS="$FLAGS `rpm --eval %{?_temp}`"
fi
mvn -Psystemvm,developer $FLAGS clean package
cd ui && npm install && npm run build && cd ..
%install
[ ${RPM_BUILD_ROOT} != "/" ] && rm -rf ${RPM_BUILD_ROOT}
# Common directories
mkdir -p ${RPM_BUILD_ROOT}%{_bindir}
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/agent
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/ipallocator
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/cache/%{name}/management/work
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/cache/%{name}/management/temp
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/%{name}/mnt
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/%{name}/management
mkdir -p ${RPM_BUILD_ROOT}%{_initrddir}
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/default
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/profile.d
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/sudoers.d
# Common
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/scripts
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/vms
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/python-site
mkdir -p ${RPM_BUILD_ROOT}/usr/bin
cp -r scripts/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/scripts
install -D systemvm/dist/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/vms/
install python/lib/cloud_utils.py ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/python-site/cloud_utils.py
cp -r python/lib/cloudutils ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/python-site/
python3 -m py_compile ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/python-site/cloud_utils.py
python3 -m compileall ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/python-site/cloudutils
cp build/gitrev.txt ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/scripts
cp packaging/centos8/cloudstack-sccs ${RPM_BUILD_ROOT}/usr/bin
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/scripts/network/cisco
cp -r plugins/network-elements/cisco-vnmc/src/main/scripts/network/cisco/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/scripts/network/cisco
# Management
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/lib
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/management
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/management
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/run
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup/wheel
# Setup Jetty
ln -sf /etc/%{name}/management ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/conf
ln -sf /var/log/%{name}/management ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/logs
install -D client/target/utilities/bin/cloud-migrate-databases ${RPM_BUILD_ROOT}%{_bindir}/%{name}-migrate-databases
install -D client/target/utilities/bin/cloud-set-guest-password ${RPM_BUILD_ROOT}%{_bindir}/%{name}-set-guest-password
install -D client/target/utilities/bin/cloud-set-guest-sshkey ${RPM_BUILD_ROOT}%{_bindir}/%{name}-set-guest-sshkey
install -D client/target/utilities/bin/cloud-setup-databases ${RPM_BUILD_ROOT}%{_bindir}/%{name}-setup-databases
install -D client/target/utilities/bin/cloud-setup-encryption ${RPM_BUILD_ROOT}%{_bindir}/%{name}-setup-encryption
install -D client/target/utilities/bin/cloud-setup-management ${RPM_BUILD_ROOT}%{_bindir}/%{name}-setup-management
install -D client/target/utilities/bin/cloud-setup-baremetal ${RPM_BUILD_ROOT}%{_bindir}/%{name}-setup-baremetal
install -D client/target/utilities/bin/cloud-sysvmadm ${RPM_BUILD_ROOT}%{_bindir}/%{name}-sysvmadm
install -D client/target/utilities/bin/cloud-update-xenserver-licenses ${RPM_BUILD_ROOT}%{_bindir}/%{name}-update-xenserver-licenses
# Bundle cmk in cloudstack-management
wget https://github.com/apache/cloudstack-cloudmonkey/releases/download/6.3.0/cmk.linux.x86-64 -O ${RPM_BUILD_ROOT}%{_bindir}/cmk
chmod +x ${RPM_BUILD_ROOT}%{_bindir}/cmk
cp -r client/target/utilities/scripts/db/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup
cp -r client/target/cloud-client-ui-%{_maventag}.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/
cp -r client/target/classes/META-INF/webapp ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/webapp
cp ui/dist/config.json ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/management/
cp -r ui/dist/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/webapp/
rm -f ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/webapp/config.json
ln -sf /etc/%{name}/management/config.json ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/webapp/config.json
mv ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/cloud-client-ui-%{_maventag}.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/lib/cloudstack-%{_maventag}.jar
cp client/target/lib/*jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/lib/
# Don't package the scripts in the management webapp
rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/webapps/client/WEB-INF/classes/scripts
rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/webapps/client/WEB-INF/classes/vms
for name in db.properties server.properties log4j-cloud.xml environment.properties java.security.ciphers
do
cp client/target/conf/$name ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/management/$name
done
ln -sf log4j-cloud.xml ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/management/log4j2.xml
install python/bindir/cloud-external-ipallocator.py ${RPM_BUILD_ROOT}%{_bindir}/%{name}-external-ipallocator.py
install -D client/target/pythonlibs/jasypt-1.9.3.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/lib/jasypt-1.9.3.jar
install -D utils/target/cloud-utils-%{_maventag}-bundled.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-common/lib/%{name}-utils.jar
install -D packaging/centos8/cloud-ipallocator.rc ${RPM_BUILD_ROOT}%{_initrddir}/%{name}-ipallocator
install -D packaging/centos8/cloud.limits ${RPM_BUILD_ROOT}%{_sysconfdir}/security/limits.d/cloud
install -D packaging/systemd/cloudstack-management.service ${RPM_BUILD_ROOT}%{_unitdir}/%{name}-management.service
install -D packaging/systemd/cloudstack-management.default ${RPM_BUILD_ROOT}%{_sysconfdir}/default/%{name}-management
install -D server/target/conf/cloudstack-sudoers ${RPM_BUILD_ROOT}%{_sysconfdir}/sudoers.d/%{name}-management
touch ${RPM_BUILD_ROOT}%{_localstatedir}/run/%{name}-management.pid
#install -D server/target/conf/cloudstack-catalina.logrotate ${RPM_BUILD_ROOT}%{_sysconfdir}/logrotate.d/%{name}-catalina
# SystemVM template
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/templates/systemvm
cp -r engine/schema/dist/systemvm-templates/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/templates/systemvm
rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/templates/systemvm/md5sum.txt
# UI
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/ui
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-ui/
cp -r client/target/classes/META-INF/webapp/WEB-INF ${RPM_BUILD_ROOT}%{_datadir}/%{name}-ui
cp ui/dist/config.json ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/ui/
cp -r ui/dist/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-ui/
rm -f ${RPM_BUILD_ROOT}%{_datadir}/%{name}-ui/config.json
ln -sf /etc/%{name}/ui/config.json ${RPM_BUILD_ROOT}%{_datadir}/%{name}-ui/config.json
# Package mysql-connector-python
wget -P ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup/wheel https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
wget -P ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup/wheel https://files.pythonhosted.org/packages/e9/93/4860cebd5ad3ff2664ad3c966490ccb46e3b88458b2095145bca11727ca4/setuptools-47.3.1-py3-none-any.whl
wget -P ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup/wheel https://files.pythonhosted.org/packages/32/27/1141a8232723dcb10a595cc0ce4321dcbbd5215300bf4acfc142343205bf/protobuf-3.19.6-py2.py3-none-any.whl
wget -P ${RPM_BUILD_ROOT}%{_datadir}/%{name}-management/setup/wheel https://files.pythonhosted.org/packages/08/1f/42d74bae9dd6dcfec67c9ed0f3fa482b1ae5ac5f117ca82ab589ecb3ca19/mysql_connector_python-8.0.31-py2.py3-none-any.whl
chmod 440 ${RPM_BUILD_ROOT}%{_sysconfdir}/sudoers.d/%{name}-management
chmod 770 ${RPM_BUILD_ROOT}%{_localstatedir}/%{name}/mnt
chmod 770 ${RPM_BUILD_ROOT}%{_localstatedir}/%{name}/management
chmod 770 ${RPM_BUILD_ROOT}%{_localstatedir}/cache/%{name}/management/work
chmod 770 ${RPM_BUILD_ROOT}%{_localstatedir}/cache/%{name}/management/temp
chmod 770 ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/management
chmod 770 ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/agent
# KVM Agent
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/agent
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/agent
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/plugins
install -D packaging/systemd/cloudstack-agent.service ${RPM_BUILD_ROOT}%{_unitdir}/%{name}-agent.service
install -D packaging/systemd/cloudstack-rolling-maintenance@.service ${RPM_BUILD_ROOT}%{_unitdir}/%{name}-rolling-maintenance@.service
install -D packaging/systemd/cloudstack-agent.default ${RPM_BUILD_ROOT}%{_sysconfdir}/default/%{name}-agent
install -D agent/target/transformed/agent.properties ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/agent/agent.properties
install -D agent/target/transformed/environment.properties ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/agent/environment.properties
install -D agent/target/transformed/log4j-cloud.xml ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/agent/log4j-cloud.xml
install -D agent/target/transformed/cloud-setup-agent ${RPM_BUILD_ROOT}%{_bindir}/%{name}-setup-agent
install -D agent/target/transformed/cloudstack-agent-upgrade ${RPM_BUILD_ROOT}%{_bindir}/%{name}-agent-upgrade
install -D agent/target/transformed/cloud-guest-tool ${RPM_BUILD_ROOT}%{_bindir}/%{name}-guest-tool
install -D agent/target/transformed/libvirtqemuhook ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib/libvirtqemuhook
install -D agent/target/transformed/rolling-maintenance ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib/rolling-maintenance
install -D agent/target/transformed/cloud-ssh ${RPM_BUILD_ROOT}%{_bindir}/%{name}-ssh
install -D agent/target/transformed/cloudstack-agent-profile.sh ${RPM_BUILD_ROOT}%{_sysconfdir}/profile.d/%{name}-agent-profile.sh
install -D agent/target/transformed/cloudstack-agent.logrotate ${RPM_BUILD_ROOT}%{_sysconfdir}/logrotate.d/%{name}-agent
install -D plugins/hypervisors/kvm/target/cloud-plugin-hypervisor-kvm-%{_maventag}.jar ${RPM_BUILD_ROOT}%{_datadir}/%name-agent/lib/cloud-plugin-hypervisor-kvm-%{_maventag}.jar
cp plugins/hypervisors/kvm/target/dependencies/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib
cp plugins/storage/volume/storpool/target/*.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib
cp plugins/storage/volume/linstor/target/*.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib
# Usage server
mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/usage
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-usage/lib
install -D usage/target/cloud-usage-%{_maventag}.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-usage/cloud-usage-%{_maventag}.jar
install -D usage/target/transformed/db.properties ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/usage/db.properties
install -D usage/target/transformed/log4j-cloud_usage.xml ${RPM_BUILD_ROOT}%{_sysconfdir}/%{name}/usage/log4j-cloud.xml
cp usage/target/dependencies/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-usage/lib/
cp client/target/lib/mysql*jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-usage/lib/
install -D packaging/systemd/cloudstack-usage.service ${RPM_BUILD_ROOT}%{_unitdir}/%{name}-usage.service
install -D packaging/systemd/cloudstack-usage.default ${RPM_BUILD_ROOT}%{_sysconfdir}/default/%{name}-usage
mkdir -p ${RPM_BUILD_ROOT}%{_localstatedir}/log/%{name}/usage/
# Marvin
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-marvin
cp tools/marvin/dist/Marvin-*.tar.gz ${RPM_BUILD_ROOT}%{_datadir}/%{name}-marvin/
# integration-tests
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-integration-tests
cp -r test/integration/* ${RPM_BUILD_ROOT}%{_datadir}/%{name}-integration-tests/
# MYSQL HA
if [ "x%{_ossnoss}" == "xnoredist" ] ; then
mkdir -p ${RPM_BUILD_ROOT}%{_datadir}/%{name}-mysql-ha/lib
cp -r plugins/database/mysql-ha/target/cloud-plugin-database-mysqlha-%{_maventag}.jar ${RPM_BUILD_ROOT}%{_datadir}/%{name}-mysql-ha/lib
fi
#License files from whisker
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-management-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-management-%{version}/LICENSE
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-common-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-common-%{version}/LICENSE
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-agent-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-agent-%{version}/LICENSE
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-usage-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-usage-%{version}/LICENSE
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-ui-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-ui-%{version}/LICENSE
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-marvin-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-marvin-%{version}/LICENSE
install -D tools/whisker/NOTICE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-integration-tests-%{version}/NOTICE
install -D tools/whisker/LICENSE ${RPM_BUILD_ROOT}%{_defaultdocdir}/%{name}-integration-tests-%{version}/LICENSE
%clean
[ ${RPM_BUILD_ROOT} != "/" ] && rm -rf ${RPM_BUILD_ROOT}
%posttrans common
unalias cp
python_dir=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")
if [ ! -z $python_dir ];then
cp -f -r /usr/share/cloudstack-common/python-site/* $python_dir/
fi
%preun management
/usr/bin/systemctl stop cloudstack-management || true
/usr/bin/systemctl disable cloudstack-management || true
%pre management
id cloud > /dev/null 2>&1 || /usr/sbin/useradd -M -U -c "CloudStack unprivileged user" \
-r -s /bin/sh -d %{_localstatedir}/cloudstack/management cloud || true
rm -rf %{_localstatedir}/cache/cloudstack
# in case of upgrade to 4.9+ copy commands.properties if not exists in /etc/cloudstack/management/
if [ "$1" == "2" ] ; then
if [ -f "%{_datadir}/%{name}-management/webapps/client/WEB-INF/classes/commands.properties" ] && [ ! -f "%{_sysconfdir}/%{name}/management/commands.properties" ] ; then
cp -p %{_datadir}/%{name}-management/webapps/client/WEB-INF/classes/commands.properties %{_sysconfdir}/%{name}/management/commands.properties
fi
fi
# Remove old tomcat symlinks and env config file
if [ -L "%{_datadir}/%{name}-management/lib" ]
then
rm -f %{_datadir}/%{name}-management/bin
rm -f %{_datadir}/%{name}-management/lib
rm -f %{_datadir}/%{name}-management/temp
rm -f %{_datadir}/%{name}-management/work
rm -f %{_sysconfdir}/default/%{name}-management
fi
%post management
# Install mysql-connector-python
pip3 install %{_datadir}/%{name}-management/setup/wheel/six-1.15.0-py2.py3-none-any.whl %{_datadir}/%{name}-management/setup/wheel/setuptools-47.3.1-py3-none-any.whl %{_datadir}/%{name}-management/setup/wheel/protobuf-3.19.6-py2.py3-none-any.whl %{_datadir}/%{name}-management/setup/wheel/mysql_connector_python-8.0.31-py2.py3-none-any.whl
/usr/bin/systemctl enable cloudstack-management > /dev/null 2>&1 || true
/usr/bin/systemctl enable --now rngd > /dev/null 2>&1 || true
grep -s -q "db.cloud.driver=jdbc:mysql" "%{_sysconfdir}/%{name}/management/db.properties" || sed -i -e "\$adb.cloud.driver=jdbc:mysql" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "db.usage.driver=jdbc:mysql" "%{_sysconfdir}/%{name}/management/db.properties" || sed -i -e "\$adb.usage.driver=jdbc:mysql" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "db.simulator.driver=jdbc:mysql" "%{_sysconfdir}/%{name}/management/db.properties" || sed -i -e "\$adb.simulator.driver=jdbc:mysql" "%{_sysconfdir}/%{name}/management/db.properties"
# Update DB properties having master and slave(s), with source and replica(s) respectively (for inclusiveness)
grep -s -q "^db.cloud.slaves=" "%{_sysconfdir}/%{name}/management/db.properties" && sed -i "s/^db.cloud.slaves=/db.cloud.replicas=/g" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "^db.cloud.secondsBeforeRetryMaster=" "%{_sysconfdir}/%{name}/management/db.properties" && sed -i "s/^db.cloud.secondsBeforeRetryMaster=/db.cloud.secondsBeforeRetrySource=/g" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "^db.cloud.queriesBeforeRetryMaster=" "%{_sysconfdir}/%{name}/management/db.properties" && sed -i "s/^db.cloud.queriesBeforeRetryMaster=/db.cloud.queriesBeforeRetrySource=/g" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "^db.usage.slaves=" "%{_sysconfdir}/%{name}/management/db.properties" && sed -i "s/^db.usage.slaves=/db.usage.replicas=/g" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "^db.usage.secondsBeforeRetryMaster=" "%{_sysconfdir}/%{name}/management/db.properties" && sed -i "s/^db.usage.secondsBeforeRetryMaster=/db.usage.secondsBeforeRetrySource=/g" "%{_sysconfdir}/%{name}/management/db.properties"
grep -s -q "^db.usage.queriesBeforeRetryMaster=" "%{_sysconfdir}/%{name}/management/db.properties" && sed -i "s/^db.usage.queriesBeforeRetryMaster=/db.usage.queriesBeforeRetrySource=/g" "%{_sysconfdir}/%{name}/management/db.properties"
if [ ! -f %{_datadir}/cloudstack-common/scripts/vm/hypervisor/xenserver/vhd-util ] ; then
echo Please download vhd-util from http://download.cloudstack.org/tools/vhd-util and put it in
echo %{_datadir}/cloudstack-common/scripts/vm/hypervisor/xenserver/
fi
if [ -f %{_sysconfdir}/sysconfig/%{name}-management ] ; then
rm -f %{_sysconfdir}/sysconfig/%{name}-management
fi
chown -R cloud:cloud /var/log/cloudstack/management
systemctl daemon-reload
%posttrans management
# Print help message
if [ -f "/usr/share/cloudstack-common/scripts/installer/cloudstack-help-text" ];then
sed -i "s,^ACS_VERSION=.*,ACS_VERSION=%{_maventag},g" /usr/share/cloudstack-common/scripts/installer/cloudstack-help-text
/usr/share/cloudstack-common/scripts/installer/cloudstack-help-text management
fi
%preun agent
/sbin/service cloudstack-agent stop || true
if [ "$1" == "0" ] ; then
/sbin/chkconfig --del cloudstack-agent > /dev/null 2>&1 || true
fi
%pre agent
# save old configs if they exist (for upgrade). Otherwise we may lose them
# when the old packages are erased. There are a lot of properties files here.
if [ -d "%{_sysconfdir}/cloud" ] ; then
mv %{_sysconfdir}/cloud %{_sysconfdir}/cloud.rpmsave
fi
%posttrans agent
if [ "$1" == "2" ] ; then
echo "Running %{_bindir}/%{name}-agent-upgrade to update bridge name for upgrade from CloudStack 4.0.x (and before) to CloudStack 4.1 (and later)"
%{_bindir}/%{name}-agent-upgrade
fi
if [ ! -d %{_sysconfdir}/libvirt/hooks ] ; then
mkdir %{_sysconfdir}/libvirt/hooks
fi
cp -a ${RPM_BUILD_ROOT}%{_datadir}/%{name}-agent/lib/libvirtqemuhook %{_sysconfdir}/libvirt/hooks/qemu
mkdir -m 0755 -p /usr/share/cloudstack-agent/tmp
/usr/bin/systemctl restart libvirtd
/usr/bin/systemctl enable cloudstack-agent > /dev/null 2>&1 || true
/usr/bin/systemctl enable cloudstack-rolling-maintenance@p > /dev/null 2>&1 || true
/usr/bin/systemctl enable --now rngd > /dev/null 2>&1 || true
# if saved configs from upgrade exist, copy them over
if [ -f "%{_sysconfdir}/cloud.rpmsave/agent/agent.properties" ]; then
mv %{_sysconfdir}/%{name}/agent/agent.properties %{_sysconfdir}/%{name}/agent/agent.properties.rpmnew
cp -p %{_sysconfdir}/cloud.rpmsave/agent/agent.properties %{_sysconfdir}/%{name}/agent
# make sure we only do this on the first install of this RPM, don't want to overwrite on a reinstall
mv %{_sysconfdir}/cloud.rpmsave/agent/agent.properties %{_sysconfdir}/cloud.rpmsave/agent/agent.properties.rpmsave
fi
systemctl daemon-reload
# Print help message
if [ -f "/usr/share/cloudstack-common/scripts/installer/cloudstack-help-text" ];then
sed -i "s,^ACS_VERSION=.*,ACS_VERSION=%{_maventag},g" /usr/share/cloudstack-common/scripts/installer/cloudstack-help-text
/usr/share/cloudstack-common/scripts/installer/cloudstack-help-text agent
fi
%pre usage
id cloud > /dev/null 2>&1 || /usr/sbin/useradd -M -U -c "CloudStack unprivileged user" \
-r -s /bin/sh -d %{_localstatedir}/cloudstack/management cloud|| true
%preun usage
/sbin/service cloudstack-usage stop || true
if [ "$1" == "0" ] ; then
/sbin/chkconfig --del cloudstack-usage > /dev/null 2>&1 || true
fi
%post usage
if [ -f "%{_sysconfdir}/%{name}/management/db.properties" ]; then
echo "Replacing usage server's db.properties with a link to the management server's db.properties"
rm -f %{_sysconfdir}/%{name}/usage/db.properties
ln -s %{_sysconfdir}/%{name}/management/db.properties %{_sysconfdir}/%{name}/usage/db.properties
/usr/bin/systemctl enable cloudstack-usage > /dev/null 2>&1 || true
fi
if [ -f "%{_sysconfdir}/%{name}/management/key" ]; then
echo "Replacing usage server's key with a link to the management server's key"
rm -f %{_sysconfdir}/%{name}/usage/key
ln -s %{_sysconfdir}/%{name}/management/key %{_sysconfdir}/%{name}/usage/key
fi
if [ ! -f "%{_sysconfdir}/%{name}/usage/key" ]; then
ln -s %{_sysconfdir}/%{name}/management/key %{_sysconfdir}/%{name}/usage/key
fi
mkdir -p /usr/local/libexec
if [ ! -f "/usr/local/libexec/sanity-check-last-id" ]; then
echo 1 > /usr/local/libexec/sanity-check-last-id
fi
chown cloud:cloud /usr/local/libexec/sanity-check-last-id
%posttrans usage
# Print help message
if [ -f "/usr/share/cloudstack-common/scripts/installer/cloudstack-help-text" ];then
sed -i "s,^ACS_VERSION=.*,ACS_VERSION=%{_maventag},g" /usr/share/cloudstack-common/scripts/installer/cloudstack-help-text
/usr/share/cloudstack-common/scripts/installer/cloudstack-help-text usage
fi
%post marvin
pip3 install --upgrade https://files.pythonhosted.org/packages/08/1f/42d74bae9dd6dcfec67c9ed0f3fa482b1ae5ac5f117ca82ab589ecb3ca19/mysql_connector_python-8.0.31-py2.py3-none-any.whl
pip3 install --upgrade /usr/share/cloudstack-marvin/Marvin-*.tar.gz
#No default permission as the permission setup is complex
%files management
%defattr(-,root,root,-)
%dir %{_datadir}/%{name}-management
%dir %attr(0770,root,cloud) %{_localstatedir}/%{name}/mnt
%dir %attr(0770,cloud,cloud) %{_localstatedir}/%{name}/management
%dir %attr(0770,root,cloud) %{_localstatedir}/cache/%{name}/management
%dir %attr(0770,root,cloud) %{_localstatedir}/log/%{name}/management
%config(noreplace) %{_sysconfdir}/default/%{name}-management
%config(noreplace) %{_sysconfdir}/sudoers.d/%{name}-management
%config(noreplace) %{_sysconfdir}/security/limits.d/cloud
%config(noreplace) %attr(0640,root,cloud) %{_sysconfdir}/%{name}/management/db.properties
%config(noreplace) %attr(0640,root,cloud) %{_sysconfdir}/%{name}/management/server.properties
%config(noreplace) %attr(0640,root,cloud) %{_sysconfdir}/%{name}/management/config.json
%config(noreplace) %{_sysconfdir}/%{name}/management/log4j-cloud.xml
%config(noreplace) %{_sysconfdir}/%{name}/management/log4j2.xml
%config(noreplace) %{_sysconfdir}/%{name}/management/environment.properties
%config(noreplace) %{_sysconfdir}/%{name}/management/java.security.ciphers
%attr(0644,root,root) %{_unitdir}/%{name}-management.service
%attr(0755,cloud,cloud) %{_localstatedir}/run/%{name}-management.pid
%attr(0755,root,root) %{_bindir}/%{name}-setup-management
%attr(0755,root,root) %{_bindir}/%{name}-update-xenserver-licenses
%{_datadir}/%{name}-management/conf
%{_datadir}/%{name}-management/lib/*.jar
%{_datadir}/%{name}-management/logs
%{_datadir}/%{name}-management/templates
%attr(0755,root,root) %{_bindir}/%{name}-setup-databases
%attr(0755,root,root) %{_bindir}/%{name}-migrate-databases
%attr(0755,root,root) %{_bindir}/%{name}-set-guest-password
%attr(0755,root,root) %{_bindir}/%{name}-set-guest-sshkey
%attr(0755,root,root) %{_bindir}/%{name}-sysvmadm
%attr(0755,root,root) %{_bindir}/%{name}-setup-encryption
%attr(0755,root,root) %{_bindir}/cmk
%{_datadir}/%{name}-management/setup/*.sql
%{_datadir}/%{name}-management/setup/*.sh
%{_datadir}/%{name}-management/setup/server-setup.xml
%{_datadir}/%{name}-management/webapp/*
%attr(0755,root,root) %{_bindir}/%{name}-external-ipallocator.py
%attr(0755,root,root) %{_initrddir}/%{name}-ipallocator
%dir %attr(0770,root,root) %{_localstatedir}/log/%{name}/ipallocator
%{_defaultdocdir}/%{name}-management-%{version}/LICENSE
%{_defaultdocdir}/%{name}-management-%{version}/NOTICE
%{_datadir}/%{name}-management/setup/wheel/*.whl
%files agent
%attr(0755,root,root) %{_bindir}/%{name}-setup-agent
%attr(0755,root,root) %{_bindir}/%{name}-agent-upgrade
%attr(0755,root,root) %{_bindir}/%{name}-guest-tool
%attr(0755,root,root) %{_bindir}/%{name}-ssh
%attr(0644,root,root) %{_unitdir}/%{name}-agent.service
%attr(0644,root,root) %{_unitdir}/%{name}-rolling-maintenance@.service
%config(noreplace) %{_sysconfdir}/default/%{name}-agent
%attr(0644,root,root) %{_sysconfdir}/profile.d/%{name}-agent-profile.sh
%config(noreplace) %attr(0644,root,root) %{_sysconfdir}/logrotate.d/%{name}-agent
%attr(0755,root,root) %{_datadir}/%{name}-common/scripts/network/cisco
%config(noreplace) %{_sysconfdir}/%{name}/agent
%dir %{_localstatedir}/log/%{name}/agent
%attr(0644,root,root) %{_datadir}/%{name}-agent/lib/*.jar
%attr(0755,root,root) %{_datadir}/%{name}-agent/lib/libvirtqemuhook
%attr(0755,root,root) %{_datadir}/%{name}-agent/lib/rolling-maintenance
%dir %{_datadir}/%{name}-agent/plugins
%{_defaultdocdir}/%{name}-agent-%{version}/LICENSE
%{_defaultdocdir}/%{name}-agent-%{version}/NOTICE
%files common
%dir %attr(0755,root,root) %{_datadir}/%{name}-common/python-site/cloudutils
%dir %attr(0755,root,root) %{_datadir}/%{name}-common/vms
%attr(0755,root,root) %{_datadir}/%{name}-common/scripts
%attr(0755,root,root) /usr/bin/cloudstack-sccs
%attr(0644, root, root) %{_datadir}/%{name}-common/vms/agent.zip
%attr(0644, root, root) %{_datadir}/%{name}-common/vms/cloud-scripts.tgz
%attr(0644, root, root) %{_datadir}/%{name}-common/vms/patch-sysvms.sh
%attr(0644,root,root) %{_datadir}/%{name}-common/python-site/cloud_utils.py
%attr(0644,root,root) %{_datadir}/%{name}-common/python-site/__pycache__/*
%attr(0644,root,root) %{_datadir}/%{name}-common/python-site/cloudutils/*
%attr(0644, root, root) %{_datadir}/%{name}-common/lib/jasypt-1.9.3.jar
%attr(0644, root, root) %{_datadir}/%{name}-common/lib/%{name}-utils.jar
%{_defaultdocdir}/%{name}-common-%{version}/LICENSE
%{_defaultdocdir}/%{name}-common-%{version}/NOTICE
%files ui
%config(noreplace) %attr(0640,root,cloud) %{_sysconfdir}/%{name}/ui/config.json
%{_datadir}/%{name}-ui/*
%{_defaultdocdir}/%{name}-ui-%{version}/LICENSE
%{_defaultdocdir}/%{name}-ui-%{version}/NOTICE
%files usage
%attr(0644,root,root) %{_unitdir}/%{name}-usage.service
%config(noreplace) %{_sysconfdir}/default/%{name}-usage
%attr(0644,root,root) %{_datadir}/%{name}-usage/*.jar
%attr(0644,root,root) %{_datadir}/%{name}-usage/lib/*.jar
%dir %attr(0770,root,cloud) %{_localstatedir}/log/%{name}/usage
%attr(0644,root,root) %{_sysconfdir}/%{name}/usage/db.properties
%attr(0644,root,root) %{_sysconfdir}/%{name}/usage/log4j-cloud.xml
%{_defaultdocdir}/%{name}-usage-%{version}/LICENSE
%{_defaultdocdir}/%{name}-usage-%{version}/NOTICE
%files marvin
%attr(0644,root,root) %{_datadir}/%{name}-marvin/Marvin*.tar.gz
%{_defaultdocdir}/%{name}-marvin-%{version}/LICENSE
%{_defaultdocdir}/%{name}-marvin-%{version}/NOTICE
%files integration-tests
%attr(0755,root,root) %{_datadir}/%{name}-integration-tests/*
%{_defaultdocdir}/%{name}-integration-tests-%{version}/LICENSE
%{_defaultdocdir}/%{name}-integration-tests-%{version}/NOTICE
%if "%{_ossnoss}" == "noredist"
%files mysql-ha
%defattr(0644,cloud,cloud,0755)
%attr(0644,root,root) %{_datadir}/%{name}-mysql-ha/lib/*
%endif
%files baremetal-agent
%attr(0755,root,root) %{_bindir}/cloudstack-setup-baremetal
%changelog
* Thu Dec 22 2022 Rohit Yadav <rohit@apache.org> 4.18.0
- Add support for EL9
* Fri Oct 14 2022 Daan Hoogland <daan.hoogland@gmail.com> 4.18.0
- initialising sanity check pointer file
* Tue Jun 29 2021 David Jumani <dj.davidjumani1994@gmail.com> 4.16.0
- Adding SUSE 15 support
* Thu Apr 30 2015 Rohit Yadav <bhaisaab@apache.org> 4.6.0
- Remove awsapi package
* Wed Nov 19 2014 Hugo Trippaers <hugo@apache.org> 4.6.0
- Create a specific spec for CentOS 7
* Fri Jul 4 2014 Hugo Trippaers <hugo@apache.org> 4.5.0
- Add a package for the mysql ha module
* Fri Oct 5 2012 Hugo Trippaers <hugo@apache.org> 4.1.0
- new style spec file