527 Commits

Author SHA1 Message Date
Wei Zhou
574ed7857b
.github: Update to JDK 17 in ci.yml and build.yml (#11030)
* .github: Update to JDK 17 in build.yml

* .github: update JDK in ci.yml

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-09-01 10:14:09 +02:00
Suresh Kumar Anaparti
1033be4b31
Updating pom.xml version numbers for release 4.22.0.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-28 12:00:42 +05:30
Suresh Kumar Anaparti
f9513b47bf
Updating pom.xml version numbers for release 4.21.0.0
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-22 11:42:37 +05:30
Pearl Dsilva
0d4147f3f6
Netris Network Plugin Integration with CloudStack (#10458)
The Netris Plugin introduces Netris as a network service provider in CloudStack to be able to create and manage Virtual Private Clouds (VPCs) in CloudStack, being able to orchestrate the following network functionalities:

- Network segmentation with Netris-VXLAN isolation method
- Routing between "public" IP and network segments with an ACS ROUTED mode offering
- SourceNAT, DNAT, 1:1 NAT between "public" IP and network segments with an ACS NATTED mode offering
- Routing between VPC network segments (tiers in ACS nomenclature)
- Access Lists (ACLs) between VPC tiers and "public" network (TCP, UDP, ICMP) both as global egress rules and "public" IP specific ingress rules.
- ACLs between VPC network tiers (TCP, UDP, ICMP)
- External load balancing – between VPC network tiers and "public" IP
- Internal load balancing – between VPC network tiers
- CloudStack Virtual Router services (DHCP, DNS, UserData, Password Injection, etc…)
2025-07-25 15:26:42 +05:30
Daan Hoogland
b57994eeb0 Merge branch '4.20' 2025-07-16 14:05:19 +02:00
Wei Zhou
9688cbb095
systemvm: build 4.20.2 template with 'depmod -a' (#11128) 2025-07-15 17:39:41 +05:30
Wei Zhou
ea7919b0c1
Remove test/selenium/ test/src-not-used/ (#11013) 2025-06-16 09:02:35 +02:00
Pearl Dsilva
b5e2c181f9 Updating pom.xml version numbers for release 4.20.2.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 15:38:12 +05:30
Pearl Dsilva
c61a5eb430 Updating pom.xml version numbers for release 4.20.1.0
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-05-30 12:43:00 +05:30
Daan Hoogland
64828f66e8 Merge branch '4.20' 2025-05-13 13:34:23 +02:00
Pearl Dsilva
0648d000b2
Support XenServer 8.4 / XCP 8.3 - make scripts python3 compatible (#10684) 2025-05-13 12:35:04 +02:00
Daan Hoogland
dd84c74e82 Merge branch '4.19' into 4.20 2025-05-13 11:41:36 +02:00
Rene Peinthor
88ce639255
Linstor: implement volume and storage stats (#10850) 2025-05-13 10:06:35 +02:00
Eric Kalendra
55c811547a
Update Mockito to 5.16.1 (#10686)
Dependency name change mockito-inline to mockito-core. Inline is now the default and the last version of mockito-inline released is 5.2.0.

assertj-core in user-authenticators/saml2 pulls in an incompatible version of byte-buddy and required an exclusion. Updating the version of assertj is left for a future PR.
The upgrade requires Java 11+, dropping support for Java 8. CloudStack documentation already says to use Java 11 and does not indicate that java 8 is supported.

Test classes using @RunWith(MockitoJUnitRunner.class) now get run in strict mode. Changes were made to tests where the stubbing intention was clear. In ManagementServerMaintenanceManagerImplTest there are 5 tests where the intention of the test is unclear. Each of the statements now use Mockito.lenient() to avoid the exception. Other cases in the tests follow a similar pattern.
Minor clean up.

Both @Spy and Mockito.spy( should not be used. Favored the annotation.
Both @RunWith(MockitoJUnitRunner.class) and MockitoAnnotations.openMocks(this); should not be used. Favored the annotation.
Unnecessary extends TestCase removed.
@InjectMocks and new in statement unnecessary. Removed new when issue presented.
Some of the Cmd classes like UpdateNetworkCmd have a type tree that includes fields of type Object. This appears to cause issues with injection, requiring that @Mock fields be available. This is where the following fields were added in multiple places:
Object job;
ResponseGenerator _responseGenerator;
Wrong number of parameters for Mockito.when in LibvirtRevertSnapshotCommandWrapperTest.java
2025-04-16 18:10:28 +05:30
Dahn Highland
4c31f9d533 Merge release branch 4.20 to main
* 4.20:
  xenserver: do not destroy halted hypervisor vm (#9175)
  define the limit of projects through the UI (#10652)
  fix projects metrics on dashboard (#10651)
  systemvm: Bump systemvm template version to debian 12.10 (#10628)
  Enhance VPC Network Tier form to auto-populate  Gateway, and Netmask (#10617)
2025-04-09 13:02:31 +02:00
Pearl Dsilva
bc5728a7ff
systemvm: Bump systemvm template version to debian 12.10 (#10628)
* systemvm: Bumpt systemvm template version to debian 12.10

* package right version of sysvm templates
2025-03-29 11:34:53 +05:30
Thomas O'Dowd
d94aaa8b59
Add Cloudian HyperStore Object Storage (#9748) 2025-03-10 07:38:40 +01:00
Daan Hoogland
4a3686297d Updating pom.xml version numbers for release 4.19.3.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-02-25 10:43:11 +01:00
Daan Hoogland
4e321d4356 Updating pom.xml version numbers for release 4.19.2.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-02-20 09:32:07 +01:00
Daan Hoogland
0dcb8da03a Merge branch '4.20' 2025-02-12 16:54:05 +01:00
Daan Hoogland
4f3e8e8c5a Merge branch '4.19' into 4.20 2025-02-12 15:00:51 +01:00
Rene Peinthor
55e8eaab89
Linstor: encryption support (#10126)
This introduces a new encryption mode, instead of a simple bool.
Now also storage driver can just provide encrypted volumes to CloudStack.
2025-02-04 15:18:49 +01:00
Wei Zhou
da94ae2c1c
Merge remote-tracking branch 'apache/4.20' 2024-12-03 09:44:42 +01:00
Wei Zhou
8f63660424
pom.xml: revert systemvm template version to 4.20.0.0 (#10004) 2024-11-29 09:08:45 +01:00
João Jandre
d9774a8462 Updating pom.xml version numbers for release 4.21.0.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:47:06 -03:00
João Jandre
c63c7ee63e Updating pom.xml version numbers for release 4.20.1.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:40:45 -03:00
João Jandre
2fe3fcef7c Updating pom.xml version numbers for release 4.20.0.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-19 08:54:07 -03:00
Vishesh
0655075f51
Feature: Forgot password (#9509)
* Feature: Forgot password

* Address comments

* fixups

* Make forgot password disabled by default

* Apply suggestions from code review

* Address comments
2024-09-10 12:55:28 -03:00
Rohit Yadav
2245d98598 Merge remote-tracking branch 'origin/4.19' 2024-09-05 09:59:33 +05:30
Abhishek Kumar
7e085d5e1d
framework/db: use HikariCP as default and improvements (#9518)
Per docs, if the mysql connector is JDBC2 compliant then it should use
the Connection.isValid API to test a connection.
(https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html#isValid-int-)

This would significantly reduce query lags and API throughput, as for
every SQL query one or two SELECT 1 are performed everytime a Connection
is given to application logic.

This should only be accepted when the driver is JDBC4 complaint.

As per the docs, the connector-j can use /* ping */ before calling
SELECT 1 to have light weight application pings to the server:
https://dev.mysql.com/doc/connector-j/en/connector-j-usagenotes-j2ee-concepts-connection-pooling.html

Replaces dbcp2 connection pool library with more performant HikariCP.
With this unit tests are failing but build is passing.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2024-09-05 09:36:58 +05:30
Abhishek Kumar
31b0ed0a18
framework/config,server: configkey caching (#9628)
Added caching for ConfigKey value retrievals based on the Caffeine
in-memory caching library.
https://github.com/ben-manes/caffeine
Currently, expire time for a cache is 30s and each update of the
config key invalidates the cache. On any update or reset of the
configuration, cache automatically invalidates for it.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-09-05 09:32:56 +05:30
dependabot[bot]
5ff0b999da
Bump org.apache.commons:commons-compress from 1.21 to 1.26.0 (#8683) 2024-09-04 19:14:40 +05:30
Rene Peinthor
97c1a86b64
linstor: update java-linstor dependency to 0.5.2 (#9632)
0.5.1 had a bug while retrieving the bestErrorMessage
for an apicall answer.
2024-09-04 15:15:56 +05:30
Wei Zhou
d7ca05e5ba
Build: drop EL7 support, support JRE17 for packages and sonar check (#8609)
This PR fixes the issue with sonar check

```
Error:  Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184:sonar (default-cli) on project cloudstack:
Error:
Error:  The version of Java (11.0.22) used to run this analysis is deprecated, and SonarCloud no longer supports it. Please upgrade to Java 17 or later.
Error:  You can find more information here: https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/
```

main changes
- Support build/packaging using JDK17
- Still supports JDK11 for building
- Support JRE17 for use in production installation
- Drop EL7 support

The community packages will be still packaged using JDK11. 
If uses want, they can build by JDK17 as well.

Signed-off-by: Wei Zhou <wei.zhou@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-27 16:09:10 +05:30
Suresh Kumar Anaparti
3faf7cd2f1
Updating pom.xml version numbers for release 4.19.2.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-07-19 10:29:26 +05:30
Suresh Kumar Anaparti
9f4c895974
Updating pom.xml version numbers for release 4.19.1.0
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-07-15 17:19:29 +05:30
John Bampton
2919cac643
pre-commit: add codespell to check spelling (#8197) 2024-07-12 09:44:14 +02:00
Rohit Yadav
5c365fa820
systemvmtemplate: bump new systemvmtemplate 4.19.1 (#9349)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-10 10:01:08 +05:30
Daan Hoogland
cb9b3134f7 Merge branch '4.19' 2024-06-14 10:30:10 +02:00
Rene Peinthor
37f4398c80
linstor: Support VM-Instance Disk snapshots (#8796)
* linstor: update to java-linstor 0.5.1

* linstor: Support VM-Instance Disk snapshots

This adds VM-Instance disk snapshot support for
Linstor primary storage. Instance snapshots are stored on
the used Linstor storage pool backend and can be converted
into regular volume snapshots and also reverted.

Instance VM snapshots are not fully atomic but with the
create multi snapshot feature as good as it gets.
Snapshots are done over multiple volumes in the same devicemanager run.
2024-06-13 15:26:33 +05:30
Rohit Yadav
433956c6ae Merge remote-tracking branch 'origin/4.19' 2024-05-01 12:06:39 +05:30
Rohit Yadav
aad410ae6c Merge remote-tracking branch 'origin/4.18' into 4.19 2024-05-01 12:05:52 +05:30
Vishesh
5c9d79e3fb
Upgrade org.json:json version (#8750) 2024-04-30 18:23:51 +05:30
João Jandre
8a101fbbc1 Updating pom.xml version numbers for release 4.18.3.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-04-17 11:11:57 -03:00
João Jandre
154566f914 Updating pom.xml version numbers for release 4.18.2.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-04-12 08:25:04 -03:00
Rohit Yadav
01d44347e7 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

 Conflicts:
	plugins/network-elements/juniper-contrail/pom.xml
	pom.xml
	tools/marvin/setup.py

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-03-21 18:38:01 +05:30
Marcus Sorensen
2e88eb45a3
Update mysql-connector version (#8753)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-03-21 18:09:06 +05:30
Wei Zhou
87284f03f0
Upgrade to JRE17 and Upgrade System VMs/VRs to Python3 and Debian 12 (#8497)
* Update to 4.20.0

* Update to python3

* Upgrade to JRE 17

* Upgrade to Debian 12.4.0

* VR: upgrade to python3

for f in `find systemvm/ -name *.py`;do
    if grep "print " $f >/dev/null;then
        2to3-2.7 -w $f
    else
        2to3-2.7 -p -w $f
    fi
done

* java: Use JRE17 in cloudstack packages and systemvmtemplate

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Add --add-opens to JAVA_OPTS in systemd config

* Add --add-opens to JAVA_OPTS in systemd config for usage

* python3: fix "TypeError: a bytes-like object is required, not 'str'"

* python3: fix "ValueError: must have exactly one of create/read/write/append mode"

* Add --add-exports=java.base/sun.security.x509=ALL-UNNAMED for management server

* Use pip3 instead of pip for centos8

* python3: fix "TypeError: write() argument must be str, not bytes"

```
root@r-1037-VM:~# /opt/cloud/bin/passwd_server_ip.py 10.1.1.1
Traceback (most recent call last):
  File "/opt/cloud/bin/passwd_server_ip.py", line 201, in <module>
    serve()
  File "/opt/cloud/bin/passwd_server_ip.py", line 187, in serve
    initToken()
  File "/opt/cloud/bin/passwd_server_ip.py", line 60, in initToken
    f.write(secureToken)
TypeError: write() argument must be str, not bytes
root@r-1037-VM:~#
```

* Python3: fix "name 'file' is not defined"

```
root@r-1037-VM:~# /opt/cloud/bin/passwd_server_ip.py 10.1.1.1
Traceback (most recent call last):
  File "/opt/cloud/bin/passwd_server_ip.py", line 201, in <module>
    serve()
  File "/opt/cloud/bin/passwd_server_ip.py", line 188, in serve
    loadPasswordFile()
  File "/opt/cloud/bin/passwd_server_ip.py", line 67, in loadPasswordFile
    with file(getPasswordFile()) as f:
NameError: name 'file' is not defined
```

* python3: fix "TypeError: write() argument must be str, not bytes" (two more files)

* Upgrade jaxb version

* python3: fix more "TypeError: a bytes-like object is required, not str"

* python3: fix "Failed to update password server"

Failed to update password server due to: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

* python3: fix "bad duration value: ikelifetime=24.0h"

Jan 15 13:57:20 systemvm ipsec[3080]: # bad duration value: ikelifetime=24.0h

* python3: fix password server "invalid save_password token"

* test: incease retries in test_vpc_vpn.py

* python3: fix passwd_server_ip.py

see error below
```
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: ----------------------------------------
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: Exception occurred during processing of request from ('10.1.1.129', 32782)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: Traceback (most recent call last):
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 650, in process_request_thread
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.finish_request(request, client_address)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.RequestHandlerClass(request, client_address, self)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 720, in __init__
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.handle()
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/http/server.py", line 427, in handle
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.handle_one_request()
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/http/server.py", line 415, in handle_one_request
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     method()
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/opt/cloud/bin/passwd_server_ip.py", line 120, in do_GET
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self.wfile.write(password)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:   File "/usr/lib/python3.9/socketserver.py", line 799, in write
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]:     self._sock.sendall(b)
Jan 15 18:51:21 systemvm passwd_server_ip.py[1507]: TypeError: a bytes-like object is required, not 'str'
```

* python3: fix self.cl.get_router_password in Redundant VRs

```
File "/opt/cloud/bin/cs/CsDatabag.py", line 154, in get_router_password
    md5.update(passwd)
TypeError: Unicode-objects must be encoded before hashing"]
```

* scripts: mark multipath scripts as executable

* systemvm template: remove hyperv packages and do not export

* VR: update default RAM size of System VMs/VRs to 512MiB

Before
```
mysql> select id,name,cpu,speed,ram_size,unique_name,system_use from service_offering where name like "System%";
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
| id | name                                                     | cpu  | speed | ram_size | unique_name                      | system_use |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
|  3 | System Offering For Software Router                      |    1 |   500 |      256 | Cloud.Com-SoftwareRouter         |          1 |
|  4 | System Offering For Software Router - Local Storage      |    1 |   500 |      256 | Cloud.Com-SoftwareRouter-Local   |          1 |
|  5 | System Offering For Internal LB VM                       |    1 |   256 |      256 | Cloud.Com-InternalLBVm           |          1 |
|  6 | System Offering For Internal LB VM - Local Storage       |    1 |   256 |      256 | Cloud.Com-InternalLBVm-Local     |          1 |
|  7 | System Offering For Console Proxy                        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy           |          1 |
|  8 | System Offering For Console Proxy - Local Storage        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy-Local     |          1 |
|  9 | System Offering For Secondary Storage VM                 |    1 |   500 |      512 | Cloud.com-SecondaryStorage       |          1 |
| 10 | System Offering For Secondary Storage VM - Local Storage |    1 |   500 |      512 | Cloud.com-SecondaryStorage-Local |          1 |
| 11 | System Offering For Elastic LB VM                        |    1 |   128 |      128 | Cloud.Com-ElasticLBVm            |          1 |
| 12 | System Offering For Elastic LB VM - Local Storage        |    1 |   128 |      128 | Cloud.Com-ElasticLBVm-Local      |          1 |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
10 rows in set (0.00 sec)
```

New value
```
mysql> select id,name,cpu,speed,ram_size,unique_name,system_use from service_offering where name like "System%";
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
| id | name                                                     | cpu  | speed | ram_size | unique_name                      | system_use |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
|  3 | System Offering For Software Router                      |    1 |   500 |      512 | Cloud.Com-SoftwareRouter         |          1 |
|  4 | System Offering For Software Router - Local Storage      |    1 |   500 |      512 | Cloud.Com-SoftwareRouter-Local   |          1 |
|  5 | System Offering For Internal LB VM                       |    1 |   256 |      512 | Cloud.Com-InternalLBVm           |          1 |
|  6 | System Offering For Internal LB VM - Local Storage       |    1 |   256 |      512 | Cloud.Com-InternalLBVm-Local     |          1 |
|  7 | System Offering For Console Proxy                        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy           |          1 |
|  8 | System Offering For Console Proxy - Local Storage        |    1 |   500 |     1024 | Cloud.com-ConsoleProxy-Local     |          1 |
|  9 | System Offering For Secondary Storage VM                 |    1 |   500 |      512 | Cloud.com-SecondaryStorage       |          1 |
| 10 | System Offering For Secondary Storage VM - Local Storage |    1 |   500 |      512 | Cloud.com-SecondaryStorage-Local |          1 |
| 11 | System Offering For Elastic LB VM                        |    1 |   128 |      512 | Cloud.Com-ElasticLBVm            |          1 |
| 12 | System Offering For Elastic LB VM - Local Storage        |    1 |   128 |      512 | Cloud.Com-ElasticLBVm-Local      |          1 |
+----+----------------------------------------------------------+------+-------+----------+----------------------------------+------------+
10 rows in set (0.01 sec)
```

* debian12: fix test_network_ipv6 and test_vpc_ipv6

* python3: remove duplicated imports

* debian12: failed to start Apache2 server (SSLCipherSuite @SECLEVEL=0)

error message
```
[Sat Jan 20 22:51:14.595143 2024] [ssl:emerg] [pid 10200:tid 140417063888768] AH02562: Failed to configure certificate cloudinternal.com:443:0 (with chain), check /etc/ssl/certs/cert_apache.crt
[Sat Jan 20 22:51:14.595234 2024] [ssl:emerg] [pid 10200:tid 140417063888768] SSL Library Error: error:0A00018E:SSL routines::ca md too weak
AH00016: Configuration Failed
```

openssl version
```
root@s-167-VM:~# openssl version -a
OpenSSL 3.0.11 19 Sep 2023 (Library: OpenSSL 3.0.11 19 Sep 2023)
built on: Mon Oct 23 17:52:22 2023 UTC
platform: debian-amd64
options:  bn(64,64)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -fzero-call-used-regs=used-gpr -DOPENSSL_TLS_SECURITY_LEVEL=2 -Wa,--noexecstack -g -O2 -ffile-prefix-map=/build/reproducible-path/openssl-3.0.11=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_BUILDING_OPENSSL -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-3"
MODULESDIR: "/usr/lib/x86_64-linux-gnu/ossl-modules"
Seeding source: os-specific
CPUINFO: OPENSSL_ia32cap=0x80202001478bfffd:0x0
```

certificate
```
root@s-167-VM:~# keytool -printcert -rfc -file /usr/local/cloud/systemvm/certs/realhostip.crt
-----BEGIN CERTIFICATE-----
MIIFZTCCBE2gAwIBAgIHKBCduBUoKDANBgkqhkiG9w0BAQUFADCByjELMAkGA1UE
BhMCVVMxEDAOBgNVBAgTB0FyaXpvbmExEzARBgNVBAcTClNjb3R0c2RhbGUxGjAY
BgNVBAoTEUdvRGFkZHkuY29tLCBJbmMuMTMwMQYDVQQLEypodHRwOi8vY2VydGlm
aWNhdGVzLmdvZGFkZHkuY29tL3JlcG9zaXRvcnkxMDAuBgNVBAMTJ0dvIERhZGR5
IFNlY3VyZSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTERMA8GA1UEBRMIMDc5Njky
ODcwHhcNMTIwMjAzMDMzMDQwWhcNMTcwMjA3MDUxMTIzWjBZMRkwFwYDVQQKDBAq
LnJlYWxob3N0aXAuY29tMSEwHwYDVQQLDBhEb21haW4gQ29udHJvbCBWYWxpZGF0
ZWQxGTAXBgNVBAMMECoucmVhbGhvc3RpcC5jb20wggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQCDT9AtEfs+s/I8QXp6rrCw0iNJ0+GgsybNHheU+JpL39LM
TZykCrZhZnyDvwdxCoOfE38Sa32baHKNds+y2SHnMNsOkw8OcNucHEBX1FIpOBGp
h9D6xC+umx9od6xMWETUv7j6h2u+WC3OhBM8fHCBqIiAol31/IkcqDxxsHlQ8S/o
CfTlXJUY6Yn628OA1XijKdRnadV0hZ829cv/PZKljjwQUTyrd0KHQeksBH+YAYSo
2JUl8ekNLsOi8/cPtfojnltzRI1GXi0ZONs8VnDzJ0a2gqZY+uxlz+CGbLnGnlN4
j9cBpE+MfUE+35Dq121sTpsSgF85Mz+pVhn2S633AgMBAAGjggG+MIIBujAPBgNV
HRMBAf8EBTADAQEAMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAOBgNV
HQ8BAf8EBAMCBaAwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL2NybC5nb2RhZGR5
LmNvbS9nZHMxLTY0LmNybDBTBgNVHSAETDBKMEgGC2CGSAGG/W0BBxcBMDkwNwYI
KwYBBQUHAgEWK2h0dHA6Ly9jZXJ0aWZpY2F0ZXMuZ29kYWRkeS5jb20vcmVwb3Np
dG9yeS8wgYAGCCsGAQUFBwEBBHQwcjAkBggrBgEFBQcwAYYYaHR0cDovL29jc3Au
Z29kYWRkeS5jb20vMEoGCCsGAQUFBzAChj5odHRwOi8vY2VydGlmaWNhdGVzLmdv
ZGFkZHkuY29tL3JlcG9zaXRvcnkvZ2RfaW50ZXJtZWRpYXRlLmNydDAfBgNVHSME
GDAWgBT9rGEyk2xF1uLuhV+auud2mWjM5zArBgNVHREEJDAighAqLnJlYWxob3N0
aXAuY29tgg5yZWFsaG9zdGlwLmNvbTAdBgNVHQ4EFgQUZyJz9/QLy5TWIIscTXID
E8Xk47YwDQYJKoZIhvcNAQEFBQADggEBAKiUV3KK16mP0NpS92fmQkCLqm+qUWyN
BfBVgf9/M5pcT8EiTZlS5nAtzAE/eRpBeR3ubLlaAogj4rdH7YYVJcDDLLoB2qM3
qeCHu8LFoblkb93UuFDWqRaVPmMlJRnhsRkL1oa2gM2hwQTkBDkP7w5FG1BELCgl
gZI2ij2yxjge6pOEwSyZCzzbCcg9pN+dNrYyGEtB4k+BBnPA3N4r14CWbk+uxjrQ
6j2Ip+b7wOc5IuMEMl8xwTyjuX3lsLbAZyFI9RCyofwA9NqIZ1GeB6Zd196rubQp
93cmBqGGjZUs3wMrGlm7xdjlX6GQ9UvmvkMub9+lL99A5W50QgCmFeI=
-----END CERTIFICATE-----

Warning:
The certificate uses the SHA1withRSA signature algorithm which is considered a security risk. This algorithm will be disabled in a future update.
```

it comes from
```
$ openssl x509 -in ./systemvm/agent/certs/realhostip.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 11277268652730408 (0x28109db8152828)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.", OU = http://certificates.godaddy.com/repository, CN = Go Daddy Secure Certification Authority, serialNumber = 07969287
        Validity
            Not Before: Feb  3 03:30:40 2012 GMT
            Not After : Feb  7 05:11:23 2017 GMT
        Subject: O = *.realhostip.com, OU = Domain Control Validated, CN = *.realhostip.com
```

* debian12: use ed25519 instead of rsa as ssh-rsa has been deprecated in OpenSSH

on xenserver
```
[root@pr8497-t8906-xenserver-71-xs2 ~]# ssh -i .ssh/id_rsa.cloud -p 3922 169.254.214.153
Warning: Permanently added '[169.254.214.153]:3922' (ECDSA) to the list of known hosts.
Permission denied (publickey).
```
in the CPVM
Jan 22 19:31:09 v-1-VM sshd[2869]: userauth_pubkey: signature algorithm ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]
Jan 22 19:31:09 v-1-VM sshd[2869]: Connection closed by authenticating user root 169.254.0.1 port 54704 [preauth]
```

ssh-dss (DSA) is not supported either

* debian12: add PubkeyAcceptedAlgorithms=+ssh-rsa to sshd_config

* VR: install python3 packages in case of Debian 11

* pom.xml: exclude systemvm/agent/packages/* in license check

* systemvm: do not patch router/systemvm during startup

this will cause 4.19 SYSTEM template not work, but may be expected
- python3 VS python2 (default)
- openSSL 3.0.1 VS 1.1.1w
- openssh-server 9.1 VS 8.4

* VR: patch router/systemvm if template is debian11

This supports debian 11 template by
- revert change in systemvm/debian/etc/ssh/sshd_config
- patch VR/systemvms during startup
- install packages during patching system vm/routers

* python3 flake: fix E502 the backslash is redundant between brackets

```
../debian/root/health_checks/router_version_check.py:55:70: E502 the backslash is redundant between brackets
../debian/root/health_checks/router_version_check.py:58:61: E502 the backslash is redundant between brackets
../debian/root/health_checks/router_version_check.py:67:71: E502 the backslash is redundant between brackets
../debian/root/health_checks/router_version_check.py:70:60: E502 the backslash is redundant between brackets
../debian/root/health_checks/haproxy_check.py:47:71: E502 the backslash is redundant between brackets
../debian/root/health_checks/haproxy_check.py:48:64: E502 the backslash is redundant between brackets
../debian/root/health_checks/cpu_usage_check.py:43:54: E502 the backslash is redundant between brackets
../debian/root/health_checks/cpu_usage_check.py:46:58: E502 the backslash is redundant between brackets
../debian/root/health_checks/memory_usage_check.py:31:65: E502 the backslash is redundant between brackets
../debian/root/health_checks/memory_usage_check.py:42:57: E502 the backslash is redundant between brackets
../debian/root/health_checks/memory_usage_check.py:45:63: E502 the backslash is redundant between brackets
```

* python3 flake: fix E275 missing whitespace after keyword

```
../debian/opt/cloud/bin/cs_firewallrules.py:29:20: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_dhcp.py:27:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_dhcp.py:36:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_guestnetwork.py:33:20: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_guestnetwork.py:35:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_vpnusers.py:37:16: E275 missing whitespace after keyword
../debian/opt/cloud/bin/merge.py:230:11: E275 missing whitespace after keyword
../debian/opt/cloud/bin/merge.py:239:19: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_remoteaccessvpn.py:24:12: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs_site2sitevpn.py:24:12: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs/CsHelper.py:90:15: E275 missing whitespace after keyword
../debian/opt/cloud/bin/cs/CsAddress.py:367:15: E275 missing whitespace after keyword
```

* python3 flake: fix configure.py

```
../debian/opt/cloud/bin/configure.py:24:22: E401 multiple imports on one line
../debian/opt/cloud/bin/configure.py:43:180: E501 line too long (294 > 179 characters)
../debian/opt/cloud/bin/configure.py:46:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/configure.py:63:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/configure.py:65:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/configure.py:72:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/configure.py:310:25: E711 comparison to None should be 'if cond is not None:'
../debian/opt/cloud/bin/configure.py:312:29: E711 comparison to None should be 'if cond is None:'
../debian/opt/cloud/bin/configure.py:378:25: E711 comparison to None should be 'if cond is not None:'
../debian/opt/cloud/bin/configure.py:380:29: E711 comparison to None should be 'if cond is None:'
../debian/opt/cloud/bin/configure.py:490:29: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
../debian/opt/cloud/bin/configure.py:642:16: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/configure.py:644:18: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/configure.py:1416:1: E305 expected 2 blank lines after class or function definition, found 1
```

* python3 flake: fix other python files

```
../debian/opt/cloud/bin/vmdata.py:97:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
../debian/opt/cloud/bin/vmdata.py:99:14: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`

../debian/opt/cloud/bin/cs/CsRedundant.py:438:53: E203 whitespace before ':'
../debian/opt/cloud/bin/cs/CsRedundant.py:461:53: E203 whitespace before ':'
../debian/opt/cloud/bin/cs/CsRedundant.py:499:5: E303 too many blank lines (2)

../debian/opt/cloud/bin/cs/CsDatabag.py:189:1: E302 expected 2 blank lines, found 1
../debian/opt/cloud/bin/cs/CsDatabag.py:193:37: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`

../debian/opt/cloud/bin/cs/CsHelper.py:118:30: E231 missing whitespace after ','
../debian/opt/cloud/bin/cs/CsHelper.py:119:15: E225 missing whitespace around operator
../debian/opt/cloud/bin/cs/CsHelper.py:127:19: E225 missing whitespace around operator

../debian/opt/cloud/bin/cs/CsAddress.py:324:43: E221 multiple spaces before operator

../debian/opt/cloud/bin/cs/CsVpcGuestNetwork.py:28:1: E302 expected 2 blank lines, found 1
```

* python3 flake: fix CsNetfilter.py

```
../debian/opt/cloud/bin/cs/CsNetfilter.py:226:13: E117 over-indented
../debian/opt/cloud/bin/cs/CsNetfilter.py:233:180: E501 line too long (197 > 179 characters)
../debian/opt/cloud/bin/cs/CsNetfilter.py:241:14: E201 whitespace after '{'
../debian/opt/cloud/bin/cs/CsNetfilter.py:242:14: E201 whitespace after '{'
../debian/opt/cloud/bin/cs/CsNetfilter.py:247:18: E201 whitespace after '{'
../debian/opt/cloud/bin/cs/CsNetfilter.py:247:74: E202 whitespace before '}'
../debian/opt/cloud/bin/cs/CsNetfilter.py:248:18: E201 whitespace after '{'
```

* systemvm/test: fix sys.path

```
$ bash runtests.sh
/usr/bin/python
Python 3.10.12
Running pycodestyle to check systemvm/python code for errors
Running pylint to check systemvm/python code for errors
Python 3.10.12
pylint 2.12.2
astroid 2.9.3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Running systemvm/python unit tests
....Device "eth0" does not exist.
.....................
----------------------------------------------------------------------
Ran 25 tests in 0.008s

OK
```

* Revert "systemvm template: remove hyperv packages and do not export"

This reverts commit 4383d59d031bde6eae7ebba261ff641ca0a66cd5.

* debian12: move SQL change to schema-41900to42000.sql

* debian12: update systemvm template version to 4.20 in pom.xml

* pom.xml: fix NPE if templates do not exist on download.cloudstack.org

* debian12: increase default system offering for routers to 384MiB RAM

* CKS: fix addkubernetessupportedversion failed with JRE17

```
marvin.cloudstackException.CloudstackAPIException: Execute cmd: addkubernetessupportedversion failed, due to: errorCode: 530, errorText:Cannot invoke "org.apache.cloudstack.engine.subsystem.api.storage.ObjectInDataStoreStateMachine$State.toString()" because the return value of "com.cloud.api.query.vo.TemplateJoinVO.getState()" is null
```

* python3: revert changes by 2to3 with systemvm/debian/root/health_checks/*.py

* debian12: use ISO/packages on download.cloudstack.org

* VR: Update default ram size to 384

* debian12: fix router_version_check.py after VR live-patch and add health check in test_routers.py

* debian12: fix build error after log4j 2.x merge

* VR: Update default ram size to 512MB (again)

This reverts commit 578dd2b73f380e8231ae1eb59827230757cac5e8 and efafa8c4d63775653a2cd406fca10784fbcec3e3.

* systemvmtemplate: Upgrade to Debian 12.5.0

* systemvm template: increase swap to 512MB

* VR: fix health check error due to deprecated SafeConfigParser

warning below
```
root@r-20-VM:~# /opt/cloud/bin/getRouterMonitorResults.sh true
/root/monitorServices.py:59: DeprecationWarning: The SafeConfigParser class has been renamed to ConfigParser in Python 3.2. This alias will be removed in Python 3.12. Use ConfigParser directly instead.
  parser = SafeConfigParser()
```

* test: fix wget does not work in macchinina vms on vmware80u1

fixes error below
```
{Cmd: wget -t 1 -T 1 www.google.com via Host: 10.0.55.186} {returns: ["wget: '/usr/lib/libpcre.so.1' is not an ELF file", "wget: can't load library 'libpcre.so.1'"]}
```

* packaging: add message for VR memory upgrade after packages installation

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-02-26 18:07:50 +05:30
João Jandre
49cecaed06
Normalize loggers and upgrade log4j 1.2 to log4j 2.19 (#7131)
* Normalize logs

All classes that could have their loggers inherited from their fathers had their own loggers deleted;
Most loggers didn't have to be static, so most of them were normalized so that they wouldn't be;
All loggers are protected now;
Static logger's name are now 'LOGGER';
Non-static logger's name are now 'logger';
New class DbUpgradeAbstractImpl created so that all Upgraders extend it and inherit its logger

* Upgrade log4j

* fix errors caused by the merge

* Refactor cglibThrowableRenderer functionality to log4j2 and upgrade the last configuration files

* fix sonarcloud bug

* Fix errors caused by merge, remove some unused loggers, and rename a variable that was mistakenly renamed on the normalization commit

* Readd snmpTrapAppender, remove TestAppender

* Regenerate changes

* regenerate changes

* refactor last custom appender

* fix systemvm configuration xml

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Fix utils pom

* fix some tests

* regenerate changes

* Fix jar being printed on exception

* fix logging in system VMs, fix commands not having log4j2 classpath.

* regenerate changes

* Fix some unwanted renomeations

* fix end of file

* regenerate changes

* regenerate changes

* fix merge error

* regenerate changes

* fix tests

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* readd reload4j to tungsten as juniper depends on it

* Regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* re-add reload4j dependency to network-contrail, as juniper depends on it

* regenerate changes

* regenerate changes

* regenerate changes

* fix typo

* regenerate changes

* regenerate changes

* Fix end of files

* regenerate changes

* add logj42 to cloud-utils-SHADED.jar

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* Regenerate changes

* regenerate changes

* Regenerate changes

* Regenerate changes

* fix some tests

* Regenerate changes

* Regenerate changes

* fix test

* Regenerate changes

* Regenerate changes
2024-02-08 09:55:41 -03:00
Vishesh
399bd0a067
Upgrade to mockito 4 and handle Mockito deprecations (#8427) 2024-02-06 14:20:37 +01:00