36825 Commits

Author SHA1 Message Date
nvazquez
d541e9014d
Merge branch '4.18' into 4.19 2024-08-06 12:29:18 -03:00
Abhishek Kumar
2e0024e216
server, api: account and api entity access improvements
Fixes domain-admin access check to prevent unauthorized access.
Introduces a new non-dynamic global setting - api.allow.internal.db.ids
to control whether to allow using internal DB IDs as API parameters or
not. Default value for the global setting is false.

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-08-06 12:24:19 -03:00
João Jandre
9033ab709e
Fix snapshot chain being deleted on XenServer (#9447)
Using XenServer as the hypervisor, when deleting a snapshot that has a parent, that parent will also get erased on storage, causing data loss. This behavior was introduced with #7873, where the list of snapshot states that can be deleted was changed to add BackedUp snapshots.

This PR changes the states list back to the original list, and swaps the while loop for a do while loop to account for the changes in #7873.

Fixes #9446
2024-08-01 17:33:04 +05:30
Vishesh
f0ba90568d
api: Fix missing keys in listZonesMetrics response (#9419)
This PR fixes #9379
Fix missing keys in listZonesMetrics.

We use BeanUtils.copyProperties to copy the properties of ZoneResponse to ZoneMetricsResponse. Fields with missing getters are skipped from the response because of this.
2024-07-29 10:54:54 +05:30
Abhishek Kumar
d606ee7a07
ui: vm metrics note about behaviour across hypervisors (#9399)
The metrics for VMs and VM disks in CloudStack depend on the hypervisor plugin used for each hypervisor. The behavior can vary across different hypervisors. For VMware, metrics are retrieved as average value for the time interval defined in the global configuration - vmware.stats.time.window.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-29 10:52:31 +05:30
Vishesh
0be87ac654
Fixup CKS UI for external managed clusters (#9434) 2024-07-29 10:48:58 +05:30
Rohit Yadav
3d32ab5a49 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-07-29 09:42:38 +05:30
Pearl Dsilva
cf0e44dd7e
UI: Display Firewall, LB and Port Forwading rules tab for CKS clusters deployed on isolated networks (#9458) 2024-07-29 09:35:55 +05:30
Vishesh
33be0ae745
Fix removal of usage records (#9442) 2024-07-26 11:40:02 +03:00
Rohit Yadav
f8385a48da
Add systemvmtemplate arm64 build support (#9437)
* Move and copy files to new filenames

* Adjust bash scripts to support arm64

* change preseed for arm64 to create EFI partition

* change packer template files to support arm64

* Correct preseed file name in x86_64 packer template

* fixes

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* try different machine cpu

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* don't accelerate KVM for different arch (arm64 vm on amd64 host)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix building arm64 on x86

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update README.md

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: StepBee <stephan@bienek.org>
2024-07-26 13:38:03 +05:30
Fabricio Duarte
49cd5ba64a
Fix link to removed volumes being shown in info card and list view (#8833)
* Framework for validating links in the front-end

* Rename valid links map in the list view
2024-07-24 09:09:07 -03:00
Abhishek Kumar
54c8b71fb5 Merge remote-tracking branch 'apache/4.18' into 4.19 2024-07-22 11:33:34 +05:30
Abhishek Kumar
e41add31e7 saml: signature check improvements
Adminstrators should ensure that IDP configuration has a signing certificate for the actual signature check to be performed. In addition to this, this change introduces a new global setting saml2.check.signature, with the default value of true, which can deliberately fail a SAML login attempt when the SAML response has a missing signature.
Purges the SAML token upon handling the first SAML response.

Authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-19 14:24:43 +05:30
Suresh Kumar Anaparti
3faf7cd2f1
Updating pom.xml version numbers for release 4.19.2.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-07-19 10:29:26 +05:30
Suresh Kumar Anaparti
9f4c895974
Updating pom.xml version numbers for release 4.19.1.0
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
4.19.1.0
2024-07-15 17:19:29 +05:30
Vishesh
a53365d1c7
Merge release branch 4.18 to 4.19
* 4.18:
  Fix list backups after backup vm is expunged (#9392)
2024-07-15 17:07:36 +05:30
Suresh Kumar Anaparti
8a00e25de9
Fix list backups after backup vm is expunged (#9392) 2024-07-15 17:03:59 +05:30
Vishesh
21c98af13d
Fix backup offering buttons for instances in data view (#9391) 2024-07-15 15:53:00 +05:30
Fabricio Duarte
7db83db3dd
Fix radio buttons in instance migration pop-up (#8836) 2024-07-15 10:30:22 +05:30
Abhishek Kumar
6a8494be3b
engine-schema,cks: fix finding non removed network vms (#9339)
Fixes #9331

Only those VMs should be considered network VM which have a NIC entry
that is not marked removed.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-15 09:46:29 +05:30
Rohit Yadav
2cfb541a1d
saml: purge token after first response and improve setting description (#9377)
* saml: purge token after first response and improve setting description

This improves the description of a saml signature checking global
setting, and purges the SAML token upon handling the first SAML
response.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix failing unit test

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-15 09:45:28 +05:30
Rohit Yadav
50586a9481
UI assorted fixes (#9381)
* config.json: make configuration names consistent with other

Makes login related config param to be consistent with the original
`loginFooter`. Group them together.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* ui: assorted UI fixes and copy console URL fix

This brings in related buttons across some infra views and new columns
which are missing. Also fixes the copy console URL issue on some
browsers.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-15 09:44:39 +05:30
Suresh Kumar Anaparti
7f8f113709
Merge branch '4.18' into 4.19 2024-07-15 00:59:06 +05:30
Suresh Kumar Anaparti
877c4d9a64
LibvirtOvsFetchInterfaceCommandWrapperTest fix (test fails in mac) - skip it if no interfaces with eth and wl (#9382)
* test fix (fails in mac) - skip it if no interfaces with eth and wl

* review comments
2024-07-15 00:56:23 +05:30
Vishesh
22ef08154d
Switch back to CLOUD_DB after purging usage records (#9367) 2024-07-14 14:58:36 +05:30
Suresh Kumar Anaparti
d9f77b841c
Ignore non-managed pools for storage pool access preparation (#9376) 2024-07-14 14:50:26 +05:30
Abhishek Kumar
c71ab9598e
test: refactor ActionEventInterceptorTest to prevent failures (#9384)
Try to intercept test calss methods in new CallContext to prevent
getting any leftover data during assertions.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-12 15:10:55 +02:00
Rene Glover
32cc1d46a5
Copy on pool host when storage pool has ScopeType.HOST (#9356) 2024-07-10 12:30:47 +05:30
Rohit Yadav
5c365fa820
systemvmtemplate: bump new systemvmtemplate 4.19.1 (#9349)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-10 10:01:08 +05:30
Suresh Kumar Anaparti
37c91abd3d
NPE fix, for test_hostha_kvm_host_fencing (#9355) 2024-07-09 12:20:10 +05:30
Henrique Sato
ca66b76cb9
Fix instance & backup resource table columns (#9147)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-07-09 10:23:31 +05:30
Rohit Yadav
7977d1475e
plugins: make default signature check mandatory (#9357)
This improves upon #9219, to make the signature checks mandatory by
default but allows for users to relax the setting if they really must.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-09 09:49:27 +05:30
Wei Zhou
f381763917
test: fix simulator CI failure test_cluster_drs (#9354) 2024-07-09 01:18:50 +05:30
Abhishek Kumar
c35ba606a1
ui: fix account list on domain change in filterview (#9345)
* ui: fix account list on domain change in filterview

Fixes #9274

* fix

* filter accounts for selected domain

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-08 17:24:38 +05:30
Pearl Dsilva
f35f3f6a96
UI: create network form constantly tries to list domains - which is not permitted for normal users (#9348) 2024-07-08 08:28:54 +02:00
Abhishek Kumar
b080215c08 utils: honor custom timeout for script execution for exit value
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-08 11:35:28 +05:30
Abhishek Kumar
d99cf93d7c Merge remote-tracking branch 'apache/4.18' into 4.19 2024-07-06 11:31:23 +05:30
Abhishek Kumar
2cf838cc1d framework/cluster: improve cluster service, integration API server
- mTLS implementation for cluster service communication
- Listen only on the specified cluster node IP address instead of all interfaces
- Validate incoming cluster service requests are from peer management servers based on the server's certificate dns name which can be through global config - ca.framework.cert.management.custom.san
- Hardening of KVM command wrapper script exeicution
- Improve API server integration port check
- cloudstack-management.default: don't have JMX configuration if not needed. JMX is used for instrumentation; users who need to use it should enable it explicitly

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-06 11:29:26 +05:30
Abhishek Kumar
beebeed5e2 framework/cluster: improve cluster service and integration API service
- mTLS implementation for cluster service communication
- Listen only on the specified cluster node IP address instead of all interfaces
- Validate incoming cluster service requests are from peer management servers based on the server's certificate dns name which can be through global config - ca.framework.cert.management.custom.san
- Hardening of KVM command wrapper script execution
- Improve API server integration port check
- cloudstack-management.default: don't have JMX configuration if not needed. JMX is used for instrumentation; users who need to use it should enable it explicitly

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-05 19:05:06 +05:30
Vishesh
ba85b4d4ff
Merge release branch 4.18 to 4.19
* 4.18:
  linstor: deleteAsync fallback to volume UUID if path not set yet (#9325)
  linstor: do not use the same static ApiClient for all calls (#9326)
2024-07-05 12:45:27 +05:30
Rene Peinthor
eb1a0ef7f2
linstor: deleteAsync fallback to volume UUID if path not set yet (#9325) 2024-07-04 17:35:01 +05:30
Wei Zhou
374ea7d482
UI: fix "Request failed." error when open vnf appliances tab of network (#9334) 2024-07-04 17:32:52 +05:30
Rene Peinthor
94e5ceff7f
linstor-ui: Fix add primary storage if Linstor protocol selected (#9327)
url param wasn't set if protocol Linstor is selected
2024-07-04 15:03:38 +05:30
Abhisar Sinha
79b1427d3e
test_primary_storage_scope.py should only run with kvm, vmware and simulator (#9323)
* test_primary_storage_scope should only run with kvm, vmware and simulator

* move cluster create and storage pool create from setup to test so that they are cleaned up in case of failure

* fixed lint failure

* using super class' tearDown
2024-07-04 13:29:05 +05:30
Rene Peinthor
121a35d666
linstor: do not use the same static ApiClient for all calls (#9326)
The client.setBasePath() would overwrite the Linstor controller IP/host
for all current client users. This is basically a race condition
that triggered as soon as you had configured 2 different primary storages
with different Linstor controllers.
2024-07-03 16:13:16 +05:30
Henrique Sato
d79735606b
Handle public IP race conditions (#9234)
* Lock public IP

* Release IP if ID is not null

* Fix NPEs

Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-06-29 10:28:01 +05:30
Abhisar Sinha
063dc60114
Change storage pool scope from Cluster to Zone and vise versa (#8875)
* New feature: Change storage pool scope

* Added checks for Ceph/RBD

* Update op_host_capacity table on primary storage scope change

* Storage pool scope change integration test

* pull 8875 : Addressed review comments

* Pull 8875: remove storage checks, AbstractPrimayStorageLifeCycleImpl class

* Pull 8875: Fixed integration test failure

* Pull 8875: Review comments

* Pull 8875: review comments + broke changeStoragePoolScope into smaller functions

* Added UT for changeStoragePoolScope

* Rename AbstractPrimaryDataStoreLifeCycleImpl to BasePrimaryDataStoreLifeCycleImpl

* Pull 8875: Dao review comments

* Pull 8875: Rename changeStoragePoolScope.vue to ChangeStoragePoolScope.vue

* Pull 8875: Created a new smokes test file + A single warning msg in ui

* Pull 8875: Added cleanup in test_primary_storage_scope.py

* Pull 8875: Type in en.json

* Pull 8875: cleanup array in test_primary_storage_scope.py

* Pull:8875 Removing extra whitespace at eof of StorageManagerImplTest

* Pull 8875: Added UT for PrimaryDataStoreHelper and BasePrimaryDataStoreLifeCycleImpl

* Pull 8875: Added license header

* Pull 8875: Fixed sql query for vmstates

* Pull 8875: Changed icon plus info on disabled mode in apidoc

* Pull 8875: Change scope should not work for local storage

* Pull 8875: Change scope completion event

* Pull 8875: Added api findAffectedVmsForStorageScopeChange

* Pull 8875: Added UT for findAffectedVmsForStorageScopeChange and removed listByPoolIdVMStatesNotInCluster

* Pull 8875: Review comments + Vm name in response

* Pull 8875: listByVmsNotInClusterUsingPool was returning duplicate VM entries because of multiple volumes in the VM satisfying the criteria

* Pull 8875: fixed listAffectedVmsForStorageScopeChange UT

* listAffectedVmsForStorageScopeChange should work if the pool is not disabled

* Fix listAffectedVmsForStorageScopeChangeTest UT

* Pull 8875: add volume.removed not null check in VmsNotInClusterUsingPool query

* Pull 8875: minor refactoring in changeStoragePoolScopeToCluster

* Update server/src/main/java/com/cloud/storage/StorageManagerImpl.java

* fix eof

* changeStoragePoolScopeToZone should connect pool to all Up hosts

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-29 10:03:34 +05:30
Suresh Kumar Anaparti
2ca1b474bd
PowerFlex/ScaleIO SDC client connection improvements (#9268)
* Mitigation for non-scalable Powerflex/ScaleIO clients
- Added ScaleIOSDCManager to manage SDC connections, checks clients limit, prepare and unprepare SDC on the hosts.
- Added commands for prepare and unprepare storage clients to prepare/start and stop SDC service respectively on the hosts.
- Introduced config 'storage.pool.connected.clients.limit' at storage level for client limits, currently support for Powerflex only.

* tests issue fixed

* refactor / improvements

* lock with powerflex systemid while checking connections limit

* updated powerflex systemid lock to hold till sdc preparation

* Added custom stats support for storage pool, through listStoragePools API

* code improvements, and unit tests

* unit tests fixes

* Update config 'storage.pool.connected.clients.limit' to dynamic, and some improvements

* Stop SDC on host after migration if no volumes mapped to host

* Wait for SDC to connect after scini service start, and some log improvements

* Do not throw exception (log it) when SDC is not connected while revoking access for the powerflex volume

* some log improvements
2024-06-29 10:01:50 +05:30
GaOrtiga
814c8b6a12
differentiate between instalation ISO and attached ISO (#9146)
Co-authored-by: Gabriel <gabriel.fernandes@scclouds.com.br>
2024-06-28 21:01:22 +05:30
Vishesh
bcbf152a05
Merge branch '4.18' into 4.19 2024-06-28 20:14:21 +05:30