3649 Commits

Author SHA1 Message Date
Suresh Kumar Anaparti
40dec99659
server: Cleanup allocated snapshots / vm snapshots, and update pending ones to Error on MS start (#8452)
* Remove allocated snapshots / vm snapshots on start

* Check and Cleanup snapshots / vm snapshots on MS start

* rebase fixes

* Update volume state (from Snapshotting) on MS start when its snapshot job not finished and snapshot in Creating state
2025-09-23 08:37:10 +02:00
Wei Zhou
7c7497c624
Merge remote-tracking branch 'apache/4.19' into 4.20 2025-09-15 10:19:27 +02:00
Suresh Kumar Anaparti
6d16ac2113
ScaleIO/PowerFlex smoke tests improvements, and some fixes (#11554)
* ScaleIO/PowerFlex smoke tests improvements, and some fixes

* Fix test_volumes.py, encrypted volume size check (for powerflex volumes)

* Fix test_over_provisioning.py (over provisioning supported for powerflex)

* Update vm snapshot tests

* Update volume size delta in primary storage resource count for user vm volumes only
The VR volumes resource count for PowerFlex volumes is updated here, resulting in resource count discrepancy
(which is re-calculated through ResourceCountCheckTask later, and skips the VR volumes)

* Fix test_import_unmanage_volumes.py (unsupported for powerflex)

* Fix test_sharedfs_lifecycle.py (volume size check for powerflex)

* Update powerflex.connect.on.demand config default to true
2025-09-12 16:17:20 +02:00
Nicolas Vazquez
036fd00170
kvm: Fix NPE in case host UEFI detail is not set on agent connection (#11610) 2025-09-11 10:40:08 +02:00
Vitor Hugo Homem Marzarotto
2e113e5ed7
Change log level of AgentHandler#processRequest() (#10869)
Co-authored-by: Vitor Hugo Homem Marzarotto <vitor.marzarotto@scclouds.com.br>
2025-09-10 11:06:16 +02:00
Suresh Kumar Anaparti
ca62a7dd50
Merge branch '4.19' into 4.20 2025-08-29 15:31:00 +05:30
Wei Zhou
ba7ec88650
SG: Apply rules for both ipv4/ipv6 of VMs with associated account/SG (#11243) 2025-08-29 11:39:50 +02:00
Pearl Dsilva
6e59f4f4cc
Fix deployment of CKS clusters in Basic zone (#11457) 2025-08-21 18:32:01 +05:30
João Jandre
c6daeb4f78
fix snapshot physical size for primary storage (#11448) 2025-08-15 19:52:50 +05:30
Harikrishna
53bc435bdb
Fix create statement for safer upgrades (#11388) 2025-08-04 18:11:47 +05:30
Suresh Kumar Anaparti
742e131283
Update System VM template Guest OS version (#11291) 2025-08-01 13:50:35 +02:00
Nicolas Vazquez
ed0d606e98
Find system VM templates for CKS clusters and SharedFS honouring the preferred architecture (#10946)
* Find system VM templates for CKS cluster honouring the preferred architecture

* Fix unit tests

* Fix checkstyle

* Sort instead of filtering by preferred arch

* Remove unnecesary stubs

* Restore java version

* Address review comments

* Fail and display error message in case the CKS ISO arch doesnt match the selected template arch

* Prefer CKS ISO arch instead of the system VM setting
2025-07-31 16:42:47 +05:30
Suresh Kumar Anaparti
1961aea770
Merge branch '4.19' into 4.20 2025-07-29 18:18:33 +05:30
Manoj Kumar
a32738c52e
Prevent infinite autoscaling (#11244)
* Prevent infinite autoscaling

* Update server/src/main/java/com/cloud/network/as/AutoScaleManagerImpl.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-29 11:41:30 +05:30
Pearl Dsilva
890386e949
Allow custom NTP servers for CPVM (#11210) 2025-07-25 15:21:06 +02:00
Daan Hoogland
0d65c8c454 Merge branch '4.19' into 4.20 2025-07-16 10:28:17 +02:00
Suresh Kumar Anaparti
06c80cdbe9
Remove unfinished usage job entries of the host (#10848) 2025-07-14 12:06:30 +02:00
Abhishek Kumar
49c6fbdfaf
schema: fix missing columns index (#11171)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-07-14 10:10:23 +05:30
Nicolas Vazquez
75147b7811
[Vmware to KVM Migration] Display virt-v2v and ovftool versions for supported hosts for migration (#11019)
* [Vmware to KVM Migration] Display virt-v2v and ovftool versions for supported hosts for migration

* Fix UI display

* Address review comments

* Fix ovftool and version display - also display versions on host details view
2025-06-23 12:49:51 +02:00
Pearl Dsilva
379ee07d88 Updating pom.xml version numbers for release 4.19.4.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 18:00:09 +05:30
Pearl Dsilva
b5e2c181f9 Updating pom.xml version numbers for release 4.20.2.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 15:38:12 +05:30
Pearl Dsilva
c61a5eb430 Updating pom.xml version numbers for release 4.20.1.0
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-05-30 12:43:00 +05:30
Daan Hoogland
0c7d47138d Updating pom.xml version numbers for release 4.19.3.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-05-30 09:08:58 +02:00
Abhishek Kumar
19d6b979af cks: create separate service account in project
A separate service account will be created and added in the project, if
not exist already, when a Kubernetes cluster is deployed in a project.
This account will have a role with limited API access.

Cleanup clusters on owner account cleanup, delete service account
if needed

When the owner account of k8s clusters is deleted, while its node VMs
get expunged, the cluster entry in DB remain present. This fixes the
issue by cleaning up all clusters for the account deleted.

Project k8s service account will be deleted on account cleanup or when
there is no active k8s cluster remaining

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-27 20:08:12 +05:30
Abhishek Kumar
e2f187912c cks: create separate service account in project
A separate service account will be created and added in the project, if
not exist already, when a Kubernetes cluster is deployed in a project.
This account will have a role with limited API access.

Cleanup clusters on owner account cleanup, delete service account
if needed

When the owner account of k8s clusters is deleted, while its node VMs
get expunged, the cluster entry in DB remain present. This fixes the
issue by cleaning up all clusters for the account deleted.

Project k8s service account will be deleted on account cleanup or when
there is no active k8s cluster remaining

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-27 16:24:29 +02:00
Pearl Dsilva
16fc2cd1f0 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-05-27 19:27:33 +05:30
dahn
bb79f0b727
engine/schema: create default network offering for vpc tier with conserve_mode=1 for fresh installation (#10744) (#10843)
Co-authored-by: Wei Zhou <weizhou@apache.org>
2025-05-27 08:17:49 +02:00
Wei Zhou
5444261902
test: fix several simulator CI failures (#10890)
* test: fix several simulator CI failures

* Inject dataStoreProviderManager
2025-05-19 18:33:14 +02:00
Daan Hoogland
8f8c685d17 Merge branch '4.19' into 4.20 2025-05-16 15:51:37 +02:00
Suresh Kumar Anaparti
112dfddd40
Reset the pool id when create volume fails on the allocated pool, and update the resize error when no endpoint exists (#10777)
* Reset the pool id when create volume fails on the allocated pool

- the pool id is persisted while creating the volume, when it fails the pool id is not reverted. On next create volume attempt, CloudStack couldn't find any suitable primary storage even there are pools available with enough capacity as the pool is already assigned to volume which is in Allocated state (and storage pool compatibility check fails). Ensure volume is not assigned to any pool if create volume fails (so the next creation job would pick the suitable pool).

* endpoint check for resize

* update the resize error through callback result instead of exception
2025-05-16 10:26:28 +02:00
slavkap
c183fc9859
Prevent data corruption for StorPool volumes (#10799) 2025-05-16 10:02:33 +02:00
Suresh Kumar Anaparti
52d986081b
Updated Endpoint Selector to pick the Cluster in Enabled state (in addition to Host state) (#10757)
* Consider the clusters with allocation state 'Enabled' for EndPoint selection (in addition to Host state)

* Reset the pool id when create volume fails on the allocated pool

- the pool id is persisted while creating the volume, when it fails the pool id is not reverted. On next create volume attempt, CloudStack couldn't find any suitable primary storage even there are pools available with enough capacity as the pool is already assigned to volume which is in Allocated state (and storage pool compatibility check fails). Ensure volume is not assigned to any pool if create volume fails (so the next creation job would pick the suitable pool).

* endpoint check for resize

* update the resize error through callback result instead of exception

* logger fix
2025-05-13 17:48:49 +05:30
Suresh Kumar Anaparti
95489b8bdd
Direct agents rebalance improvements with multiple management server nodes (#10674)
Sometimes hypervisor hosts (direct agents) stuck with Disconnect state during agent rebalancing activity across multiple management server nodes. This issue was noticed during frequent restart of the management server nodes in the cluster.

When there are multiple management server nodes in a cluster, if one or more nodes are shutdown/start/restart, CloudStack will rebalance the hosts among the remaining nodes or move the nodes to the newly joined management server nodes. During the rebalancing period multiple operations could happen including:

- DirectAgentScan at interval of configured direct.agent.scan.interval
- AgentRebalanceScan to identify and schedule rebalance agents
- TransferAgentScan to transfer the host from original owner to future owner

**Current Rebalance behavior**

1. For hosts that have AgentAttache && not forForward but in Disconnect state, CloudStack simply ignore these hosts without trying to ping again or update the status of the host.
2. For hosts that have AgentAttache && forForward, CloudStack removes the agent but still try to loadDirectlyConnectedHost.

**Improved Rebalance behavior**
During DirectAgentScan: scanDirectAgentToLoad(),  identify hosts that for self-managed hosts that are in Disconnect state (disconnected after pingtimeout).

1. For hosts that have AgentAttache and is forForward, CloudStack should remove the agent
2. For hosts that have AgentAttache and is not forForward but in Disconnect state, CloudStack should try to investigate and update the status to Up if host is pingable.
3. For hosts that don't have AgentAttache, CloudStack should try to loadDirectlyConnectedHost.
2025-05-13 17:47:46 +05:30
Pearl Dsilva
0648d000b2
Support XenServer 8.4 / XCP 8.3 - make scripts python3 compatible (#10684) 2025-05-13 12:35:04 +02:00
Phsm Qwerty
011fced91e
ehancement: add password to configdrive vendor_data.json (#10061) 2025-05-12 16:16:54 +02:00
Pearl Dsilva
a21f912be3 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-05-12 12:41:34 +05:30
Wei Zhou
7e2aa0efe4
engine/schema: create default network offering for vpc tier with conserve_mode=1 for fresh installation (#10744) 2025-05-09 13:51:43 +05:30
Fabricio Duarte
696bc50f3b
Backport #9888 to 4.19: Fix Usage inconsistencies (#10712) 2025-05-09 09:49:14 +02:00
Abhishek Kumar
919c9797cc
server: prevent duplicate HA works and alerts (#10624)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-06 10:42:30 +02:00
Pearl Dsilva
91c1168861 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-05-02 14:59:05 +05:30
Suresh Kumar Anaparti
32cc45e840
[UI] Allow quiescevm and asyncbackup flags while taking volume snapshot from UI when these are supported for the volume (#10265) 2025-05-02 10:45:50 +02:00
Suresh Kumar Anaparti
9f229600e6
Add new config (non-dynamic) for agent connections monitor thread, and keep timeunit to secs (in sync with the earlier Wait config) (#10525) 2025-04-28 15:32:03 +02:00
Pearl Dsilva
0785ba046e Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-04-28 11:10:08 +05:30
Fabricio Duarte
9d263cd71b
Network Usage event model adjustments (#10755) 2025-04-26 17:35:28 +02:00
Wei Zhou
5d28e66b52
vTPM: support KVM and VMware (#10543)
* KVM: add Virtual TPM model and version

* KVM: add admin-only VM setting GUEST.CPU.MODE and GUEST.CPU.MODEL

* VMware: add vTPM

* vTPM: do not set Key due to 'Cannot add multiple devices using the same device key..'

* vTPM: add unit test testTpmModel

* engine/schema: remove user vm details for guest CPU mode/model

* vTPM: extra methods as Daan's requests

* vTPM: add unit tests in VmwareResourceTest

* vTPM: update unit tests in VmwareResourceTest

* vTPM: add unit test in LibvirtComputingResourceTest

* vTPM: use the default TPM version if an invalid version is passed

* vTPM: requires UEFI on vmware and do nothing if it is not enabled/disabled

* vTPM: let uses to add UEFI on vmware

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* vTPM: remove template details for guest CPU mode/model

* UI: boot vm from ISO into UEFI/SECURE mode

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-04-26 14:35:36 +05:30
Abhishek Kumar
12c077d704
api,ui: multi arch improvements (#10289) 2025-04-25 11:02:27 +02:00
Daan Hoogland
d7765343ef Merge branch '4.19' into 4.20 2025-04-17 15:40:10 +02:00
Wei Zhou
7b68615bd9
HA: set correct hostId of HA work for vm migration (#10591) 2025-04-17 10:02:46 +02:00
Bernardo De Marco Gonçalves
40d549b075
backport 10273 (#10702) 2025-04-16 08:45:31 +02:00
Fabricio Duarte
ac6b1b382c
Migrate public templates that have URLs on data migration across secondary storages (#10364)
Co-authored-by: Fabricio Duarte <fabricio.duarte@scclouds.com.br>
2025-04-15 13:48:45 +02:00