3766 Commits

Author SHA1 Message Date
Wei Zhou
6089c161a6
Merge remote-tracking branch 'apache/4.20' 2025-10-08 15:40:33 +02:00
Wei Zhou
89d2b17461
storage: change storage pool to Up state when cancel storage migration (#11773)
* storage: change storage pool to Up state when cancel storage migration

* Update 11773: connect host to shared pool after cancelling storage migration

* Update 11773: update db only

* Update 11773: skip capacity update for storpool
2025-10-08 15:34:59 +02:00
Henrique Sato
cc3170577c
Add Hypervisor default as cache mode for disk offerings (#10282)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2025-10-08 13:39:28 +02:00
Suresh Kumar Anaparti
09b63bc2e8
Storage pool response improvements (#10740)
* Return details of the storage pool in the response including url, and update capacityBytes and capacityIops if applicable while creating storage pool

* Added capacitybytes parameter to the storage pool response in sync with the capacityiops response parameter and createStoragePool cmd request parameter (existing disksizetotal parameter in the storage pool response can be deprecated)

* Don't keep url in details

* Persist the capacityBytes and capacityIops in the storage_pool_details table while creating storage pool as well, for consistency - as these are updated with during update storage pool

* rebase with main fixes
2025-10-08 11:20:37 +05:30
Vishesh
d2615bb142
Add support for providing userdata to system VMs (#11654)
This PR adds support for specifying user data (cloud-init) for system VMs via Zone Scoped global settings. This allows the operators to customize the System VMs and setup monitoring, logging or execute any custom commands.

We set the user data from the global setting in /var/cache/cloud/cmdline, and use the NoCloud datasource to process user data. cloud-init service is still disabled in the system VMs and it's executed as part of the cloud-postinit service which executes the postinit.sh script.

Added global settings:
systemvm.userdata.enabled - Disabled by default. Needs to be enabled to utilize the feature.
console.proxy.vm.userdata - UUID of the User data to be used for Console Proxy
secstorage.vm.userdata - UUID of the User data to be used for Secondary Storage VM
virtual.router.userdata - UUID of the User data to be used for Virtual Routers
2025-10-08 10:44:26 +05:30
Daan Hoogland
5f63e8493c Merge branch '4.20' 2025-10-07 15:12:53 +02:00
Suresh Kumar Anaparti
823cb00a0a
server: do not enable the disabled local storage(s) on host connection during mgmt server / agent start (#11722) 2025-10-07 11:21:47 +02:00
Manoj Kumar
9bcd98876d
Make kvm domain persistent when unmanaged from CS (#11541)
CS creates transient KVM domain.xml. When instance is unmanaged from CS, explicit dump of domain has to be taken to manage is outside of CS.

With this PR

    domainXML gets backed up and becomes persistent for further management of Instance.
    Stopped instance also can be unmanaged, last host for instance is considered for defining domain
    hostid param is supported in unmanageVirtualMachine API for KVM hypervisor and for stopped Instances
    hostid field in response of unmanageVirtualMachine, representing host used for unmanage operation
    Disable unmanaging instance with config drive, can unmanage from API using forced=true param for KVM
2025-10-07 10:32:33 +05:30
Abhisar Sinha
1efa46cb4d
fix removeUsage for backups (#11522) 2025-10-02 17:15:54 +05:30
Manoj Kumar
ca7138b3bd
server: Consider Instance in Starting state as well for allocation algorithm (#11751)
* Consider Instance in Starting state as well for allocation algorithm

* use IN instead of OR statement
2025-10-02 08:13:48 +02:00
Pearl Dsilva
cd12fa5848
Add UUID field for LDAP configuration (#11462)
* Add UUID field for LDAP configuration

* move db changes to the lastest schema file

* Add ID param to list ldapConf API & delete ldapConf API

* fix ui test

* fix 1 ui test

* fix test

* fix api description

---------

Co-authored-by: dahn <daan@onecht.net>
2025-10-01 14:43:22 +02:00
Abhishek Kumar
928972f767
extension/proxmox: add console access for instances (#11601)
This PR introduces console access support for instances deployed using Orchestrator Extensions, available via either VNC or a direct URL.

- CloudStack queries the extension using the getconsole action.
- For VNC-based access, the extension must return host/port/ticket details. CloudStack then forwards these to the Console Proxy VM (CPVM) in the instance’s zone. It is assumed that the CPVM can reach the specified host and port.
- For direct URL access, the extension returns a console URL with the protocol set to `direct`. The URL is then provided directly to the user.
- The built-in Proxmox Orchestrator Extension now supports console access via VNC. The extension calls the Proxmox API to fetch console details and returns them in the required format.

Also, adds changes to send caller details to the extension payload.
```
# cat /var/lib/cloudstack/management/extensions/Proxmox/02b650f6-bb98-49cb-8cac-82b7a78f43a2.json | jq
{
  "caller": {
    "roleid": "6b86674b-7e61-11f0-ba77-1e00c8000158",
    "rolename": "Root Admin",
    "name": "admin",
    "roletype": "Admin",
    "id": "93567ed9-7e61-11f0-ba77-1e00c8000158",
    "type": "ADMIN"
  },
  "virtualmachineid": "126f4562-1f0f-4313-875e-6150cabeb72f",
  ...
```

Documentation PR: https://github.com/apache/cloudstack-documentation/pull/560

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-09-27 08:54:27 +05:30
Abhisar Sinha
23c9e83047
Create Instance from backup on another Zone (DRaaS use case) (#11560)
* draas initial changes

* Added option to enable disaster recovery on a backup respository. Added UpdateBackupRepositoryCmd api.

* Added timeout for mount operation in backup restore configurable via global setting

* Addressed review comments

* fix for simulator test failures

* Added UT for coverage

* Fix create instance from backup ui for other providers

* Added events to add/update backup repository

* Fix race in fetchZones

* One more fix in fetchZones in DeployVMFromBackup.vue

* Fix zone selection in createNetwork via Create Instance from backup form.

* Allow template/iso selection in create instance from backup ui

* rename draasenabled to crosszoneinstancecreation

* Added Cross-zone instance creation in test_backup_recovery_nas.py

* Added UT in BackupManagerTest and UserVmManagerImplTest

* Integration test added for Cross-zone instance creation in test_backup_recovery_nas.py
2025-09-25 13:28:29 +05:30
Wei Zhou
e78b5cc3cc
Merge remote-tracking branch 'apache/4.20' 2025-09-24 09:27:08 +02:00
Suresh Kumar Anaparti
a749206eb8
storage: Mount disabled pools by default when host is booted (#11666) 2025-09-24 08:22:49 +02:00
Suresh Kumar Anaparti
40dec99659
server: Cleanup allocated snapshots / vm snapshots, and update pending ones to Error on MS start (#8452)
* Remove allocated snapshots / vm snapshots on start

* Check and Cleanup snapshots / vm snapshots on MS start

* rebase fixes

* Update volume state (from Snapshotting) on MS start when its snapshot job not finished and snapshot in Creating state
2025-09-23 08:37:10 +02:00
dahn
aca8732102
[router] make a distinction between fatal errors, warnings and unknown as healthcheck result (#10710)
* [routers] distiction between fatal failure and warning or unknown on healthchecks

* UI status for router health checks

* status from scripts varied

* automation signalled errors

* revert removal of update sql

* upgradeversion

* move config item and further cleanup

* handling services better

* backwards compatible response

---------

Co-authored-by: Daan Hoogland <dahn@apache.org>
2025-09-22 11:39:05 +05:30
John Bampton
0108ffd1b0
java: fix one typo in many files (#11284) 2025-09-17 22:05:38 +05:30
John Bampton
33cfedee46
pre-commit add hook check-shebang-scripts-are-executable for Shell (#10150) 2025-09-17 14:22:28 +02:00
John Bampton
f9a72de500
misc: pre-commit auto remove unneeded trailing whitespace (#11289) 2025-09-17 14:21:56 +02:00
CodeBleu
273ae03274
Allow updating of Load Balancer source CIDR list (#11568) 2025-09-17 13:49:46 +02:00
vishesh92
ada750e391
Merge branch '4.20' 2025-09-17 14:26:06 +05:30
Wei Zhou
7c7497c624
Merge remote-tracking branch 'apache/4.19' into 4.20 2025-09-15 10:19:27 +02:00
Suresh Kumar Anaparti
6d16ac2113
ScaleIO/PowerFlex smoke tests improvements, and some fixes (#11554)
* ScaleIO/PowerFlex smoke tests improvements, and some fixes

* Fix test_volumes.py, encrypted volume size check (for powerflex volumes)

* Fix test_over_provisioning.py (over provisioning supported for powerflex)

* Update vm snapshot tests

* Update volume size delta in primary storage resource count for user vm volumes only
The VR volumes resource count for PowerFlex volumes is updated here, resulting in resource count discrepancy
(which is re-calculated through ResourceCountCheckTask later, and skips the VR volumes)

* Fix test_import_unmanage_volumes.py (unsupported for powerflex)

* Fix test_sharedfs_lifecycle.py (volume size check for powerflex)

* Update powerflex.connect.on.demand config default to true
2025-09-12 16:17:20 +02:00
Wei Zhou
70a4503ea1
Merge remote-tracking branch 'apache/4.20' 2025-09-11 14:04:52 +02:00
Wei Zhou
b46e29dc67
Improvement: SSL offloading with Virtual Router (#11468)
* SSL offloading with Virtual Router

* PR11468: fix pre-commit errors

* PR11468: api->getAPI/postAPI in UI

* SSL: add smoke tests for VPC in user project

* PR11468: address Daan's comments

* Fix test/integration/smoke/test_ssl_offloading.py

* SSL: remove ssl certificates when clean up account

* SSL offloading: add unit tests

* SSL offloading: UI fixes part 1

* SSL offloading: UI changes part 2

* SSL offloading: add more unit tests

* SSL offloading: more unit tests 3

* SSL offloading: wrong check

* SSL offloading: more and more unit tests

* SSL offloading: add testUpdateLoadBalancerRule5
2025-09-11 16:37:18 +05:30
Nicolas Vazquez
036fd00170
kvm: Fix NPE in case host UEFI detail is not set on agent connection (#11610) 2025-09-11 10:40:08 +02:00
Vitor Hugo Homem Marzarotto
2e113e5ed7
Change log level of AgentHandler#processRequest() (#10869)
Co-authored-by: Vitor Hugo Homem Marzarotto <vitor.marzarotto@scclouds.com.br>
2025-09-10 11:06:16 +02:00
Wei Zhou
66a584624f
Update md5sum to sha512sum (#11134)
* Update md5sum to sha512sum

* Move SQL to schema-42100to42200.sql
2025-09-03 11:12:43 +02:00
Wei Zhou
ca0c3530ad
utils: add UuidUtils.nameUUIDFromBytes (#11136)
* utils: add UuidUtils.nameUUIDFromBytes

* Fix PR 13922
2025-09-01 08:10:31 +02:00
Wei Zhou
889fc62b60
schema: Add upgrade path from 4.21.0.0 to 4.22.0.0 (#11469)
* Add upgrade path from 4.21.0.0 to 4.22.0.0
* Optimize DbUpgrade files
2025-08-30 16:07:50 +05:30
Suresh Kumar Anaparti
2eb80e0361
Merge branch '4.20' 2025-08-29 15:36:16 +05:30
Suresh Kumar Anaparti
ca62a7dd50
Merge branch '4.19' into 4.20 2025-08-29 15:31:00 +05:30
Wei Zhou
ba7ec88650
SG: Apply rules for both ipv4/ipv6 of VMs with associated account/SG (#11243) 2025-08-29 11:39:50 +02:00
Suresh Kumar Anaparti
1033be4b31
Updating pom.xml version numbers for release 4.22.0.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-28 12:00:42 +05:30
Suresh Kumar Anaparti
f9513b47bf
Updating pom.xml version numbers for release 4.21.0.0
Signed-off-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-22 11:42:37 +05:30
slavkap
1272b13087
Fix of create a template from a StorPool snapshot on another zone (#11490)
* Fix of create template from snapshot on another zone

When a snapshot has a copy on StorPool primary storage in another zone, but the original snapshot resides on secondary storage, creating a template from the copied snapshot results in the template being created in the first zone.
If the snapshot.backup.to.secondary setting is disabled, and a user creates a volume or template from a snapshot, the snapshot is temporarily backed up to secondary storage during the operation. After the operation, this backup should be deleted. However, the snapshot currently remains on both primary and secondary storage.

* update snapshot info depending on the data store role
2025-08-22 00:53:55 +05:30
Suresh Kumar Anaparti
e0bc8c3b1a
Merge branch '4.20' 2025-08-21 18:35:34 +05:30
Pearl Dsilva
6e59f4f4cc
Fix deployment of CKS clusters in Basic zone (#11457) 2025-08-21 18:32:01 +05:30
João Jandre
f2d635683b
Update error message when no snapshot strategy is found while creating snapshot (#11455) 2025-08-18 14:14:20 +05:30
Suresh Kumar Anaparti
f671461d4c
Fix for create template from snapshot (for snapshots on primary storage and storage doesn't support create snapshot to template directly) (#11452)
* Fix for create template from snapshot

* code improvements, for create volume from snapshot
2025-08-15 22:17:03 +05:30
Suresh Kumar Anaparti
2c34f5e495
Merge branch '4.20' 2025-08-15 19:54:41 +05:30
João Jandre
c6daeb4f78
fix snapshot physical size for primary storage (#11448) 2025-08-15 19:52:50 +05:30
Wei Zhou
03da8cbd3e
server: fix conserve_mode of DefaultIsolatedNetworkOfferingForVpcNetworks (#11133)
* server: fix conserve_mode of DefaultIsolatedNetworkOfferingForVpcNetworks

* test: update test_network_acl.py to test conserve mode

* Update test/integration/smoke/test_network_acl.py

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-08-15 17:58:28 +05:30
Harikrishna
0b381d4270
Exclude External hypervisor type during upgrade for System VM template checks (#11435)
* Exclude External hypervisor type during upgrade for System VM template checks
2025-08-15 16:28:56 +05:30
Abhisar Sinha
9fd2b90b80
Fix ConfigurationVO load exception after schema change (#10485) 2025-08-11 21:20:22 +05:30
Suresh Kumar Anaparti
4c3f29de1e
Agent manager connection handling improvements (#11376)
* Agent manager connection handling improvements

* Fix to send LB check interval in ready command
2025-08-05 15:07:02 +05:30
Suresh Kumar Anaparti
15740136dc
Merge branch '4.20' 2025-08-04 18:13:29 +05:30
Harikrishna
53bc435bdb
Fix create statement for safer upgrades (#11388) 2025-08-04 18:11:47 +05:30
slavkap
e5f61164b3
Support of snapshot copy to primary storage in different zones. (#9478)
* Support of snapshot copy to different StorPool primary storage between zones
2025-08-04 16:35:16 +05:30