4846 Commits

Author SHA1 Message Date
Suresh Kumar Anaparti
572fc11a64
[PowerFlex] Add & Remove PowerFlex/ScaleIO MDMs for the storage SDC connections (#9903)
* Add & Remove PowerFlex/ScaleIO MDMs while preparing & unpreparing the storage SDC connections (instead of start & stop scini)

* Add/Remove MDM IP addresses during Host connection/disconnection to/from storage pool when powerflex.connect.on.demand is false

* unit test fixes

* Don't remove MDM IPs from SDC when any volumes mapped to SDC

* Don't remove MDM IPs when other pools of same ScaleIO/PowerFlex cluster are connected

* rebase fixes

* update changes, to not remove/disconnect MDMs on maintenance

* import fixes after rebase
2025-05-15 12:42:13 +05:30
Daan Hoogland
64828f66e8 Merge branch '4.20' 2025-05-13 13:34:23 +02:00
Pearl Dsilva
0648d000b2
Support XenServer 8.4 / XCP 8.3 - make scripts python3 compatible (#10684) 2025-05-13 12:35:04 +02:00
Daan Hoogland
dd84c74e82 Merge branch '4.19' into 4.20 2025-05-13 11:41:36 +02:00
Rene Peinthor
88ce639255
Linstor: implement volume and storage stats (#10850) 2025-05-13 10:06:35 +02:00
João Jandre
6fdaf51ddc
KVM incremental snapshot feature (#9270)
* KVM incremental snapshot feature

* fix log

* fix merge issues

* fix creation of folder

* fix snapshot update

* Check for hypervisor type during parent search

* fix some small bugs

* fix tests

* Address reviews

* do not remove storPool snapshots

* add support for downloading diff snaps

* Add multiple zones support

* make copied snapshots have normal names

* address reviews

* Fix in progress

* continue fix

* Fix bulk delete

* change log to trace

* Start fix on multiple secondary storages for a single zone

* Fix multiple secondary storages for a single zone

* Fix tests

* fix log

* remove bitmaps when deleting snapshots

* minor fixes

* update sql to new file

* Fix merge issues

* Create new snap chain when changing configuration

* add verification

* Fix snapshot operation selector

* fix bitmap removal

* fix chain on different storages

* address reviews

* fix small issue

* fix test

---------

Co-authored-by: João Jandre <joao@scclouds.com.br>
2025-05-12 10:50:30 -03:00
Suresh Kumar Anaparti
47a268202b
[VMware] Sync the disk path or datastore changes for IDE disks, and before any volume resize during start vm (for the volumes on datastore cluster) (#10748) 2025-05-12 13:21:04 +02:00
Pearl Dsilva
1e5d133033 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-05-12 13:12:09 +05:30
Pearl Dsilva
04731b0618 fix logger 2025-05-12 12:58:41 +05:30
Pearl Dsilva
a21f912be3 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-05-12 12:41:34 +05:30
Abhishek Kumar
e68abcd85d
xenserver: destroy halted vm on expunge (#10833)
* xenserver: destroy halted vm on expunge

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-12 10:46:37 +05:30
slavkap
17e062a381
StorPool notify libvirt when volume is resized (#10775) 2025-05-09 11:34:52 +02:00
Wei Zhou
ea32a1a71a
server: fetch IP of VMs on L2 networks (#10431) 2025-05-09 10:19:30 +02:00
Abhisar Sinha
2674da2ee9
Nas BnR: Fix for restore not working correctly (#10785) 2025-05-08 09:20:06 +02:00
Wei Zhou
fd74895ad0
New feature: Reconcile commands (CopyCommand, MigrateCommand, MigrateVolumeCommand) (#10514) 2025-05-02 09:15:03 +02:00
Daan Hoogland
d7d9d131b2 Merge branch '4.20' 2025-05-01 15:44:09 +02:00
Abhisar Sinha
dfd64b1a67
Ceph object store: Fix LocationConstraint error (#10772)
* Don't set signingRegion as auto for creating the s3 client in ceph object store provider.

* replace getBucketAcl with doesBucketExistV2 in CephObjectStoreDriverImplTest
2025-05-01 11:47:18 +05:30
Pearl Dsilva
0e0ae226bd
Xenserver smoke-test: Allow emojis to be accepted in volume name during volume creation (#10774)
* Allow emojis to be accepted in volume name during volume creation

* escape only non-ASCII characters
2025-04-29 11:43:07 +05:30
Pearl Dsilva
2df1ac5106 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-28 12:15:48 +05:30
Pearl Dsilva
0785ba046e Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-04-28 11:10:08 +05:30
Wei Zhou
5d28e66b52
vTPM: support KVM and VMware (#10543)
* KVM: add Virtual TPM model and version

* KVM: add admin-only VM setting GUEST.CPU.MODE and GUEST.CPU.MODEL

* VMware: add vTPM

* vTPM: do not set Key due to 'Cannot add multiple devices using the same device key..'

* vTPM: add unit test testTpmModel

* engine/schema: remove user vm details for guest CPU mode/model

* vTPM: extra methods as Daan's requests

* vTPM: add unit tests in VmwareResourceTest

* vTPM: update unit tests in VmwareResourceTest

* vTPM: add unit test in LibvirtComputingResourceTest

* vTPM: use the default TPM version if an invalid version is passed

* vTPM: requires UEFI on vmware and do nothing if it is not enabled/disabled

* vTPM: let uses to add UEFI on vmware

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* vTPM: remove template details for guest CPU mode/model

* UI: boot vm from ISO into UEFI/SECURE mode

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-04-26 14:35:36 +05:30
Abhishek Kumar
12c077d704
api,ui: multi arch improvements (#10289) 2025-04-25 11:02:27 +02:00
dahn
5a5c865133
Revert "Add the option to filter by host when retrieving of unregistered VMs (#9925)" (#10647)
This reverts commit aa6c581e405d7652916a9290e042f0bd26c3913a.
2025-04-24 09:06:57 +02:00
Pearl Dsilva
576b97ba22 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-24 09:22:40 +05:30
slavkap
f6f33c6add
Fix the size of a template downloaded from secondary storage (#10662)
Fixing the size of a template that is downloaded from secondary storage
to StorPool
2025-04-23 16:07:47 +05:30
Pearl Dsilva
ffc0d51616 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-22 03:31:44 -04:00
Pearl Dsilva
0da243d660
Fix issue with allocator not considering subsequent clusters (#10603)
* Fix issue with allocator not considering subsequent clusters

* add missing return

* remove unused import

* update as per inputs
2025-04-22 03:06:12 -04:00
Daan Hoogland
2771586b87 Merge release branch 4.19 to 4.20
* 4.19:
  plugin/shutdown: use mgmt server uuid in the shutdown response (#10717)
2025-04-18 11:54:03 +02:00
Rohit Yadav
29e58e906f
plugin/shutdown: use mgmt server uuid in the shutdown response (#10717) 2025-04-18 11:08:53 +02:00
Daan Hoogland
3c75d9363b Merge branch '4.20' 2025-04-17 15:59:41 +02:00
Daan Hoogland
d7765343ef Merge branch '4.19' into 4.20 2025-04-17 15:40:10 +02:00
Eric Kalendra
55c811547a
Update Mockito to 5.16.1 (#10686)
Dependency name change mockito-inline to mockito-core. Inline is now the default and the last version of mockito-inline released is 5.2.0.

assertj-core in user-authenticators/saml2 pulls in an incompatible version of byte-buddy and required an exclusion. Updating the version of assertj is left for a future PR.
The upgrade requires Java 11+, dropping support for Java 8. CloudStack documentation already says to use Java 11 and does not indicate that java 8 is supported.

Test classes using @RunWith(MockitoJUnitRunner.class) now get run in strict mode. Changes were made to tests where the stubbing intention was clear. In ManagementServerMaintenanceManagerImplTest there are 5 tests where the intention of the test is unclear. Each of the statements now use Mockito.lenient() to avoid the exception. Other cases in the tests follow a similar pattern.
Minor clean up.

Both @Spy and Mockito.spy( should not be used. Favored the annotation.
Both @RunWith(MockitoJUnitRunner.class) and MockitoAnnotations.openMocks(this); should not be used. Favored the annotation.
Unnecessary extends TestCase removed.
@InjectMocks and new in statement unnecessary. Removed new when issue presented.
Some of the Cmd classes like UpdateNetworkCmd have a type tree that includes fields of type Object. This appears to cause issues with injection, requiring that @Mock fields be available. This is where the following fields were added in multiple places:
Object job;
ResponseGenerator _responseGenerator;
Wrong number of parameters for Mockito.when in LibvirtRevertSnapshotCommandWrapperTest.java
2025-04-16 18:10:28 +05:30
Harikrishna
a09354ddf5
Fix volume migration failure response (#10707) 2025-04-16 10:58:06 +02:00
Rene Glover
f13cf597a2
4.19 fix saml account selector (#10311) 2025-04-14 12:59:43 +02:00
Dahn Highland
4c31f9d533 Merge release branch 4.20 to main
* 4.20:
  xenserver: do not destroy halted hypervisor vm (#9175)
  define the limit of projects through the UI (#10652)
  fix projects metrics on dashboard (#10651)
  systemvm: Bump systemvm template version to debian 12.10 (#10628)
  Enhance VPC Network Tier form to auto-populate  Gateway, and Netmask (#10617)
2025-04-09 13:02:31 +02:00
Dahn Highland
6850147c9e Merge branch '4.19' into 4.20 2025-04-09 13:00:25 +02:00
Abhishek Kumar
5d5ac17c68
xenserver: do not destroy halted hypervisor vm (#9175)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-04-09 09:33:01 +02:00
John Bampton
f206137f83
docs: fixes grammar and spelling in Markdown files only (#10656) 2025-04-08 12:44:14 +02:00
Bryan Lima
cb4848bc1a
Add support to RBD erasure code pools (#9808)
* Readd filename string on qemuimg create

* Remove empty object on the data pool details of storage pools with no data pool

* Only use the method createPhysicalDiskByLibVirt with RBD when the pool is of erasure code type. Also added javadoc for createPhysicalDisk method

* Change literal '/' string to File.separator

* Add support for erasure code pools

* Fix null on putAll
2025-04-02 08:19:00 -03:00
Daan Hoogland
8af021c6f6 Merge branch '4.20' 2025-03-27 17:03:13 +01:00
Daan Hoogland
5f93ce71bb Merge branch '4.19' into 4.20 2025-03-27 16:44:42 +01:00
Phsm Qwerty
0b7aef7a0f
enhancement: iterate directly on HostJoinVO (#10613) 2025-03-27 10:17:28 +01:00
Suresh Kumar Anaparti
9dceae4614
MS maintenance improvements (#10417)
* Update last agents during ms maintenance, and some code improvements

* Send 503 (Service Unavailable) response status when maintenance or shutdown is initiated
[Any load balancer in the clustered environment can avoid routing requests to this MS node]

* Migrate systemvm agents before routing host agents, and some code improvements

* Added events for ms maintenance and shutdown operations

* Added the following ms maintenance and shutdown improvements

- block new agent connections during prepare for maintenance of ms

- maintain avoids ms list

- propagate updated management servers list and lb algorithm in host and indirect.agent.lb.algorithm settings respectively, to systemvm (non-routing) agents

- updated setup ms list and migrate agent connections to executor service

- migrate agent connection through executor, and send the answer to the ms host that initiated the migration

- re-initialize ssl handshake executor if it is shutdown

- don't allow prepare for maintenance or shutdown when other management server nodes are in preparing states

- don't allow trigger shutdown when management server is up and other management server nodes are in preparing states

- stop agent connections monitor on ms maintenance

- update avoid ms list in ready command

- updated connected host from the client connection

- update last agents in ms metrics from the database

- updated some agent config descriptions

- update last management server in the hosts during shutdown

- added agents and lastagents in management server response

- updated management server maintenance & shutdown unit tests

- some code improvements

* refactored code / addressed comments

* removed shutdown testcase (maybe, calling System.exit)

* Revert "removed shutdown testcase (maybe, calling System.exit)"

This reverts commit e14b0717152ef6c8be102d61c80f42803a53172e.

* avoid system.exit during shutdown test

* code improvements

* testcase fix

* Fix cutoff time in agent connections monitor thread
2025-03-19 14:18:05 +05:30
Rene Peinthor
f4a7c8ab89
linstor: implement missing deleteDatastore (#10561)
Somehow deleteDatastore was never implemented, that meant:
templates haven't been cleaned up on datastore delete and
also agents have never been informed about storage pool removal.
2025-03-18 08:50:19 -04:00
Pearl Dsilva
1732f676db Merge branch '4.20' of https://github.com/apache/cloudstack 2025-03-17 13:26:50 -04:00
Pearl Dsilva
e091aad792 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-03-17 13:26:26 -04:00
Abhisar Sinha
35e809e7ce
Set external Id to null after backupProvider.removeVMFromBackup (#10562) 2025-03-17 12:46:05 -04:00
Daan Hoogland
9c6f2a9e14 Merge release branch 4.20 to main
* 4.20:
  Fix Stats Collector to not divide by zero (#10492)
  linstor: try to delete -rst resource before snapshot backup (#10443)
2025-03-12 11:31:56 +01:00
Daan Hoogland
f8adedc280 Merge release branch 4.19 to 4.20
* 4.19:
  linstor: try to delete -rst resource before snapshot backup (#10443)
2025-03-12 11:31:16 +01:00
Rene Peinthor
95c24810ab
linstor: try to delete -rst resource before snapshot backup (#10443)
If a -rst resource wasn't deleted because of a failed copy,
a reoccurring snapshot attempt couldn't be done, because there
was still the "old" -rst resource. To prevent this always
try to remove the -rst resource before, if it doesn't exist it is a noop.
2025-03-10 16:23:01 +01:00