1327 Commits

Author SHA1 Message Date
Vishesh
bcd738caa6
Fix GPU discovery script to make it run with mdev for SR-IOV enabled devices (#11340) 2025-07-31 18:29:35 +05:30
João Jandre
5ea1ada59a
Allow full clone volumes with thin provisioning in KVM (#11177)
It adds a configuration called create.full.clone to the agent.properties file. When set to true, all QCOW2 volumes created will be full-clone. If false (default), the current behavior remains, where only FAT and SPARSE volumes are full-clone and THIN volumes are linked-clone.
2025-07-31 16:12:17 +05:30
Abhisar Sinha
a87c5c2b3a
Create new Instance from VM backup (#10140)
This feature adds the ability to create a new instance from a VM backup for dummy, NAS and Veeam backup providers. It works even if the original instance used to create the backup was expunged or unmanaged. There are two parts to this functionality:
Saving all configuration details that the VM had at the time of taking the backup. And using them to create an instance from backup.
Enabling a user to expunge/unmanage an instance that has backups.
2025-07-31 15:47:22 +05:30
Suresh Kumar Anaparti
70468a6258
Merge branch '4.20' 2025-07-30 18:30:07 +05:30
Rohit Yadav
2d025bd074
kvm: fix regression 5a52ca78ae5e165211c618525613c3d62cfd1b28 (#11342)
Somehow the commit 5a52ca78ae5e165211c618525613c3d62cfd1b28 was reverted
so cloud-init templates don't work on arm64 anymore :(

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2025-07-30 17:03:10 +05:30
Suresh Kumar Anaparti
4d2beea777
logger fix 2025-07-30 14:50:41 +05:30
Suresh Kumar Anaparti
76cfcb48fb
Merge branch '4.20' 2025-07-30 11:21:12 +05:30
Suresh Kumar Anaparti
96ed9d47bc
Merge branch '4.19' into 4.20 2025-07-30 11:10:17 +05:30
Abhishek Kumar
7506a547af
kvm, ui: fix interface when using vlan subnet for storage traffic type (#11245)
* kvm, ui: fix interface when using vlan subnet for storage traffic type

Fixes #7816

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-07-30 11:04:58 +05:30
Daan Hoogland
0b3959221b Merge branch '4.20' 2025-07-29 16:50:55 +02:00
Vishesh
f6ad184ea2
Feature: Add support for GPU with KVM hosts (#11143)
This PR allows attaching of GPU devices via PCI, mdev or VF to an Instance for KVM.

It allows the operator to discover the GPU devices on the KVM host and create a Compute Offering with GPU support based on the available GPU devices on the host. Once the operator has created the Compute offering, it can be used by users to launch Instances with GPU devices.
2025-07-29 13:46:24 +05:30
Daan Hoogland
609efcc231 Merge branch '4.19' into 4.20 2025-07-25 22:01:17 +02:00
Pearl Dsilva
0d4147f3f6
Netris Network Plugin Integration with CloudStack (#10458)
The Netris Plugin introduces Netris as a network service provider in CloudStack to be able to create and manage Virtual Private Clouds (VPCs) in CloudStack, being able to orchestrate the following network functionalities:

- Network segmentation with Netris-VXLAN isolation method
- Routing between "public" IP and network segments with an ACS ROUTED mode offering
- SourceNAT, DNAT, 1:1 NAT between "public" IP and network segments with an ACS NATTED mode offering
- Routing between VPC network segments (tiers in ACS nomenclature)
- Access Lists (ACLs) between VPC tiers and "public" network (TCP, UDP, ICMP) both as global egress rules and "public" IP specific ingress rules.
- ACLs between VPC network tiers (TCP, UDP, ICMP)
- External load balancing – between VPC network tiers and "public" IP
- Internal load balancing – between VPC network tiers
- CloudStack Virtual Router services (DHCP, DNS, UserData, Password Injection, etc…)
2025-07-25 15:26:42 +05:30
Suresh Kumar Anaparti
75a2b3cc54
Validate qcow2 file during import operation (#11264) 2025-07-25 11:17:14 +02:00
Abhisar Sinha
1b74c2dd3f
Fix restore from NAS backup when datadisk is older than the root disk. (#11258) 2025-07-23 12:45:47 +02:00
Wei Zhou
30deec89e6
kvm: consider Debian same as Ubuntu (#10917)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-07-17 09:56:48 +02:00
Daan Hoogland
b57994eeb0 Merge branch '4.20' 2025-07-16 14:05:19 +02:00
Suresh Kumar Anaparti
c94f75c7ea
PowerFlex/ScaleIO - Wait after SDC service start/restart/stop, and retry to fetch SDC id/guid (#11099)
* [PowerFlex/ScaleIO] Added wait time after SDC service start/restart/stop, and retries to fetch SDC id/guid

* Added agent property 'powerflex.sdc.service.wait' for the time (in secs) to wait after SDC service start/restart/stop

* code improvements
2025-07-16 12:32:09 +05:30
João Jandre
53eb2c5b9b
File-based disk-only VM snapshot with KVM as hypervisor (#10632)
Co-authored-by: João Jandre <joao@scclouds.com.br>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-07-16 08:54:07 +02:00
Suresh Kumar Anaparti
3220eb442a
PowerFlex/ScaleIO - MDM and host SDC connection enhancements (#11047)
* Cumulative enhancements fix for ScaleIO: MDM add/remove, Host prepare/unprepare, validate Storage Pool can be created in Agent.

- Implemented validation to fail Host disconnect from Storage Pool if there are Volumes attached and SDC client MDM removal requires scini service to be restarted
- Implemented Storage Pool validation by checking whether MDM addresses from configuration file and from memory (using CLI) matches, otherwise file ModifyStoragePool command.
- Introduced configuration key to apply timeout after making MDM changes for ScaleIO: powerflex.mdm.change.apply.timeout.ms (default 1000ms)
- Implemented logic to apply timeout after making MDM changes for ScaleIO in prepare and unprepare logic
- Added detection of MDM removal support via CLI
- If MDM removal support via CLI supported then use CLI, fall back to edit drv_cfg.txt and restart scini instead

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: mprokopchuk <mprokopchuk@apple.com>
2025-07-16 08:25:28 +02:00
Daan Hoogland
0dbd761fbb Merge branch '4.20' 2025-07-14 11:17:08 +02:00
Daan Hoogland
61e74e086b Merge branch '4.19' into 4.20 2025-07-14 10:37:06 +02:00
Wei Zhou
e47b78b2bb
directdownload: fix keytool importcert (#11113)
* directdownload: fix keytool importcert

```
$ /usr/bin/keytool -importcert file /etc/cloudstack/agent/CSCERTIFICATE-full -keystore /etc/cloudstack/agent/cloud.jks -alias full -storepass DAWsfkJeeGrmhta6
Illegal option:  file
keytool -importcert [OPTION]...

Imports a certificate or a certificate chain

Options:

 -noprompt               do not prompt
 -trustcacerts           trust certificates from cacerts
 -protected              password through protected mechanism
 -alias <alias>          alias name of the entry to process
 -file <file>            input file name
 -keypass <arg>          key password
 -keystore <keystore>    keystore name
 -cacerts                access the cacerts keystore
 -storepass <arg>        keystore password
 -storetype <type>       keystore type
 -providername <name>    provider name
 -addprovider <name>     add security provider by name (e.g. SunPKCS11)
   [-providerarg <arg>]    configure argument for -addprovider
 -providerclass <class>  add security provider by fully-qualified class name
   [-providerarg <arg>]    configure argument for -providerclass
 -providerpath <list>    provider classpath
 -v                      verbose output

Use "keytool -?, -h, or --help" for this help message
```

* DirectDownload: drop HttpsMultiTrustManager
2025-07-07 13:36:16 +02:00
Suresh Kumar Anaparti
80f46ad55d
[VMware to KVM Migration] Fix for converted instance npe issue when source vmware instance ovf is exported from management server (#11003) 2025-07-04 13:54:54 +05:30
Daan Hoogland
3e3a0c0678 Merge branch '4.20' 2025-07-03 15:29:05 +02:00
Nicolas Vazquez
16c60c7528
Support Direct Download on Ceph (#11069) 2025-06-24 14:19:01 +02:00
Nicolas Vazquez
75147b7811
[Vmware to KVM Migration] Display virt-v2v and ovftool versions for supported hosts for migration (#11019)
* [Vmware to KVM Migration] Display virt-v2v and ovftool versions for supported hosts for migration

* Fix UI display

* Address review comments

* Fix ovftool and version display - also display versions on host details view
2025-06-23 12:49:51 +02:00
João Jandre
7f13beb36a
Check Qcow2 version before using --bitmaps (#10896)
* check version before using --bitmaps

* use cloudruntimeexception
2025-06-13 20:30:46 +02:00
Pearl Dsilva
379ee07d88 Updating pom.xml version numbers for release 4.19.4.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 18:00:09 +05:30
Pearl Dsilva
b5e2c181f9 Updating pom.xml version numbers for release 4.20.2.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 15:38:12 +05:30
Pearl Dsilva
c61a5eb430 Updating pom.xml version numbers for release 4.20.1.0
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-05-30 12:43:00 +05:30
Daan Hoogland
0c7d47138d Updating pom.xml version numbers for release 4.19.3.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2025-05-30 09:08:58 +02:00
Wei Zhou
96b8e23767
kvm: fix vm deployment from RAW template (#10880)
* kvm: fix vm deployment from RAW template

* Update plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java

Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>

---------

Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2025-05-19 21:44:01 +02:00
Wei Zhou
842b2f8c24
Merge remote-tracking branch 'apache/4.20' 2025-05-19 21:25:37 +02:00
Suresh Kumar Anaparti
f199783c75
VMware import - logs sanitation (#10433) 2025-05-16 09:21:19 +02:00
Suresh Kumar Anaparti
572fc11a64
[PowerFlex] Add & Remove PowerFlex/ScaleIO MDMs for the storage SDC connections (#9903)
* Add & Remove PowerFlex/ScaleIO MDMs while preparing & unpreparing the storage SDC connections (instead of start & stop scini)

* Add/Remove MDM IP addresses during Host connection/disconnection to/from storage pool when powerflex.connect.on.demand is false

* unit test fixes

* Don't remove MDM IPs from SDC when any volumes mapped to SDC

* Don't remove MDM IPs when other pools of same ScaleIO/PowerFlex cluster are connected

* rebase fixes

* update changes, to not remove/disconnect MDMs on maintenance

* import fixes after rebase
2025-05-15 12:42:13 +05:30
Abhisar Sinha
d55aa70f7e
Restore single backed-up volume on a live instance attaches the volume as a Raw image making it unreadable (#10844) 2025-05-14 15:23:07 +02:00
João Jandre
6fdaf51ddc
KVM incremental snapshot feature (#9270)
* KVM incremental snapshot feature

* fix log

* fix merge issues

* fix creation of folder

* fix snapshot update

* Check for hypervisor type during parent search

* fix some small bugs

* fix tests

* Address reviews

* do not remove storPool snapshots

* add support for downloading diff snaps

* Add multiple zones support

* make copied snapshots have normal names

* address reviews

* Fix in progress

* continue fix

* Fix bulk delete

* change log to trace

* Start fix on multiple secondary storages for a single zone

* Fix multiple secondary storages for a single zone

* Fix tests

* fix log

* remove bitmaps when deleting snapshots

* minor fixes

* update sql to new file

* Fix merge issues

* Create new snap chain when changing configuration

* add verification

* Fix snapshot operation selector

* fix bitmap removal

* fix chain on different storages

* address reviews

* fix small issue

* fix test

---------

Co-authored-by: João Jandre <joao@scclouds.com.br>
2025-05-12 10:50:30 -03:00
Pearl Dsilva
1e5d133033 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-05-12 13:12:09 +05:30
Pearl Dsilva
04731b0618 fix logger 2025-05-12 12:58:41 +05:30
Pearl Dsilva
a21f912be3 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-05-12 12:41:34 +05:30
Wei Zhou
ea32a1a71a
server: fetch IP of VMs on L2 networks (#10431) 2025-05-09 10:19:30 +02:00
Abhisar Sinha
2674da2ee9
Nas BnR: Fix for restore not working correctly (#10785) 2025-05-08 09:20:06 +02:00
Wei Zhou
fd74895ad0
New feature: Reconcile commands (CopyCommand, MigrateCommand, MigrateVolumeCommand) (#10514) 2025-05-02 09:15:03 +02:00
Pearl Dsilva
2df1ac5106 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-28 12:15:48 +05:30
Wei Zhou
5d28e66b52
vTPM: support KVM and VMware (#10543)
* KVM: add Virtual TPM model and version

* KVM: add admin-only VM setting GUEST.CPU.MODE and GUEST.CPU.MODEL

* VMware: add vTPM

* vTPM: do not set Key due to 'Cannot add multiple devices using the same device key..'

* vTPM: add unit test testTpmModel

* engine/schema: remove user vm details for guest CPU mode/model

* vTPM: extra methods as Daan's requests

* vTPM: add unit tests in VmwareResourceTest

* vTPM: update unit tests in VmwareResourceTest

* vTPM: add unit test in LibvirtComputingResourceTest

* vTPM: use the default TPM version if an invalid version is passed

* vTPM: requires UEFI on vmware and do nothing if it is not enabled/disabled

* vTPM: let uses to add UEFI on vmware

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* vTPM: remove template details for guest CPU mode/model

* UI: boot vm from ISO into UEFI/SECURE mode

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-04-26 14:35:36 +05:30
Eric Kalendra
55c811547a
Update Mockito to 5.16.1 (#10686)
Dependency name change mockito-inline to mockito-core. Inline is now the default and the last version of mockito-inline released is 5.2.0.

assertj-core in user-authenticators/saml2 pulls in an incompatible version of byte-buddy and required an exclusion. Updating the version of assertj is left for a future PR.
The upgrade requires Java 11+, dropping support for Java 8. CloudStack documentation already says to use Java 11 and does not indicate that java 8 is supported.

Test classes using @RunWith(MockitoJUnitRunner.class) now get run in strict mode. Changes were made to tests where the stubbing intention was clear. In ManagementServerMaintenanceManagerImplTest there are 5 tests where the intention of the test is unclear. Each of the statements now use Mockito.lenient() to avoid the exception. Other cases in the tests follow a similar pattern.
Minor clean up.

Both @Spy and Mockito.spy( should not be used. Favored the annotation.
Both @RunWith(MockitoJUnitRunner.class) and MockitoAnnotations.openMocks(this); should not be used. Favored the annotation.
Unnecessary extends TestCase removed.
@InjectMocks and new in statement unnecessary. Removed new when issue presented.
Some of the Cmd classes like UpdateNetworkCmd have a type tree that includes fields of type Object. This appears to cause issues with injection, requiring that @Mock fields be available. This is where the following fields were added in multiple places:
Object job;
ResponseGenerator _responseGenerator;
Wrong number of parameters for Mockito.when in LibvirtRevertSnapshotCommandWrapperTest.java
2025-04-16 18:10:28 +05:30
Bryan Lima
cb4848bc1a
Add support to RBD erasure code pools (#9808)
* Readd filename string on qemuimg create

* Remove empty object on the data pool details of storage pools with no data pool

* Only use the method createPhysicalDiskByLibVirt with RBD when the pool is of erasure code type. Also added javadoc for createPhysicalDisk method

* Change literal '/' string to File.separator

* Add support for erasure code pools

* Fix null on putAll
2025-04-02 08:19:00 -03:00
Pearl Dsilva
b387bc1664 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-03-10 09:34:52 -04:00
Pearl Dsilva
0427abf855 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-03-10 09:34:24 -04:00