1253 Commits

Author SHA1 Message Date
Daan Hoogland
4f3e8e8c5a Merge branch '4.19' into 4.20 2025-02-12 15:00:51 +01:00
Nicolas Vazquez
4f604c00b6
Support virtio-blk root disk controller (#10229) 2025-02-10 15:03:07 +01:00
Rene Glover
3337f425ff
Primera pure patches & various small fixes (#10132)
Co-authored-by: GLOVER RENE <rg9975@cs419-mgmtserver.rg9975nprd.app.ecp.att.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-02-07 13:19:34 +01:00
Rene Peinthor
55e8eaab89
Linstor: encryption support (#10126)
This introduces a new encryption mode, instead of a simple bool.
Now also storage driver can just provide encrypted volumes to CloudStack.
2025-02-04 15:18:49 +01:00
Daan Hoogland
34d2a3bc86 Merge branch '4.19' into 4.20 2025-01-24 17:01:42 +01:00
Nicolas Vazquez
7e295ec4e1
[KVM] Add watchdog model none to disable use of watchdogs on KVM agent (#10203) 2025-01-23 13:15:02 +01:00
Harikrishna
20759187b3
Fix local storage deletion cases (#10231)
* Delete local storage properties in agent.properties during delete pool

* Fix stale entry when add local storage failed

* Smaller methods

* Comment added
2025-01-23 12:46:33 +05:30
Wei Zhou
b186272f68
kvm: add SCSI controllers based on the number of virtio-SCSI disks (#9823)
According to libvirt code, the units per scsi controller is set to 7
therefore, we need to create scsi controller every 7 disks (including CDROM).

50cc7a0d9d/src/conf/domain_conf.h (L3007-L3008)

50cc7a0d9d/src/conf/domain_conf.c (L6701-L6704)
2025-01-22 14:00:02 +01:00
Abhishek Kumar
bd488c4bba
server, plugin: enhance storage stats for IOPS (#10034)
Adds framework layer change to allow retrieving and storing IOPS stats for storage pools. Custom `PrimaryStoreDriver` can implement method - `getStorageIopsStats` for returning IOPS stats. Existing method `getUsedIops` can also be overridden by such plugins when only used IOPS is returned.
For testing purpose, implementation has been added for simulator hypervisor plugin to return capacity and used IOPS for a pool.
For local storage pool, implementation has been added using iostat to return currently used IOPS.
StoragePoolResponse class has been updated to return IOPS values which allows showing IOPS values in UI for different storage pool related views and APIs.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-07 17:17:12 +05:30
Vishesh
a4224e58cc
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin

* Update logging for scaleio plugin

* Improve logging to include more identifiable information for default volume storage plugin

* Improve logging to include more identifiable information for agent managers

* Improve logging to include more identifiable information for Listeners

* Replace ids with objects or uuids


* Improve logging to include more identifiable information for engine

* Improve logging to include more identifiable information for server

* Fixups in engine

* Improve logging to include more identifiable information for plugins

* Improve logging to include more identifiable information for Cmd classes

* Fix toString method for StorageFilterTO.java
2025-01-06 16:42:37 +05:30
Daan Hoogland
b7f0aac519 Merge branch '4.19' into 4.20 2024-12-20 14:34:39 +01:00
Suresh Kumar Anaparti
188eacd9eb
Certificate and VM hostname validation improvements (#10051)
* Certificate and VM hostname validation improvements

* Improve certificate name validation and some code/log improvements
2024-12-20 10:18:11 +05:30
Rene Peinthor
a9587bfd2e
kvm-storage: provide isVMMigrate information to storage plugins (#10093)
Particular Linstor needs can use this information to only allow
dual volume access for live migration and not enable it in general,
which can and will lead to data corruption if for some reason
2 VMs get started on 2 different hosts.
2024-12-18 09:13:41 +01:00
Suresh Kumar Anaparti
b4ad04badf
Allow config drive deletion of migrated VM, on host maintenance (#10045) 2024-12-18 09:12:28 +01:00
Daan Hoogland
da54234585 Merge branch '4.19' into 4.20.merge 2024-12-03 16:32:15 +01:00
Vishesh
37eec06166
Allow VMWare import via another host (#9787)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-11-28 17:50:08 +01:00
Rene Peinthor
d54b105a03
Linstor: add support for ISO block devices and direct download (#9792) 2024-11-28 17:47:47 +01:00
João Jandre
c63c7ee63e Updating pom.xml version numbers for release 4.20.1.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:40:45 -03:00
João Jandre
2fe3fcef7c Updating pom.xml version numbers for release 4.20.0.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-19 08:54:07 -03:00
Daan Hoogland
a82a2429f4 Merge branch '4.19' 2024-11-14 15:59:35 +01:00
Daan Hoogland
a2690e9864 imports 2024-11-14 15:28:32 +01:00
Daan Hoogland
24c28ad460 Merge branch '4.18' into 4.19 2024-11-14 14:30:06 +01:00
Daan Hoogland
6ef2eaca14 Merge tag '4.18.2.5' into 4.18 2024-11-14 13:59:25 +01:00
dahn
b75742da43
enum name changed for logger update reverted (#9922) 2024-11-14 08:38:42 -03:00
João Jandre
b38ee63c48 Merge branch '4.19' 2024-11-13 10:47:24 -03:00
Rene Peinthor
dfe4a67859
kvm: ref-count secondary storage pool usage (#9498)
If a secondary storage pool is used by e.g.
2 concurrent snapshot->template actions,
if the first action finished it removed the netfs mount
point for the other action.
Now the storage pools are usage ref-counted and will only
deleted if there are no more users.
2024-11-13 10:32:46 -03:00
Daan Hoogland
f7b7339bcf Merge branch '4.19' 2024-11-07 11:33:50 +01:00
Wei Zhou
006db90f75
kvm: fix ovs network creation issue (#9869)
This fixes the issue when create a ovs network
```
2024-10-29 16:02:45,089 WARN  [resource.wrapper.LibvirtOvsFetchInterfaceCommandWrapper] (agentRequest-Handler-2:null) (logid:e716722e) Network interface: ''cloudbr1'' not found
```

This is a regression of a previous security release
see "framework/cluster: improve cluster service, integration API server"

since we now use NetworkInterface.getByName to get network interface, we should NOT add single quotes before/after the label.
2024-11-07 14:36:53 +05:30
Daniel Augusto Veronezi Salvador
84538da27c Updating pom.xml version numbers for release 4.18.2.5
Signed-off-by: Daniel Augusto Veronezi Salvador <gutoveronezi@apache.org>
2024-11-05 00:54:55 -03:00
Daniel Augusto Veronezi Salvador
966b75d0b9 Verify QCOW2 features on direct download of template 2024-11-05 00:26:19 -03:00
João Jandre
8af08ddafb Merge branch '4.19' 2024-11-04 08:58:48 -03:00
Rene Peinthor
a6cef7a78d
linstor/kvm: Workaround a qemu bug and IDE bus discard enabled. (#9859)
qemu has a bug versions prior 7.0 with discard enabled and using the IDE bus.
It would crash the qemu process and kill the virtual machine,
this is most noticeable on installing a windows guest from the
Windows ISO installer.
2024-11-04 08:46:40 -03:00
Rohit Yadav
20901c7443 Merge remote-tracking branch 'origin/4.19' 2024-10-17 14:53:26 +05:30
Rene Peinthor
7fbd0a9fb4
linstor: enable discard for Linstor storage pools (#9770)
* linstor: enable discard for Linstor storage pools

All Linstor storage backends support discard, so it can be safely enabled.

* linstor: enable discard for Linstor storage pools CHANGELOG.md
2024-10-17 14:51:47 +05:30
Daan Hoogland
dbfc7f23a7 Merge branch '4.19' 2024-10-11 17:59:46 +02:00
Daan Hoogland
54b3519df1 Updating pom.xml version numbers for release 4.18.2.4
Signed-off-by: Daan Hoogland <daan@onecht.net>
2024-10-03 17:36:32 +02:00
João Jandre
e666dca403
linked clone for file based storage (#8911) 2024-10-02 13:15:29 -03:00
Vishesh
21d107c349
Merge branch '4.19' 2024-09-24 14:04:51 +05:30
Wei Zhou
9ce7ef49cf
build/packaging: build tungsten plugin only if noredist is passed (#9006) 2024-09-20 10:17:12 +02:00
mprokopchuk
1d4700a0bd
Provide encryption key for DATA volume type (in addition to ROOT) to copy volume. (#9663) 2024-09-12 13:03:23 -03:00
Daan Hoogland
abfa92928c merge conflicts 4.19 -> main 2024-09-09 14:48:20 +02:00
Rene Peinthor
3f5a77ef58
Linstor: Fix migrate primary storage (#9528) 2024-09-09 10:01:41 +02:00
Fabricio Duarte
b11897cdfa
Add logs to LibvirtComputingResource's metrics collection process (#8511)
* Add logs to LibvirtComputingResource's metrics collecting process

* Apply Joao's suggestions

Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>

* Adjust some logs

* Print memory statistics log in one line

---------

Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-09-06 10:49:22 -03:00
Nicolas Vazquez
8c8d115a1e
feature: Support Multi-arch Zones (#9619)
This introduces the multi-arch zones, allowing users to select the VM arch upon deployment. 

Multi-arch zone support in CloudStack can allow admins to mix x86_64 & arm64 hosts within the same zone with the following changes proposed:
- All hosts in a clusters need to be homogenous, wrt host CPU type (amd64 vs arm64) and hypevisor
- Arch-aware templates & ISOs:
   -  Add support for a new arch field (default set of: amd64 and arm64), when unspecified defaults to amd64 and for existing templates & iso
   -  Allow admins to edit the arch type of the registered template & iso
- Arch-aware clusters and host:
   - Add new attribute field for cluster and hosts (kvm host agents can automatically report this, arch of the first host of the cluster is cluster's architecture), defaults to amd64 when not specified
   - Allow admins to edit the arch of an existing cluster
- VM deployment form (UI):
   - In a multi-arch zone/env, the VM deployment form can allow some kind of template/iso filtration in the UI
   - Users should be able to select arch: amd64 & arm64; but this is shown only in a multi-arch zone (env)
- VM orchestration and lifecycle operations:
   - Use of VM/template's arch to correctly decide where to provision the VM (on the correct strictly arch-matching host/clusters) & other lifecycle operations (such as migration from/to arch-matching hosts)

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-06 12:14:54 +05:30
Rohit Yadav
85765c3125
backup: simple NAS backup plugin for KVM (#9451)
This is a simple NAS backup plugin for KVM which may be later expanded for other hypervisors. This backup plugin aims to use shared NAS storage on KVM hosts such as NFS (or CephFS and others in future), which is used to backup fully cloned VMs for backup & restore operations. This may NOT be as efficient and performant as some of the other B&R providers, but maybe useful for some KVM environments who are okay to only have full-instance backups and limited functionality.

Design & Implementation follows the `networker` B&R plugin, which is simply:

- Implement B&R plugin interfaces
- Use cmd-answer pattern to execute backup and restore operations on KVM host when VM is running (or needs to be restored) - instead of a B&R API client, relies on answers from KVM agent which executes the operations
- Backups are full VM domain snapshots, copied to a VM-specific folders on a NAS target (NFS) along with a domain XML
- Backup uses libvirt feature: https://libvirt.org/kbase/live_full_disk_backup.html orchestrated via virsh/bash script (nasbackup.sh) as the libvirt-java lacks the bindings
- Supported instance volume storage for restore operations: NFS & local storage

Refer the doc PR for feature limitations and usage details:
https://github.com/apache/cloudstack-documentation/pull/429

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-09-05 22:19:13 +05:30
Abhisar Sinha
605534b417
feature: Shared Storage Filesystem as a First Class Feature (#9208)
This PR implements Storage filesystem as a first class feature.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Filesystem+as+a+First+Class+Feature

Documentation PR: apache/cloudstack-documentation#420

Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-09-05 17:22:32 +05:30
Wido den Hollander
39152323e3
kvm: Present the UUID of the VM as serial through smbios information (#9236)
This PR makes sure a KVM VM gets the UUID of the VM as a static serialnumber through smbios.

Some applications on primarily Windows servers require a stable serial number for licensing purposes. By providing this serial number we can make sure these applications can have a license configured.

More information: https://libvirt.org/formatdomain.html#smbios-system-information
2024-09-04 12:14:17 +05:30
Abhishek Kumar
0692a296ce
engine-orchestration: fix issue for empty product in vm metadata (#9610)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-03 14:03:57 +02:00
Suresh Kumar Anaparti
c05edc40f4
Updated invalid parameter/value to com.cloud.exception.InvalidParameterValueException (#9455) 2024-08-28 08:54:43 +02:00
Abhishek Kumar
1e12a80210
orchestration,hypervisor: allow custom manufacture, product for vm (#9163)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-08-22 17:19:04 +02:00