11303 Commits

Author SHA1 Message Date
nvazquez
3674aff4d4
Merge branch '4.16' 2022-04-14 01:17:38 -03:00
Rakesh
c7941278f7
Allow creating snapshot from VM snapshot (#4739)
If `kvm.snapshot.enabled` is set to false then we cant create snapshot from
VM snapshot. With this change, its possible to create snapshot from VM
snapshot even when the global setting is set to false.
Note that you still cant directly create a snapshot from volume though
2022-04-14 01:16:59 -03:00
Nicolas Vazquez
fbf77978e1
Fix: Allow disabling the login attempts mechanism for disabling users (#6254)
* Fix: Allow disabling the login attempts mechanism for disabling users

* Refactor
2022-04-14 01:11:43 -03:00
nvazquez
67002b67ec
Merge branch '4.16' 2022-04-13 07:22:56 -03:00
Wei Zhou
91a5f0e285
server: honor global setting system.vm.default.hypervisor as first option when deploy VRs (#6160) 2022-04-13 07:22:22 -03:00
Daniel Augusto Veronezi Salvador
39fad2d9d7
KVM disk-only based snapshot of volumes instead of taking VM's full snapshot and extracting disks (#5297)
* Refactor create volume snapshot with running VM

* Refactor create volume snapshot with stopped VM

* Refactor create volume from snapshot

* Refactor create template from snapshot

* Refactor volume migration (migrateVolume/ migrateVirtualMachineWithVolume)

* Refactor snapshot deletion

* Refactor snapshot revertion

* Adjusts and fix cherry-pick conflicts

* Remove diffuse tests

* Add validation to add flag '--delete' on command 'virsh blockcommand' only if libvirt version is equal or higher 6.0.0

* Expunge temporary snapshot only if template creation is from snapshot

* Extract strings to constant

* Remove unused imports

* Fix error on revert backed up snapshot

* Turn method's return to void as it is not used

* Rename method in SnapshotHelper

* Fix folder creation when using SharedMountPoint pool

* Remove static import

* Remove unnused method

* Cover take snapshot in centos 7

* Handle right snapshot flag according to qemu version

Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2022-04-12 08:14:27 -03:00
Pearl Dsilva
431c352a6d
Synchronization of network devices on newly added hosts for Persistent Networks (#5977)
* Persistent Network feature & Marvin component tests

* Cleaned up comments and imports

* fixed small error

* add support to add setup persistent networks' resources when a disabled host is enabled

* small fix

* use wildcard instead of hard-coding the bridge name

* allow clean up of resources when removing a host in maintenance mode

* skip test for simulator hypervisor

Co-authored-by: shatoboar <sang-woo.bae@campus.tu-berlin.de>
2022-04-11 23:12:05 -03:00
Nicolas Vazquez
5435b0abfe
Direct download certificates additions and improvements (#6104)
* Add direct download certificates listing

* Restore class to original project

* Small refactor

* Register API

* Apply suggestions from code review

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Refactor after review

* Fix checkstyle

* Add hosts mapping to API response

* Improvements on revoke certificate

* Refactor revoke certificate API

* Fix condition

* Filter only certificates not revoked for revokeCertificate API

* Improve upload certificate and add provision certificate API

* Improve certificate response output

* Address review comments

* Refactor revoke cert test

* Fix marvin test

* Address review comments

* Fix issues

* Improvements

* Refactor upload template API response

* Fix response

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2022-04-11 22:57:23 -03:00
nvazquez
ed55002795
Merge branch '4.16' 2022-04-11 11:20:10 -03:00
Pearl Dsilva
b6072fc826
Allow expunging a VM on a deleted host when using host cache and ConfigDrive userdata service (#6234) 2022-04-11 11:19:37 -03:00
José Flauzino
16f2896940
Persistence of VM stats (#5984)
* Add persistence of VM stats

* Fix API 'since' attribute

* Add license

* Address GutoVeronezi's reviews

* Fix the order of VM stats in the API response

* Fix msid in VM stats data

* Fix disk stats and add minor improvements

* Add log message

* Build string using ReflectionToStringBuilderUtils

* Rerun checks

Co-authored-by: joseflauzino <jose@scclouds.com.br>
2022-04-11 10:42:21 -03:00
Rohit Yadav
5b617ba3ff Merge remote-tracking branch 'origin/4.16' 2022-04-08 15:28:20 +05:30
Abhishek Kumar
e7071ec196
server: increment deviceid while importing vm data volumes (#6123)
Fixes #6121

deviceId for a volume/disk is never used by CloudStack VMware hypervisor plugin. Still it would be correct to increment value for each data volume/disk, behaviour similar to adding data volumes/disks.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-04-08 15:24:10 +05:30
slavkap
2b075ed39e
Storage-based Snapshots for KVM VMs (#3724)
* VM snapshots of running KVM instance using storage providers plugins for disk snapshots

Added new virtual machine snapshot strategy which is using storage providers plugins to take/revert/delete snapshots.
You can take VM snapshot without VM memory on KVM instance, using storage providers implementations for disk snapshots.
Also revert and delete is added as functionality. Added Thaw/Freeze command for KVM instance.
The snapshots will be consistent, because we freeze the VM during the snapshotting. Backup to secondary storage is executed after
thaw of the VM and if it is enabled in global settings.

* Removed duplicated functionality

Set few methods in DefaultVMSnapshotStrategy to protected to reuse them
without duplicating the code. Remove code that is actualy not needed

* Added requirements in global setting kvm.vmstoragesnapshot.enabled

Added more information in kvm.vmstoragesnapshot.enabled global setting,
that it needs installation of:
- qemu version 1.6+
- qemu-guest-agent installed on guest virtual machine

when the option is enabled

* Added Apache license header

* Removed commented code

* If "kvm.vmstoragesnapshot.enabled" is null should be considered as false

* removed unused imports, replaced default template

Removed unused imports which causing failures and replaced template to
CentOS8

* "kvm.vmstoragesnapshot.enabled" set to dynamic

* Getting status of freeze/thaw commands not the return code

Will chacke the status if freeze/thaw of Guest VM succeded, rather than
looking for return code. Code refactoring

* removed "CreatingKVM" VMsnapshot state and events related to it

* renamed AllocatedKVM to AllocatedVM

the states should not be associated to a hypervisor type

* loggin the result of "drive-backup" command

* Check which VM snapshot strategy could handle the vm snapshots

gets the best match of VM snapshot strategy which could handle the vm
snapshots on KVM.
Other storage plugins could integrate with this functionality to support group snapshots

* Added poolId in canHandle for KVM hypervisors

Added poolId into canHandle method used to check if all volumes are on
the same PowerFlex's storage pool

* skip smoke tests if the hypervisor's OS type is CentOS

This PR works with functionality included in qemu-kvm-ev which
does not come by default on CentOS. The smoke tests will be skipped if
the hypervisor OS is CentOS

* Added missed import in smoke test

* Suggested change to use ` org.apache.commons.lang.StringUtils.isNotBlank`

* Fix getting device on Ubuntu

On Ubuntu the device isn't provided and we have to get it from
node-name parameter. For drive-backup command (for Ubuntu) is needed and job-id which
is the value of node-name (this extra param works on Ubuntu and CentOS as well).

* Removed new snapshot states and functionality for NFS

* throw CloudRuntimeException

provide a properer error message when delete VM snapshot fails

* exclude GROUP snapshots when listing snapshots

* Skip tests if there is pool with NFS/Local

* address comments
2022-04-07 21:42:12 -03:00
Pearl Dsilva
c1c381fcd1
api: Prevent modifying acl rules order for default ACLs (#6187) 2022-04-07 21:31:29 -03:00
Wido den Hollander
68fa9a1061
Allow storage.overprovisioning.factor to be <1 which allows underprovisioning of storage pools (#6218)
By setting the value to 0.5 (example) we can underprovision a storage pool. This is useful in
cases with Ceph where it reports the raw size of the cluster where the replication factor is
not taken into account.

Fixes: ##6205
2022-04-07 09:50:55 -03:00
JoaoJandre
afc819e5c8
Added new field to updateBackupOffering API. (#6190)
* Added new field to updateBackupOffering API.

* Fix some unit tests.

Co-authored-by: João Paraquetti <joao@scclouds.com.br>
Co-authored-by: Joao <JoaoJandre@gitlab.com>
2022-04-06 09:56:35 -03:00
Wei Zhou
ca8782b703
api: add vpcname to networkacl response (#6207)
This PR fixes the vpc uuid in info card of network acls
2022-04-06 09:33:28 +05:30
Pearl Dsilva
9cc8da2a30
api: Update account type when updating account role (#6156)
* api: Update account type when updating account role

* extract code to validate role change
2022-04-06 00:50:56 -03:00
Wei Zhou
e4b6992def
server: fix list reserved/free public ips in project (#6198) 2022-04-06 00:17:31 -03:00
David Jumani
5ab4309298
agent: Detect existing hosts with UEFI support (#6139)
* agent: Pass uefi enabled status as part of ready command

* Cleanup

* Fix checkstyle

* Save uefi status if different
2022-04-04 23:31:54 -03:00
Rakesh
b88cfc226e
Mount disabled storage pool on host reboot (#6164)
* Mount disabled storage pool on host reboot

Add a global setting so that disabled pools will be mounted
again on host reboot

* fix build error

* Update description

* add cluster-wide support

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2022-04-01 23:38:32 -03:00
Pearl Dsilva
2c8c476656
CKS: Support deployment of CKS clusters on Advanced zones with security groups (#6132)
* CKS: Support deployment of CKS clusters on Advanced zones with security groups

* use available constant

* address comments -
- Ingress sg rule for port 22 & 6443
- Use constant to define securityGroup Name
- rename variable name from type -> vmType

* unique name for security group + foreign key

* use constants
2022-04-01 13:19:54 -03:00
Wei Zhou
944b6037cc Merge remote-tracking branch 'apache/4.16' 2022-03-29 10:43:59 +02:00
Pearl Dsilva
908f594f00
configDrive: Fix failure to delete (unstarted) VM (#6146)
* configDrive: Fix failure to delete (unstarted)VM

* Address comments

* Address comments
2022-03-29 09:56:54 +02:00
Pearl Dsilva
bcd1a3274a
api: Fix reset configuration (#6168) 2022-03-29 09:54:34 +02:00
John Bampton
15937369fe
Fix spelling (#6161)
* Fix spelling

* Fix spelling
2022-03-29 00:21:07 -03:00
Pearl Dsilva
add657a899
api: Remove redundant API parameters (#6143) 2022-03-25 14:41:38 -03:00
David Jumani
6125886f3d
Set UefiCapabilty for all hypervisors in hostresponse (#6140) 2022-03-21 15:13:03 -03:00
dahn
13efa59097
Router health check notification mail to show router name next to UUID (#6130)
* alerrt message altered

* review feedback and format

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2022-03-20 23:44:06 -03:00
dahn
3e4e417389
account check made explicit - cleanup (#6122)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2022-03-20 23:41:06 -03:00
Wei Zhou
88d77c86a7
server: fix NPE when router.service.offering is set due to service/disk offering refactoring (#6120)
* server: move files in server/src/main/java/org/cloud/network/router/deployment/

* server: fix NPE when router.service.offering is set due to service/disk offering refactoring
2022-03-20 23:40:30 -03:00
nvazquez
6e061d90fe
Merge branch '4.16' 2022-03-18 07:56:27 -03:00
Pearl Dsilva
4be99fe971
api: Allow updating VM settings when custom contrained offering is used (#6136) 2022-03-18 09:09:40 +01:00
Wei Zhou
6a53517d37
New feature: Reserve and release Public IPs (#6046)
* Reserve and release a public IP

* Update #6046: show orange color for Reserved public ip

* Update #6046 reserve IP: fix ui conflicts

* Update #6046: fix resource count

* Update #6046: associate Reserved public IP to network

* Update #6046: fix unit tests

* Update #6046: fix ui bugs

* Update #6046: make api/ui available for domain admin and users
2022-03-17 14:35:40 -03:00
Rakesh
6f3c18f3a7
alert: Send alert for ha'ed vm's (#5664)
* alert: Send alert for ha'ed vm's

When ha is performed on vm's send the alert for it so that
its for admins to know which vm's got ha'ed else its time
consuming to get those details from logs

* feedback change

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2022-03-17 00:52:35 -03:00
Nicolas Vazquez
3fc31ca1e5
Revert "Honour isrecursive above listall (#6045)" (#6126)
This reverts commit 71056191f2bdad4be1a7eaf9bb73a7dcee3516f2.
2022-03-16 23:17:02 -03:00
Pearl Dsilva
ca9e28dc84
api: Fix issue observed with message publish on creation of domain (#6118)
* api: Fix issue observed with message publish on creation of domain

* add check for successful creation of domain  before publishing event
2022-03-16 07:55:36 +01:00
David Jumani
71056191f2
Honour isrecursive above listall (#6045)
* Honour isrecursive above listall

* Revert "Honour isrecursive above listall"

This reverts commit 12bc26ae1c58ff5a51937026c5be8d7cfa8bf141.

* Being smart about it

* Cleanup

* fixing npe
2022-03-10 23:59:42 -03:00
nvazquez
e3132af64e
Merge branch '4.16' 2022-03-10 08:49:43 -03:00
Wei Zhou
3a456f1b31
server: mark volume snapshots as Destroyed if it does not exist on primary and secondary storage when delete a volume (#6057)
* server: mark volume snapshots as Destroyed in some cases when delete a volume in QCOW2 format

when delete a volume in QCOW2 format, if volume snapshot does not exist on primary and secondary storage, mark the snapshot as Destroyed.

* Update #6057: remove check on volume format
2022-03-10 08:49:03 -03:00
JoaoJandre
5f07ddaca9
Refactor account type (#6048)
* Refactor account type

* Added license.

* Address reviews

* Address review.

Co-authored-by: João Paraquetti <joao@scclouds.com.br>
Co-authored-by: Joao <JoaoJandre@gitlab.com>
2022-03-09 11:14:19 -03:00
David Jumani
5534b7afda
api: Warn if query parameters have multiple values (#5009)
* api: Ensure single query parameter value

* change error to warn

* Address comments
2022-03-08 14:22:39 -03:00
John Bampton
6401c850b7
Fix spelling (#6064)
* Fix spelling

- `interupted` to `interrupted`
- `paramter` to `parameter`

* Fix more typos
2022-03-08 13:02:35 -03:00
John Bampton
08bb4ad9fe
spelling: Fix spelling Occured to Occurred (#6061)
Fix typos
2022-03-07 15:07:02 +05:30
nvazquez
cb872ab6bd
Merge branch '4.16' 2022-03-06 10:08:35 -03:00
Suresh Kumar Anaparti
2820a36f86
Check the network access when deploying VM in Advanced Security Group. (#6050)
* Check the network access when deploying VM in Advanced Security Group.

* Removed comment

* Removed redundant network access check, owner access check already exists
2022-03-06 10:07:57 -03:00
Pearl Dsilva
4d54e8e84c
Allow specifying disk size, min/max iops for offering linked with custom disk offering (#6018) 2022-03-05 00:11:25 -03:00
Pearl Dsilva
3a74ec78be
api: Fix search by name (#6032)
* Search by name

* add keyword search for projectroles

* remove unnecessary arg
2022-03-04 23:52:53 -03:00
Pearl Dsilva
4ebc7413f0
Fix API parameter description for boottype/bootmode (#6054) 2022-03-04 12:09:00 -03:00