3328 Commits

Author SHA1 Message Date
Vishesh
594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Abhishek Kumar
939ee9e153
server,engine-orchestration: allocate vm without transaction (#7695)
When deploying a VM is failed during the allocation process it may leave the resources that have been already allocated before the failure. They will get removed from the database as the whole code block is wrapped inside a transaction twice but the server would not inform the network or storage plugins to clean up the allocated resources.
This PR removes Transactions during VM allocation which results in the allocated VM and its resource records being persisted in DB even during failures. When failure is encountered VM is moved to Error state. This helps VM and its resources to be properly deallocated when it is expunged either by a server task such as ExpungeTask or during manual expunge.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-06 14:04:38 +05:30
dahn
acc6f4e725
NPE guard (#7691) 2023-07-06 09:29:42 +02:00
Harikrishna
0cbe77024a
Fix foreign key constraints and the mysql procedure that is used (#7381) 2023-07-06 08:43:49 +02:00
Abhishek Kumar
31dbdd0f5c
engine-orchestration: fix volume size resource count mismatch (#7666)
Fixes case of account/domain having negative storage count when there is a volume size difference while deploying volumes on certain stroages. In case of Powerflex volume size results in a multiple of 8. If user deploys a volume of 12GB it will result in 16GB in size. But currently CloudStack will not update resource count after deployment.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 13:00:32 +05:30
Wei Zhou
985f0ecb53
Tungsten: change conserve_mode of default network offering to 0 (#7511) 2023-06-26 13:36:36 +02:00
slavkap
faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00
Nicolas Vazquez
c809201247
Fix: Volumes on lost local storage cannot be removed (#7594) 2023-06-23 12:22:15 +02:00
Harikrishna
40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
Abhishek Kumar
3748f32bc7
engine-orchestration,vmware: hypervisor migration during start vm migration (#7444)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-20 17:13:08 +02:00
Abhishek Kumar
ef9dec11e7
guestos: add Windows Server 2022 (#7508)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-31 16:02:48 +02:00
Abhishek Kumar
32caf9057e
engine-storage: fix errored template becomes active (#7485)
* engine-storage: fix errored template becomes active

Fixes #7342

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-12 15:23:16 +02:00
Oscar Sandoval
b6443a2b1f
increase log detail for limit checking, fix getDomainReservation() (#7506)
In troubleshooting ops issues we see logs like:

Maximum domain resource limits of Type 'user_vm' for Domain Id = 763 is exceeded: Domain Resource Limit = (1 bytes) 1, Current Domain Resource Amount = (0 bytes) 0, Requested Resource Amount = (1 bytes) 1."

However there is one missing value (currentResourceReservation) that is used in the calculation of limit check but it is not logged, which leads to confusion. Above we see we are using “0” and requested 1, with our limit being 1, but was rejected. Without logging all the values used in the calculation we don’t understand why it failed.

Additionally, if we had this log above it would be clearer that a second bug is occurring. When we query for domain level resource reservations in “getDomainReservation” the actual SearchBuilder is the listAccountAndTypeSearch, not the listDomainAndTypeSearch. As a result, when we call getDomainReservation the query returns any outstanding domain reservation for any account, as domain ID is not a valid filter for the account search.

This PR:

Increases detailed information in log for checking resource limit to include reservations information for functions: checkDomainResourceLimit() and checkAccountResourceLimit

Fixes getDomainReservation() to use listDomainAndTypeSearch instead of listAccountAndTypeSearch

Co-authored-by: Oscar Sandoval <osandovalocana@apple.com>
2023-05-12 12:53:18 +05:30
Marcus Sorensen
ec0f8bddf6
Support local storage live migration for direct download templates (#7453)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-05-04 17:37:58 -03:00
dahn
8bbe2f7cb2
engine/schema: use junit 4 (same as the rest of the project tests) 2023-04-27 11:36:27 +02:00
Abhishek Kumar
afccde5cfc
server,engine-schema: fix listing service offering for vm scale (#7398)
Fixes #7389

Fixes listing of service offerings for VM scale when the current offering has  `disk_offering_strictness=true`

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-04-17 14:19:20 +05:30
Harikrishna Patnala
2a68259d0d Merge remote-tracking branch 'origin/4.17' into 4.18 2023-04-17 11:46:17 +05:30
Harikrishna
96897f07f2
Set hypervisor hostname via userdata only when the global settings are allowed (#7360)
* Set hypervisor hostname via userdata only when the corresponding global settings are enabled

* reformat code
2023-04-17 11:25:56 +05:30
Nicolas Vazquez
2dc016adde
Fix for direct download templates with multiple bypassed references (#7400)
This PR fixes an issue observed on multiple zones and direct download templates on KVM, in which a template gets multiple records on the template_store_ref table. When this happens, the template cannot be used as direct download. In case of a system VM template using direct download, system VM deployments fail
2023-04-13 12:48:29 +05:30
Harikrishna
65512af779
server: Roles search by keyword (#7384)
This PR fixes #7362 and also other search criteria to use the name as an exact search where keyword is also there.

Made UI changes for roles search to make use of keyword instead of name.
2023-04-11 22:24:29 +05:30
Harikrishna
b774ee5d11
vmware: Datastore cluster synchronization should check if the child datastores are in UP state or not (#7385)
This fix ensures when datastore cluster in VMware is added as a primary storage pool in CloudStack then all the child datastores (which already exists in CS) should be in Up state.

For example:

1. Datastore Cluster DS has two child datastores A and B in vCenter. (B is already added as a storage pool in CloudStack)
2. Now try to add datastore cluster DS into CloudStack as a primary storage pool
3. CloudStack tries to add child datastores A and B in CloudStack, since B is already there in CloudStack, it will reuse the existing storagepool entry and will keep under parent Storage pool DS.

During Step 3 we are now checking if B is Up state or not.
2023-04-11 22:23:12 +05:30
Abhishek Kumar
ce981574c3 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-03-31 17:00:00 +05:30
Abhishek Kumar
9aee625ab4
orchestration: fix error on deleted template vm start (#7327)
* server: fix error on deleted template vm start

When a VM is deployed with start flag as false and the template is deleted before the VM start, NPE is obeserved it is started for the first time.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-03-31 15:14:09 +05:30
Rohit Yadav
74a414e76f Fix build failure and 4.18.0.0->4.18.1.0 upgrade path
Unblocks smoketests/health check runs on 4.18 branch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-03-16 13:18:26 +05:30
Daan Hoogland
05cda2729f Updating pom.xml version numbers for release 4.18.1.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:38:14 +01:00
Daan Hoogland
0574087284 Updating pom.xml version numbers for release 4.18.0.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-11 09:35:41 +01:00
SadiJr
ad64ee3090
[Usage] Fix wrong usage_type (#7229) 2023-03-03 13:10:38 +01:00
Wei Zhou
8592de95fa
Move PassphraseVO to use String instead of byte[] to support Encrypt annotation (#7302)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-03-03 13:08:17 +01:00
dahn
aa881f6bf3
avoid hard coded guest OS IDs (#7279) 2023-02-23 15:19:45 +01:00
Wei Zhou
e61051e4fc
SQL: Remove snapshot references if primary storage pool has been removed (#7205) 2023-02-21 14:48:36 +01:00
Harikrishna
7484636997
Userdata sql changes using procedures (#7255) 2023-02-20 12:07:52 +01:00
Wei Zhou
80c1b6b2a2
engine/schema: fix IDEMPOTENT_ADD_FOREIGN_KEY procedure does not work on mysql8 (#7252) 2023-02-17 11:46:09 +01:00
Wei Zhou
8ef35466de
Tungsten: fix functional issues (#7173)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-02-13 09:15:28 +01:00
Harikrishna
a3feccf70c
User two factor authentication (#6924)
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-02-13 09:14:17 +01:00
Daan Hoogland
1b89a22ab0 fix merge conflicts of merge branch '4.17' 2023-02-08 16:52:35 +01:00
Daan Hoogland
2149e82856 Merge release branch 4.17 to main 2023-02-08 15:50:11 +01:00
Wei Zhou
66f351543a
server: do not deploy or upgrade vm with inactive service offering (#7063)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-02-08 09:40:30 +01:00
Abhishek Kumar
6d287793a5
guestos: add el9 distros (#7059)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-02-03 12:46:53 +01:00
Bryan Lima
5c20e7844c
Quota VM_DISK tariff calculation (#7152)
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2023-02-03 08:43:05 +01:00
Wei Zhou
62e342c1bc
utils,framework/db: Introduce new database encryption cipher based on AesGcmJce (#7003) 2023-02-02 16:25:49 +01:00
Nicolas Vazquez
89bf4750ab
Add console session cleanup task (#7132) 2023-02-01 16:53:54 +01:00
David Jumani
c774b865c9
Tungsten integration (#7065)
Co-authored-by: rtodirica <rtodirica@ena.com>
Co-authored-by: Huy Le <huylm@unitech.vn>
Co-authored-by: radu-todirica <Radu.Todirica@ness.com>
Co-authored-by: Huy Le <minh.le@ext.ewerk.com>
Co-authored-by: Simon Weller <siweller77@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-02-01 09:19:53 +01:00
Suresh Kumar Anaparti
d8c7e34b38
Improve global settings UI to be more intuitive/logical (#5797)
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-01-31 11:23:43 +01:00
Abhishek Kumar
3b6ce97097
infra: edge zones (#6840)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-01-31 09:36:45 +01:00
nvazquez
83c2bfacd8
Merge branch '4.17' 2023-01-30 07:53:58 -03:00
Abhishek Kumar
028ca74fb6
ui,server,api: resource metrics improvements (#6803)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-01-30 09:48:03 +01:00
Nicolas Vazquez
c78a777d3a
Fix: memory leak on volume allocation (#7136) 2023-01-30 09:44:50 +01:00
João Jandre
61a722548f
Create API to reassign volume (#6938) 2023-01-27 11:10:56 +01:00
slavkap
d288bb0c78
KVM support of iothreads and IO driver policy (#6909) 2023-01-25 12:34:05 +01:00
Rohit Yadav
ea14c59210
vmware: support vsphere 8.0 and 8.0.0.1 versions (#7119) 2023-01-25 09:20:21 +01:00