11530 Commits

Author SHA1 Message Date
Harikrishna
40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
dahn
1aa4f80741
accept first word from host os string for backwards compatibility (#7620) 2023-06-15 10:09:31 +03:00
Abhishek Kumar
2d6a069812
server: improve storage GC to skip expunging possible duplicate volumes (#7313)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-05 10:03:24 +02:00
Abhishek Kumar
7319debc87
cks: k8s cluster on vpc tier (#7479)
* cks: cluster on vpc tier

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* python fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix trailing space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:29:06 +05:30
Abhishek Kumar
8849e0f464
server: fix volume detach operation when no vm host (#7526)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-25 15:48:27 +02:00
Oscar Sandoval
b6443a2b1f
increase log detail for limit checking, fix getDomainReservation() (#7506)
In troubleshooting ops issues we see logs like:

Maximum domain resource limits of Type 'user_vm' for Domain Id = 763 is exceeded: Domain Resource Limit = (1 bytes) 1, Current Domain Resource Amount = (0 bytes) 0, Requested Resource Amount = (1 bytes) 1."

However there is one missing value (currentResourceReservation) that is used in the calculation of limit check but it is not logged, which leads to confusion. Above we see we are using “0” and requested 1, with our limit being 1, but was rejected. Without logging all the values used in the calculation we don’t understand why it failed.

Additionally, if we had this log above it would be clearer that a second bug is occurring. When we query for domain level resource reservations in “getDomainReservation” the actual SearchBuilder is the listAccountAndTypeSearch, not the listDomainAndTypeSearch. As a result, when we call getDomainReservation the query returns any outstanding domain reservation for any account, as domain ID is not a valid filter for the account search.

This PR:

Increases detailed information in log for checking resource limit to include reservations information for functions: checkDomainResourceLimit() and checkAccountResourceLimit

Fixes getDomainReservation() to use listDomainAndTypeSearch instead of listAccountAndTypeSearch

Co-authored-by: Oscar Sandoval <osandovalocana@apple.com>
2023-05-12 12:53:18 +05:30
Abhishek Kumar
a0eb0aa5f7
api,server: fix VM.CREATE events on vm deploy without start (#7421)
Fixes #6697

Allows the server to generate started and completed events for VM.CREATE event type when VM is deployed with startvm=false.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-11 23:23:00 +05:30
Abhishek Kumar
d9e0a3e6c3
server: add config SSVM capacity scan (#7448)
Fixes #7438

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-11 12:58:10 +05:30
Marcus Sorensen
8604cb5328
server: Fix DirectDownload certificate check initial delay (#7494)
This PR adjusts the DirectDownload certificate check initial delay. Since the time unit is in hours, I think it was a mistake to schedule the initial check to be in 60 hours after management servers start - the intention was likely 60 seconds. We had turned this feature on to run hourly, not realizing we would have to wait 2.5 days to see it first run!

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-05-08 12:09:50 +05:30
Marcus Sorensen
3cb4c801fb
server: fix null pointer on powerflex attach volume edge case (#7498)
This PR fixes a null pointer edge case where a PowerFlex volume is attached to a VM.

In this edge case, a VM has been created, started, stopped, and then reimaged. VM has a last host ID but the newly attached volume does not yet have a storage pool assigned, it will be assigned on the VM start. However, we assume that if the VM's host is not null, we need to try to revoke access to the volume for the host. Since there is no storage pool yet for the volume, we hit a null pointer when checking to see if the volume's pool is PowerFlex.

This was affecting all storage types, I could reproduce it with local storage, since the null pointer is at the check for pool's type.

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-05-08 12:07:41 +05:30
Abhishek Kumar
b84744d9a5
server: validate ip address value on update config (#7415)
Fixes #6958

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-04-28 16:11:33 +05:30
Abhishek Kumar
41bbedb530 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-04-19 12:36:10 +05:30
Vishesh
79eae89a87
ui: Add filtering by state in account, systemvms, router and storagepool (#7368)
This PR allows admin to filter resources by state for systemvms, router & storagepool. This is part of #7366 .

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-04-19 12:22:24 +05:30
Wei Zhou
fb51504dff
server: dedicate vxlan range to account (#7215)
This fixes #6871
2023-04-19 12:14:53 +05:30
Abhishek Kumar
3e9d7e0554
server: rephrase error message for iso vm reinstall (#7407)
Fixes #3603

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-04-17 10:38:54 +05:30
Harikrishna
65512af779
server: Roles search by keyword (#7384)
This PR fixes #7362 and also other search criteria to use the name as an exact search where keyword is also there.

Made UI changes for roles search to make use of keyword instead of name.
2023-04-11 22:24:29 +05:30
Harikrishna
b774ee5d11
vmware: Datastore cluster synchronization should check if the child datastores are in UP state or not (#7385)
This fix ensures when datastore cluster in VMware is added as a primary storage pool in CloudStack then all the child datastores (which already exists in CS) should be in Up state.

For example:

1. Datastore Cluster DS has two child datastores A and B in vCenter. (B is already added as a storage pool in CloudStack)
2. Now try to add datastore cluster DS into CloudStack as a primary storage pool
3. CloudStack tries to add child datastores A and B in CloudStack, since B is already there in CloudStack, it will reuse the existing storagepool entry and will keep under parent Storage pool DS.

During Step 3 we are now checking if B is Up state or not.
2023-04-11 22:23:12 +05:30
Abhishek Kumar
6886fca105
server: list vm search vm ip using keyword (#7406)
Fixes #7390

Allows searching VMs by IP address using keyword parameter of the listVirtualMachine API.
2023-04-11 12:00:38 +05:30
Wei Zhou
54606dc965 server: fix 4.18/main build error after merge forward 2023-04-05 18:29:46 +02:00
Daan Hoogland
a28b4398c5 Merge branch '4.17' into 4.18 2023-04-05 16:54:35 +02:00
Harikrishna
b2f1965ccb
Fix ScaleVM to consider resize volume in any type of service offering (#7359) 2023-04-05 16:21:24 +02:00
Abhishek Kumar
30998d0ab7
server: fix userdatadetails parsing (#7328)
Fixes the case when userdata variable value contains '=' sign. This PR considers everything before occurrence of first '=' sign as key and remaining string as value.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-04-04 17:01:04 +05:30
Harikrishna
ec3698dfbe
Added details about account/domain to which network will be assigned during creation (#7387) 2023-04-04 08:50:16 +02:00
Daan Hoogland
16694d8bec Merge branch '4.17' into 4.18 2023-03-29 17:36:55 +02:00
Harikrishna
9fb20056d5
Fixed avoid set variables which is causing deployment failures (#7372) 2023-03-29 17:14:18 +02:00
kishankavala
17b8631615
Add service ip to listManagementServers API response (#7374) 2023-03-29 14:46:39 +02:00
João Jandre
523ab58d02
Fix PR 7131 bugs and vulnerabilities (#7140) 2023-03-21 15:06:18 +01:00
Daan Hoogland
05cda2729f Updating pom.xml version numbers for release 4.18.1.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:38:14 +01:00
Daan Hoogland
0574087284 Updating pom.xml version numbers for release 4.18.0.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-11 09:35:41 +01:00
Harikrishna
a3670496ad
Change 2FA plugin related global setting names (#7275) 2023-02-22 12:33:52 +01:00
Harikrishna
a3feccf70c
User two factor authentication (#6924)
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-02-13 09:14:17 +01:00
Daan Hoogland
3b67be2d52 Merge release branch 4.17 to main
* 4.17:
  server: fix exception while list users with keyword (#7169)
2023-02-10 14:08:15 +01:00
Wei Zhou
e62062f024
server: fix exception while list users with keyword (#7169) 2023-02-10 13:39:23 +01:00
Daan Hoogland
1b89a22ab0 fix merge conflicts of merge branch '4.17' 2023-02-08 16:52:35 +01:00
Daan Hoogland
2149e82856 Merge release branch 4.17 to main 2023-02-08 15:50:11 +01:00
Wei Zhou
66f351543a
server: do not deploy or upgrade vm with inactive service offering (#7063)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-02-08 09:40:30 +01:00
Wei Zhou
62e342c1bc
utils,framework/db: Introduce new database encryption cipher based on AesGcmJce (#7003) 2023-02-02 16:25:49 +01:00
Nicolas Vazquez
89bf4750ab
Add console session cleanup task (#7132) 2023-02-01 16:53:54 +01:00
Daan Hoogland
154f3ac1a7 Merge release branch 4.17 to main
* 4.17:
  server: add/update user vm details for new root disk size (issue 7139) (#7145)
  UI: fix missing networkid and zoneid when add instance from vpc network (#7151)
2023-02-01 12:06:02 +01:00
Wei Zhou
e1d3657766
server: add/update user vm details for new root disk size (issue 7139) (#7145) 2023-02-01 11:01:33 +01:00
David Jumani
c774b865c9
Tungsten integration (#7065)
Co-authored-by: rtodirica <rtodirica@ena.com>
Co-authored-by: Huy Le <huylm@unitech.vn>
Co-authored-by: radu-todirica <Radu.Todirica@ness.com>
Co-authored-by: Huy Le <minh.le@ext.ewerk.com>
Co-authored-by: Simon Weller <siweller77@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-02-01 09:19:53 +01:00
Suresh Kumar Anaparti
d8c7e34b38
Improve global settings UI to be more intuitive/logical (#5797)
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-01-31 11:23:43 +01:00
Abhishek Kumar
3b6ce97097
infra: edge zones (#6840)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan@onecht.net>
2023-01-31 09:36:45 +01:00
Stephan Krug
26eaae7872
Allow VPC offering creation only with active VR service offerings (#6957) 2023-01-31 08:42:57 +01:00
nvazquez
83c2bfacd8
Merge branch '4.17' 2023-01-30 07:53:58 -03:00
Abhishek Kumar
028ca74fb6
ui,server,api: resource metrics improvements (#6803)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-01-30 09:48:03 +01:00
Nicolas Vazquez
c78a777d3a
Fix: memory leak on volume allocation (#7136) 2023-01-30 09:44:50 +01:00
Nicolas Vazquez
eac357cb77
kvm: Secure KVM VNC Console Access Using the CA Framework (#7015)
This PR allows securing the console access through CloudStack to the virtual machines running on KVM. The secure access is achieved through the generated certificates for the CA Framework in CloudStack, that provides mutual TLS connections between agents. These certificates are used to also secure the connection between the console proxies and the VNC ports for VM console access.

This feature is only supported on the KVM hypervisor

Design Document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Secure+KVM+VNC+connection+using+the+CA+framework
2023-01-27 17:22:06 +05:30
João Jandre
61a722548f
Create API to reassign volume (#6938) 2023-01-27 11:10:56 +01:00
Daan Hoogland
46924a5782 Merge release branch 4.17 to main
* 4.17:
  Use long instead of int in DB statistics for Queries and Uptime. (#7125)
  VR: fix public-key is missing in VR after acquiring public IP (#7103)
2023-01-26 09:59:36 +01:00