36012 Commits

Author SHA1 Message Date
Wei Zhou
3e04779f60
console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237) 2023-07-05 11:01:32 +02:00
Abhishek Kumar
5e5d194d77
server: do not check zone imagestores for directdownload template delete (#7607)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-04 09:37:06 +02:00
dahn
1275db4081
UI: Zone wizard fix (#7588)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 11:56:52 +02:00
Abhishek Kumar
31dbdd0f5c
engine-orchestration: fix volume size resource count mismatch (#7666)
Fixes case of account/domain having negative storage count when there is a volume size difference while deploying volumes on certain stroages. In case of Powerflex volume size results in a multiple of 8. If user deploys a volume of 12GB it will result in 16GB in size. But currently CloudStack will not update resource count after deployment.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 13:00:32 +05:30
Harikrishna
70820137e6
scaleio: Avoid race condition while handling host disconnect and connect scenarios (#282) (#7689)
This PR fixes an intermittent issue where SDC id (local_path) is getting deleted and not getting populated when host connects back again.

Fix is to remove the code to delete the records from storage_pool_host_ref table. We are anyways updating the entry if the SDC ID is changed during agent restart which is anyways required inorder to get the new connections. I've quickly verified the host delete scenario to check the storage_pool_host_ref entries behavior, entries are getting deleted.
2023-07-03 12:58:42 +05:30
Abhishek Kumar
de6ce503dc
api: correct error on resize volume resource allocation failure (#7687)
This PR resource throws exception with the correct error code and logs the error message when a resource allocation failure is encountered during resize volume operation.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 12:57:14 +05:30
Wei Zhou
f3ad9e6743
marvin: fix smoke test failures if mgmt server is Ubuntu (#7583) 2023-06-27 11:57:02 +02:00
Abhishek Kumar
83dca2bf51
ui: fix vm import for L2 n/w in Setup state (#7628)
L2 networks created using offerings that allow using custom VLAN can remain in Setup state. Allow importing of VMs with such networks when VLAN matches with unmanaged instance.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-27 14:57:42 +05:30
Wei Zhou
973b0e28fd test: fix Super-Linter Check error in storpool tests
2023-06-27 08:42:08 [INFO]   File:[/github/workspace/test/integration/plugins/storpool/TestEncryptedVolumes.py]
2023-06-27 08:42:08 [ERROR]   Found errors in [flake8] linter!
2023-06-27 08:42:08 [ERROR]   Error code: 1. Command output:
------
/github/workspace/test/integration/plugins/storpool/TestEncryptedVolumes.py:681:113: W292 no newline at end of file
------
2023-06-27 08:42:08 [INFO]   ---------------------------
2023-06-27 08:42:08 [INFO]   File:[/github/workspace/test/integration/plugins/storpool/sp_util.py]
2023-06-27 08:42:08 [ERROR]   Found errors in [flake8] linter!
2023-06-27 08:42:08 [ERROR]   Error code: 1. Command output:
------
/github/workspace/test/integration/plugins/storpool/sp_util.py:798:31: W292 no newline at end of file
------
2023-06-27 10:47:25 +02:00
Wei Zhou
985f0ecb53
Tungsten: change conserve_mode of default network offering to 0 (#7511) 2023-06-26 13:36:36 +02:00
slavkap
faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00
Nicolas Vazquez
c809201247
Fix: Volumes on lost local storage cannot be removed (#7594) 2023-06-23 12:22:15 +02:00
Vishesh
0acc66f51d
server: Add check on host's status while deleting config drive on host cache (#7584)
This PR adds a check on host's status. Without this if the agent is not in Up or Connecting state, expunging of a VM fails.

Steps to reproduce:
- Enable vm.configdrive.force.host.cache.use in Global Configuration.
- Create a L2 network with config drive
- Deploy a vm with the L2 network created in previous step
- Stop the vm and destroy vm (not expunge it)
- Stop the cloudstack-agent on the VM's host
- Expunge the vm

Fixes: #7428
2023-06-23 13:46:22 +05:30
Wei Zhou
9e1609dd5a
[4.18] Bump the version to 4.18.1.0-SNAPSHOT (#7664) 2023-06-22 06:59:08 +02:00
Harikrishna
40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
Abhishek Kumar
3748f32bc7
engine-orchestration,vmware: hypervisor migration during start vm migration (#7444)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-20 17:13:08 +02:00
Abhishek Kumar
658daef715
utils: fix check for mrtalink url (#7636)
Fixes incorrect condition while checking metalink URLs for accesibility
2023-06-19 12:23:22 +05:30
dahn
1aa4f80741
accept first word from host os string for backwards compatibility (#7620) 2023-06-15 10:09:31 +03:00
dahn
fb29608f61
consider last host only for root-admin (#7597) 2023-06-14 09:35:55 +02:00
Abhishek Kumar
4ef7ebbded
ssvm: pass all accessible secondary storage to ssvm (#7410)
* ssvm: pass all accessible secondary storage to ssvm

Fixes #7162

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* license

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-12 09:02:42 +02:00
dahn
0d3ac9f8d9
interpret /etc/redhet-release better (#7570) 2023-06-09 10:29:42 +02:00
Abhishek Kumar
03910c2783 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-06-08 15:34:03 +05:30
Abhishek Kumar
20865901d1 Merge remote-tracking branch 'apache/4.16' into 4.17 2023-06-08 15:33:20 +05:30
Abhishek Kumar
24da2bcf78
marvin: fix test_data for parsing failure (#7602)
PR #6512 seems to have broken the test_data.py. This PR fixes test_data.
2023-06-08 15:32:04 +05:30
Vishesh
811b6293af
.github: Use parallel builds for github actions (#7592) 2023-06-07 13:17:29 +02:00
Abhishek Kumar
b030ea19a4 Merge remote-tracking branch 'apache/4.17' into 4.18 2023-06-06 13:12:12 +05:30
Abhishek Kumar
f298008215 Merge remote-tracking branch 'apache/4.16' into 4.17 2023-06-06 13:11:24 +05:30
Pearl Dsilva
bd5d478a3a
Refactor test and change IP range (#6512)
* Refactor test and change IP range

* countdown from 254 to allow multiple pseudo public nets counting up from 0

* cleanup

* location of asserts improved

---------

Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-06-05 16:39:33 +02:00
Abhishek Kumar
2d6a069812
server: improve storage GC to skip expunging possible duplicate volumes (#7313)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-05 10:03:24 +02:00
Wei Zhou
b2e9993b0a
vmware: do not create vm datastore when attach an existing volume on solidfire (#7160) 2023-06-01 14:26:57 +02:00
Abhishek Kumar
0941d01e97
secondary-storage: delete backedup snapshot dir on delete (#7524)
* secondary-storage: delete backedup snapshot dir on delete

Fixes #7516

When a backed-up snapshot is deleted and the snapshot file is present in the same name directory (probably only for VMware), the whole directory can be deleted.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* resolve comment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:48:28 +05:30
Abhishek Kumar
7319debc87
cks: k8s cluster on vpc tier (#7479)
* cks: cluster on vpc tier

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* python fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix trailing space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:29:06 +05:30
Abhishek Kumar
ef9dec11e7
guestos: add Windows Server 2022 (#7508)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-31 16:02:48 +02:00
Abhishek Kumar
17e765e6bc
test: reduce PowerMock usage for code coverage (#7452)
JaCoCo used for code coverage calculation in the project doesn't support PowerMockito classes.
This PR attemps to reduce usage of PowerMockito.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-30 11:43:01 +02:00
Abhishek Kumar
f636580195
cks,ui: allow changing stopped cluster offering, improvements (#7475)
* cks,ui: allow changing stopped cluster offering, improvements

Fixes #7454

- Allows changing compute offering for a stopped cluster
- Allows compute offering change when the cluster has autoscaling enabled

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-29 15:58:06 +05:30
Ben
c3535880d2
Create user 'cloud' in cloudstack-usage postinstall (#7559)
This ensures the chown 'cloud:cloud' command (later in the same file)
is always performed with no error.
2023-05-26 12:44:02 +02:00
Rohit Yadav
4138ca7c5d systemvmtemplate: fix Debian 11.7.0 ISO URL
Fixes systemvmtemplate build issue

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-05-25 22:15:28 +05:30
Abhishek Kumar
8849e0f464
server: fix volume detach operation when no vm host (#7526)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-25 15:48:27 +02:00
Rohit Yadav
2b3e2e48a0
ui: show zones and offerings to users (#7557)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-05-25 13:44:44 +02:00
Nicolas Vazquez
5e71cea2c5
Fix Tungsten Fabric errors on adding a new zone due to duplicated entries on ntwk_service_map (#7455)
* Fix Tungsten Fabric errors on adding a new zone due to duplicated entries on ntwk_service_map

* Address review comments
2023-05-24 11:00:29 -03:00
Abhishek Kumar
22871ad2da
ui: fix infocard tempalte, iso fields (#7527)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-12 18:40:11 +02:00
Abhishek Kumar
32caf9057e
engine-storage: fix errored template becomes active (#7485)
* engine-storage: fix errored template becomes active

Fixes #7342

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-12 15:23:16 +02:00
Abhishek Kumar
c47554919c
ui: fix tags for security group rules (#7424)
Fixes #7412

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-12 13:31:57 +02:00
Vishesh
83fd75f707
Marvin: Fix deployDataCenter's cleanup (#7521) 2023-05-12 10:46:52 +02:00
Abhishek Kumar
44024a8a8b
github-actions: fix ui build workflow (#7522)
package-lock.json is not removed.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-12 12:54:23 +05:30
Oscar Sandoval
b6443a2b1f
increase log detail for limit checking, fix getDomainReservation() (#7506)
In troubleshooting ops issues we see logs like:

Maximum domain resource limits of Type 'user_vm' for Domain Id = 763 is exceeded: Domain Resource Limit = (1 bytes) 1, Current Domain Resource Amount = (0 bytes) 0, Requested Resource Amount = (1 bytes) 1."

However there is one missing value (currentResourceReservation) that is used in the calculation of limit check but it is not logged, which leads to confusion. Above we see we are using “0” and requested 1, with our limit being 1, but was rejected. Without logging all the values used in the calculation we don’t understand why it failed.

Additionally, if we had this log above it would be clearer that a second bug is occurring. When we query for domain level resource reservations in “getDomainReservation” the actual SearchBuilder is the listAccountAndTypeSearch, not the listDomainAndTypeSearch. As a result, when we call getDomainReservation the query returns any outstanding domain reservation for any account, as domain ID is not a valid filter for the account search.

This PR:

Increases detailed information in log for checking resource limit to include reservations information for functions: checkDomainResourceLimit() and checkAccountResourceLimit

Fixes getDomainReservation() to use listDomainAndTypeSearch instead of listAccountAndTypeSearch

Co-authored-by: Oscar Sandoval <osandovalocana@apple.com>
2023-05-12 12:53:18 +05:30
Abhishek Kumar
a0eb0aa5f7
api,server: fix VM.CREATE events on vm deploy without start (#7421)
Fixes #6697

Allows the server to generate started and completed events for VM.CREATE event type when VM is deployed with startvm=false.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-11 23:23:00 +05:30
Wei Zhou
8c546bd5a6
router: do not restart conntrackd if no guest networks (issue 6702) (#7265)
This PR fixes #6702

The service conntrackd will be started by the script /opt/cloud/bin/cs/CsRedundant.py, therefore add it to disabled_svcs so it is not started automatically when VR is started.
2023-05-11 12:59:32 +05:30
Abhishek Kumar
d9e0a3e6c3
server: add config SSVM capacity scan (#7448)
Fixes #7438

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-11 12:58:10 +05:30
Wei Zhou
5e442df5a0
ssvm: continue with other secondary storages if fails to program a secondary storage (#7264)
This PR fixes #7244
2023-05-11 12:57:51 +05:30