4475 Commits

Author SHA1 Message Date
Nicolas Vazquez
c733a23c90
Fix direct download URL checks (#7693)
This PR fixes the URL check for direct downloads, in the case of HTTPS URLs the certificates were not loaded into the SSL context
2023-07-06 13:47:13 +05:30
Daan Hoogland
fbb9dec6e5 Merge release branch 4.18 to main
* 4.18:
  server: do not check zone imagestores for directdownload template delete (#7607)
  UI: Zone wizard fix (#7588)
2023-07-04 10:17:43 +02:00
SadiJr
3c5fdeafdc
[Veeam] restored VMs without NICs (#6282) 2023-07-03 14:35:48 +02:00
dahn
1275db4081
UI: Zone wizard fix (#7588)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 11:56:52 +02:00
Rohit Yadav
3b054b2665 Merge remote-tracking branch 'origin/4.18' 2023-07-03 13:17:25 +05:30
Vishesh
2fcbe6241f
cks: Add unmanaged kubernetes cluster (#7515)
There are tools like cluster-api which create and manage kubernetes cluster on CloudStack. This PR adds the option to add unmanaged kubernetes cluster which are not managed by CKS plugin. This helps provide a consolidated view of unmanaged clusters on CloudStack. The changes done make sure that operations for managed clusters are not executed for unmanaged clusters.

Two new APIs have also been added:

1. addVirtualMachinesToKubernetesCluster - to add VMs to unmanaged clusters.
2. removeVirtualMachinesFromKubernetesCluster - to remove VMs to unmanaged clusters.

Two APIs have been updated:

1. createKubernetesCluster - made KUBERNETES_VERSION_ID, SERVICE_OFFERING_ID, SIZE as not required for unmanaged clusters. Add an additional parameter, managed, which is true by default.
2. listKubernetesClusters - Add a parameter managed to filter on managed field.

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-07-03 13:07:33 +05:30
Harikrishna
70820137e6
scaleio: Avoid race condition while handling host disconnect and connect scenarios (#282) (#7689)
This PR fixes an intermittent issue where SDC id (local_path) is getting deleted and not getting populated when host connects back again.

Fix is to remove the code to delete the records from storage_pool_host_ref table. We are anyways updating the entry if the SDC ID is changed during agent restart which is anyways required inorder to get the new connections. I've quickly verified the host delete scenario to check the storage_pool_host_ref entries behavior, entries are getting deleted.
2023-07-03 12:58:42 +05:30
Wei Zhou
41403c9cf9 Merge remote-tracking branch 'apache/4.18' 2023-06-27 11:52:37 +02:00
Vishesh
c3718ab92e
Fix storpool test & remove powermock (#7652)
This PR fixes unit tests for plugins/storage/volume/storpool/ and removes powermock.
2023-06-27 14:55:19 +05:30
Vishesh
908b3b7dfa
Remove powermock from ldap & project-role-based plugin (#7658)
This PR removes powermock from tests & fixes the path for tests of project-role-based plugin.
2023-06-27 14:53:00 +05:30
dahn
06caf32bc8
Guest os mappings improvements (#6979)
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI

* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements

* Some static analysis fixes

* Removed commented code in listview

* Guest OS list

* UI changes for adding guest os and mappings

* Added guest os mappings in guest os form

* Added new filter to guest os mapping

* Name and description changes

* VMWare Host and cluster MO unit tests

* CheckGuestOsMapping command and answer unit tests

* GetHypervisorGuestOsNames command and answer unit tests

* VmwareResource unitests

* GuestOsMapper unittests

* icon changes

* Addressed review comments

* Renaming fixes

* Removed comments

* marvin tests for guest os operations

* Added marvin tests for OS mappings

* Document links and UI improvements

* Added deduplication for the list guest OS API

* Fixed linter failure

* Few bug fixes and UI changes

* Few improvements

* Addressed code smells

* Fixed UI issues after rebase

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2023-06-27 13:49:20 +05:30
Wei Zhou
985f0ecb53
Tungsten: change conserve_mode of default network offering to 0 (#7511) 2023-06-26 13:36:36 +02:00
slavkap
faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00
Vishesh
409e3202a3
Fix and enable tests for storage plugins (#7657) 2023-06-23 13:46:14 +02:00
Wei Zhou
5347d067b4
scaleio: fix build error after 4.18 merge forward (#7669) 2023-06-21 20:24:27 +02:00
Wei Zhou
09a4a252d7 Merge remote-tracking branch 'apache/4.18' into HEAD 2023-06-21 15:08:56 +02:00
Harikrishna
40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
Abhishek Kumar
3748f32bc7
engine-orchestration,vmware: hypervisor migration during start vm migration (#7444)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-20 17:13:08 +02:00
Vishesh
27b0a64b40
Fix scaleio tests (#7622)
Co-authored-by: Wei Zhou <weizhou@apache.org>
2023-06-15 11:53:10 +02:00
Abhishek Kumar
5fa9b8f683
cks: enable by default (#7509)
This change will allow CKS to be enabled by default on new installs. It would not affect server or performance but would help highlighting k8s support in CloudStack.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-12 14:11:02 +02:00
Abhishek Kumar
6ea5f56e62 Merge remote-tracking branch 'apache/4.18' into main 2023-06-06 13:12:55 +05:30
Wei Zhou
b2e9993b0a
vmware: do not create vm datastore when attach an existing volume on solidfire (#7160) 2023-06-01 14:26:57 +02:00
Abhishek Kumar
f905f5f39a Merge remote-tracking branch 'apache/4.18' into main 2023-06-01 16:50:24 +05:30
Abhishek Kumar
7319debc87
cks: k8s cluster on vpc tier (#7479)
* cks: cluster on vpc tier

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* python fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix trailing space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-01 15:29:06 +05:30
Abhishek Kumar
17e765e6bc
test: reduce PowerMock usage for code coverage (#7452)
JaCoCo used for code coverage calculation in the project doesn't support PowerMockito classes.
This PR attemps to reduce usage of PowerMockito.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-30 11:43:01 +02:00
Abhishek Kumar
f636580195
cks,ui: allow changing stopped cluster offering, improvements (#7475)
* cks,ui: allow changing stopped cluster offering, improvements

Fixes #7454

- Allows changing compute offering for a stopped cluster
- Allows compute offering change when the cluster has autoscaling enabled

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-29 15:58:06 +05:30
Abhishek Kumar
8ff0071bec
ui,api: make description option for createKubernetesCluster (#7541)
Make the description for createKubernetesCLuster API optional. Similar to #7180

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-05-25 15:22:58 +05:30
nvazquez
0024cb0372
Merge branch '4.18' 2023-05-24 11:01:10 -03:00
Nicolas Vazquez
5e71cea2c5
Fix Tungsten Fabric errors on adding a new zone due to duplicated entries on ntwk_service_map (#7455)
* Fix Tungsten Fabric errors on adding a new zone due to duplicated entries on ntwk_service_map

* Address review comments
2023-05-24 11:00:29 -03:00
Ben
3e8c0684ed
Prometheus: Ensure tagged hosts maintenance status is reported consistently (#7471)
When a host is not tagged, its maintenance status is reported in the
cloudstack_hosts_total metric: maintenance_enabled is OFFLINE,
maintenance_disabled is ONLINE.

When a host is tagged, its maintenance status is now also verified to
ensure consistent behaviour.

In prometheus exporter, maintenance status for cloudstack_hosts_total_by_tag is not checked. While it is checked for cloudstack_hosts_total metric.
Classified by_tag or not, metrics should be the same.

Fixes: #7470
2023-05-23 11:14:43 +05:30
John Bampton
11d45654a6
misc: fix spelling (#7206)
This PR fixes spellings
2023-05-23 11:06:16 +05:30
Wei Zhou
9d46df57f2
kvm: add vm setting for nic multiqueue number and packed virtqueues (#7333)
This PR adds two vm setting for user vms on KVM

- nic multiqueue number
- packed virtqueues enabled . optional are true and false (false by default). It requires qemu>=4.2.0 and libvirt >=6.3.0

Tested ok on ubuntu 22 and rocky 8.4
2023-05-09 15:19:26 +05:30
Rohit Yadav
a2561df25b Merge remote-tracking branch 'origin/4.18' 2023-05-08 12:57:38 +05:30
Marcus Sorensen
ec0f8bddf6
Support local storage live migration for direct download templates (#7453)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-05-04 17:37:58 -03:00
Rohit Yadav
8a42ab9ce4 Merge remote-tracking branch 'origin/4.18' 2023-04-14 21:49:12 +05:30
David Jumani
941cc83372
Feature: Safely shutdown cloudstack (#6755)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-04-12 12:44:14 +02:00
Harikrishna
65512af779
server: Roles search by keyword (#7384)
This PR fixes #7362 and also other search criteria to use the name as an exact search where keyword is also there.

Made UI changes for roles search to make use of keyword instead of name.
2023-04-11 22:24:29 +05:30
Nicolas Vazquez
be66eb2a35
Auto Enable/Disable KVM hosts (#7170)
* Auto Enable Disable KVM hosts

* Improve health check result

* Fix corner cases

* Script path refactor

* Fix sonar cloud reports

* Fix last code smells

* Add marvin tests

* Fix new line on agent.properties to prevent host add failures

* Send alert on auto-enable-disable and add annotations when the setting is enabled

* Address reviews

* Add a reason for enabling or disabling a host when the automatic feature is enabled

* Fix comment on the marvin test description

* Fix for disabling the feature if the admin has manually updated the host resource state before any health check result
2023-04-04 17:03:37 +05:30
Abhishek Kumar
82a6a1f6c4 Merge remote-tracking branch 'apache/4.18' into main 2023-04-04 16:10:12 +05:30
Marcus Sorensen
5d5fa04c8b
saml: Add EncryptedElementType key resolver to SAML plugin (#7268)
There are multiple ways in which a SAML response can be formatted, especially when encryption is enabled. This PR removes the hardcoding of EncryptedKeyResolver= InlineEncryptedKeyResolver in favor of using a ChainingEncryptedKeyResolver which will try multiple resolvers. It preserves the InlineEncryptedKeyResolver as the first option but adds EncryptedElementTypeEncryptedKeyResolver to the chain of resolvers to try.

ChainingEncryptedKeyResolver is a bit finicky in that you can't provide it a list of resolvers, you can only fetch its internal list and add to it.

Theoretically we could add all of the resolver types to the chain, but for now just preserving the ones known to be in use.

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-04-03 15:16:03 +05:30
John Bampton
c2e17310d6
Add three more pre-commit checks (#7083)
Co-authored-by: dahn <daan@onecht.net>
2023-03-27 13:28:55 +02:00
Daan Hoogland
1d0762e43a Merge release branch 4.18 to main
* 4.18:
  Fix PR 7131 bugs and vulnerabilities (#7140)
2023-03-23 15:54:19 +01:00
João Jandre
523ab58d02
Fix PR 7131 bugs and vulnerabilities (#7140) 2023-03-21 15:06:18 +01:00
Daan Hoogland
fb4f6a334d Updating pom.xml version numbers for release 4.19.0.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:46:01 +01:00
Daan Hoogland
05cda2729f Updating pom.xml version numbers for release 4.18.1.0-SNAPSHOT
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-15 19:38:14 +01:00
Daan Hoogland
0574087284 Updating pom.xml version numbers for release 4.18.0.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
2023-03-11 09:35:41 +01:00
Wei Zhou
8592de95fa
Move PassphraseVO to use String instead of byte[] to support Encrypt annotation (#7302)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-03-03 13:08:17 +01:00
Bryan Lima
c8ee0e7c30
Fix Quota plugin state on QuotaSummaryResponse (#7257) 2023-02-22 10:19:59 +01:00
Abhishek Kumar
1a03f69a3a
cleanup: remove testing logs (#7270)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-02-21 14:42:33 +01:00
Wei Zhou
8ef35466de
Tungsten: fix functional issues (#7173)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-02-13 09:15:28 +01:00