11868 Commits

Author SHA1 Message Date
Vishesh
bcbf152a05
Merge branch '4.18' into 4.19 2024-06-28 20:14:21 +05:30
Abhisar Sinha
644f3a3f48
Add, Delete Storage Pool commands should be able execute on a host in maintenance (#9301)
* Restart agent when host comes out of maintenance

* Don't send CreateStoragePoolCommand to hosts in maintenance mode

* CreateStoragePoolCommand can run when host in maintenance. Reverted the change to restart agent when host was already up and in maintenance

* Reverted changes done to ResourceManagerImplTest
2024-06-28 18:18:08 +05:30
Abhishek Kumar
c7f1ba5b8e
server: refactor listNetworks api database retrievals (#9184)
* server: refactor listNetworks api database retrievals

* fixes

* remove unused methods

* imports

* fix empty searchcriteria issue

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-28 17:19:32 +05:30
Abhisar Sinha
646c894ec6
Fix for race when automatically assigning IP to Vms (#9240)
* Fix for race when automatically assigning IP to Vms

* code refactor
2024-06-28 17:11:16 +05:30
Vishesh
d4446ee09d
Revert listAll behaviour for templates (#9309) 2024-06-28 15:58:44 +05:30
GaOrtiga
a5c8bb3bfc
server: apply rules when VR of Domain VPC is recreated (#8354)
* apply rules when VR is recreated

* Apply suggestions from code review

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

Co-authored-by: Gabriel <gabriel.fernandes@scclouds.com.br>
2024-06-28 15:46:20 +05:30
Vishesh
a4e9d7f21a
Change vm.stats.remove.batch.size to delete.batch.query.size & allow delete of volume_stats in batches (#9283)
* Change vm.stats.remove.batch.size to delete.batch.query.size

* Add support for deletion of volume stats in batches

* Update server/src/main/java/com/cloud/configuration/ConfigurationManagerImpl.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update server/src/main/java/com/cloud/configuration/ConfigurationManagerImpl.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update configkey description

* Address comments

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-28 15:32:49 +05:30
Suresh Kumar Anaparti
46f672563e
Improve migration of external VMware VMs into KVM cluster (#8815)
* Create/Export OVA file of the VM on external vCenter host, to temporary conversion location (NFS)

* Fixed ova issue on untar/extract ovf from ova file
"tar -xf" cmd on ova fails with "ovf: Not found in archive" while extracting ovf file

* Updated VMware to KVM instance migration using OVA

* Refactoring and cleanup

* test fixes

* Consider zone wide pools in the destination cluster for instance conversion

* Remove local storage pool support as temporary conversion location
- OVA export not possible as the pool is not accessible outside host, NFS pools are supported.

* cleanup unused code

* some improvements, and refactoring

* import nic unit tests

* vmware guru unit tests

* Separate clone VM and create template file for VMware migration
- Export OVA (of the cloned VM) to the conversion location takes time.
- Do any validations with cloned VM before creating the template (and fail early).
- Updated unit tests.

* Check conversion support on host before clone vm / create template on vmware (and fail early)

* minor code improvements

* Auto select the host with instance conversion capability

* Skip instance conversion supported response param for non-KVM hosts

* Show supported conversion hosts in the UI

* Skip persistence map update if network doesn't exist

* Added support to export OVA from KVM host, through ovftool (when installed in KVM host)

* Updated importvm api param 'usemsforovaexport' to 'forcemstodownloadvmfiles', to be generic

* Updated hardcoded UI messages with message labels

* Updated UI to support importvm api param - forcemstodownloadvmfiles

* Improved instance conversion support checks on ubuntu hosts, and for windows guest vms

* Use OVF template (VM disks and spec files) for instance conversion from VMware, instead of OVA file
 - this would further increase the migration performance (as it reduces the time for OVA preparation / archiving of the VM files into a single file)

* OVF export tool parallel threads code improvements

* Updated 'convert.vmware.instance.to.kvm.timeout' config default value to 3 hrs

* Config values check & code improvements

* Updated import log, with time taken and vm details

* Support for parallel downloads of VMware VM disk files while exporting OVF from MS, and other changes below.
- Skip clone for powered off VMs
- Fixes to support standalone host (with its default datacenter)
- Some code improvements

* rebase fixes

* rebase fixes

* minor improvement

* code improvements - threads configuration, and api parameter changes to import vm files

* typo fix in error msg
2024-06-27 21:14:13 +05:30
Abhishek Kumar
b22315db85
server: event for HA vm start (#9202) 2024-06-26 15:38:47 +05:30
Abhishek Kumar
7a8066d3ca
server: fix resource count updated unmanage vm (#8169)
Do not decrement resource count for a running VM twice while unmanaging a VM
2024-06-26 14:34:32 +05:30
slavkap
f7f7013ed0
Added ostypeid in the snapshot response (#9150) 2024-06-26 14:18:38 +05:30
Vishesh
4f85c3fd23
Honour listall param for list templates api (#9300)
* Honour listall param for list templates api

* Don't show templates from another project inside a project view
2024-06-26 12:09:56 +05:30
Vishesh
dc74d5ba88
Let network guru decide if ipv6 cidr size can't be equal to 64 (#9289) 2024-06-26 02:43:26 +05:30
Vishesh
2bf36ef935
DRS: Ensure the destination host is part of the same cluster (#9245)
* DRS: Ensure the destination host is part of the same cluster

* Add and fix unit tests
2024-06-26 02:09:26 +05:30
Abhisar Sinha
4eb43651e2
Ability to specify NFS mount options while adding a primary storage and modify them on a pre-existing primary storage (#8947)
* Ability to specify NFS mount options while adding a primary storage and modify it later

* Pull 8947: Rename all occurrence of nfsopt to nfsMountOpt and added nfsMountOpts to ApiConstants

* Pull 8947: Refactor code - move into separate methods

* Pull 8947: CollectionsUtils.isNotEmpty and switch statement in LibvirtStoragePoolDef.java

* Pull 8947: UI - cancel maintainenace will remount the storage pool and apply the options

* Pull 8947: UI - moved edit NFS mount options to edit Primary Storage form

* Pull 8947: UI - moved 'NFS Mount Options' to below 'Type' in dataview

* Pull 8947: Fixed message in AddPrimaryStorage.vue

* Pull 8947: Convert _nfsmountOpts to Set in libvirtStoragePoolDef

* Pull 8947: Throw exception and log error if mount fails due to incorrect mount option

* Pull 8947: Added UT and moved integration test to component/maint

* Pull 8947: Review comments

* Pull 8947: Removed password from integration test

* Pull 8947: move details allocation to inside the if loop in getStoragePoolNFSMountOpts

* Pull 8947: Fixed a bug in AddPrimaryStorage.vue

* Pull 8947: Pool should remain in maintenance mode if mount fails

* Pull 8947: Removed password from integration test

* Pull 8947: Added UT

* Pull 8875: Fixed a bug in CloudStackPrimaryDataStoreLifeCycleImplTest

* Pull 8875: Fixed a bug in LibvirtStoragePoolDefTest

* Pull 8947: minor code restructuring

* Pull 8947 : added some ut for coverage

* Fix LibvirtStorageAdapterTest UT
2024-06-25 23:45:35 +05:30
Rene Glover
6ee6603359
Updates to HPE-Primera and Pure FlashArray Drivers to use Host-based VLUN Assignments (#8889)
* Updates to change PUre and Primera to host-centric vlun assignments; various small bug fixes

* update to add timestamp when deleting pure volumes to avoid future conflicts

* update to migrate to properly check disk offering is valid for the target storage pool

* Updates to change PUre and Primera to host-centric vlun assignments; various small bug fixes

* update to add timestamp when deleting pure volumes to avoid future conflicts

* update to migrate to properly check disk offering is valid for the target storage pool

* improve error handling when copying volumes to add precision to which step failed

* rename pure volume before delete to avoid conflicts if the same name is used before its expunged on the array

* remove dead code in AdaptiveDataStoreLifeCycleImpl.java

* Fix issues found in PR checks

* fix session refresh TTL logic

* updates from PR comments

* logic to delete by path ONLY on supported OUI

* fix to StorageSystemDataMotionStrategy compile error

* change noisy debug message to trace message

* fix double callback call in handleVolumeMigrationFromNonManagedStorageToManagedStorage

* fix for flash array delete error

* fix typo in StorageSystemDataMotionStrategy

* change copyVolume to use writeback to speed up copy ops

* remove returning PrimaryStorageDownloadAnswer when connectPhysicalDisk returns false during KVMStorageProcessor template copy

* remove change to only set UUID on snapshot if it is a vmSnapshot

* reverting change to UserVmManagerImpl.configureCustomRootDiskSize

* add error checking/simplification per comments from @slavkap

* Update engine/storage/datamotion/src/main/java/org/apache/cloudstack/storage/motion/StorageSystemDataMotionStrategy.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* address PR comments from @sureshanaparti

---------

Co-authored-by: GLOVER RENE <rg9975@cs419-mgmtserver.rg9975nprd.app.ecp.att.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-25 10:35:39 +05:30
Rohit Yadav
42e71175d7 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-06-24 20:39:55 +05:30
Abhishek Kumar
de266a4cb8
api,server: allow updating hypervisor capabilities with hypervisor and version (#8475)
* api,server: allow updating hypervisor capabilities with hypervisor and
version

hypervisor and hypervisorversion parameter added to the
updateHypervisorCapabilities API.

* param description

* Update server/src/main/java/com/cloud/server/ManagementServerImpl.java

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-06-24 14:41:01 +05:30
Wei Zhou
fcfa371337
kvm: support Rocky/RHEL/OL/Alma in the same cluster (#8641)
* kvm: support Rocky/RHEL/OL/Alma in the same cluster

* Update PR#8641: add unit tests
2024-06-24 13:10:41 +05:30
Suresh Kumar Anaparti
c17aa0d9ad
Import Remote KVM VM logging improvements (#9284) 2024-06-24 11:34:37 +05:30
dahn
7e00678691
set isSystem for SVM IPs (#9281)
* set isSystem for SSVM IPs

* Revert "set isSystem for SSVM IPs"

This reverts commit 4ba71b3d6b4ea377f6778d166ff9da7d41e5d007.

* set isSystem flag for SSVMs
2024-06-23 19:36:45 +05:30
Vishesh
59e9ab9efe
Fix volume response for service offering with disk offering (#9273) 2024-06-22 11:17:11 +05:30
Wei Zhou
313a165e62
server: add global setting consoleproxy.sslEnabled (#8809) 2024-06-21 15:56:20 +05:30
dahn
60f234c682
remove Project Template Permissions inhibition (#9196) 2024-06-21 15:20:49 +05:30
Gabriel Pordeus Santos
083ac069ca
fix assignvm template permission check (#8886) 2024-06-21 10:58:46 +02:00
Suresh Kumar Anaparti
5ab23cd9c9
Timeout config to copy the disks of remote KVM instance while importing the instance from an external host (#9213)
* Added timeout config to copy the disks of remote KVM instance while importing the instance from an external host

* Updated copy config units to mins

* Cleanup remote converted file and local file when copy failed
2024-06-21 10:28:18 +05:30
João Jandre
ae3fa5d0de
Add configuration to limit the number of rows deleted from vm_stats (#8740)
Co-authored-by: João Jandre <joao@scclouds.com.br>
2024-06-20 14:26:36 +02:00
Wei Zhou
81269ad62a
VNF: add API listVnfAppliances (#8994) 2024-06-19 19:49:21 +05:30
Harikrishna
2315a73a20
User friendly name of Downloaded Templates Volumes and ISOs (#9252) 2024-06-19 12:47:43 +02:00
Suresh Kumar Anaparti
8b02624e61
User data content size validation, and related code improvements (#8418)
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-06-18 20:07:13 +02:00
Daan Hoogland
050ee44137 Merge branch '4.18' into 4.19 2024-06-18 16:05:45 +02:00
Vishesh
6c7426e3a7
Prevent network disruption on adding a VPC tier for redundant VRs (#9251) 2024-06-18 13:47:45 +02:00
dahn
7c5b7ca077
Extra parameter for UpdateImageStore (#8941)
* Extra parameter for UpdateImageStore

* add name parameter

* ui

* cleanup

* update DB from storage stats results
2024-06-18 12:31:17 +05:30
Harikrishna
bb0c1f93af
Add volume encryption checks during the disk offering change (#9209) 2024-06-17 10:36:47 +02:00
Bryan Lima
00fe25ab01
Fix allocation of VMs with multiple clusters (#8611)
* Fix allocation of VMs with multiple clusters

* Readd debug guard
2024-06-14 13:54:01 +03:00
dahn
ed86dc973b
protect against missing service offering (#9235)
* protect agains missing service offering

* search removed before assuming none

* import

* javadoc
2024-06-14 15:25:46 +05:30
Abhishek Kumar
ce9b2c52f3
cks: fix events (#9070)
Fixes #8043

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-14 12:22:39 +05:30
Abhishek Kumar
df5c546559
server: remove username required condition for listVmsForImport (#9124)
* server: remove username required condition for listVmsForImport

Fixes #8414

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* revert

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-14 12:14:59 +05:30
Rohit Yadav
2ca0857bd5
api: listVM API improvement followup, change returning of stats detail (#9177)
- Changes behaviour of details param handling via global setting:
  - listVirtualMachines API: when the details param is not provided, it returns whether stats are returned controlled by a new global setting `list.vm.default.details.stats`
  - listVirtualMachinesMetrics API: when the details param is not provided, it uses `all` details including `stats`
- Users who are affected slow performance of the listVirtualMachines API response time can set `list.vm.default.details.stats` to `false`
- Remove ConfigKey vm.stats.increment.metrics.in.memory which was renamed to `vm.stats.increment.metrics` in #5984 and also remove unused/unnecessary global settings via upgrade path
- Changes default value of VM stats accumulation setting `vm.stats.increment.metrics` to false until a better solution emerges. Since #5984, this is true and during the execution of listVM APIs the stats are clubbed/calculated which can immensely slow down list VM API calls. Any costly operations such as summing of stats shouldn't be done during the course of a synchronous API, such as the list VM API.
- Fix UI that uses listVirtualMachinesMetrics to not call `stats` detail when in list view without metrics selected.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-06-14 11:03:14 +05:30
Harikrishna
517cddcb15
Fix error message if specific host does not have capacity (#9218) 2024-06-13 12:38:51 +05:30
Vishesh
c756e4370a
List volumes by service offering id (#9211)
* Allow listing of volumes by service offering id

* Address comments
2024-06-12 16:13:56 +05:30
dahn
4de975ff25
list by isEncrypted (#8643)
* list by isEncrypted

* use filter on VO and cleanup

* add encryption type to volume response

* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ListVolumesCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-12 15:11:00 +05:30
Suresh Kumar Anaparti
503ae64a3d
Upload volume format check with the supported hypervisor, fail if supported hypervisor not found on zone (#9224) 2024-06-12 11:40:29 +02:00
Pearl Dsilva
0f770194df
Configure config drive ISO with password file on reinstallation of VM with password-enabled template (#9113)
* Configure config drive ISO with password file on reinstallation of VM with password-enabled template

* address comment
2024-06-12 12:00:08 +03:00
Abhishek Kumar
1577218999
server,event: fix events for storage maintenance (#9056)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-12 09:43:13 +02:00
Abhishek Kumar
ca59720015
vpc,event: fix events for createVpc (#9055)
Fixes #8496

* vpc,event: fix events for createVpc
2024-06-12 12:56:08 +05:30
Suresh Kumar Anaparti
4e7c6682fd
While starting VM with considerlasthost enabled, don't load host tags/details for the last host when it doesn't exist (#9037) 2024-06-12 07:49:03 +02:00
Wei Zhou
edf7394919
server: search template name and display text if keyword is passed (#9215) 2024-06-11 19:35:43 +05:30
Suresh Kumar Anaparti
2e3f76ec03
Improve error messaging / logs when listing VMs on the remote KVM host (for import) (#9204) 2024-06-11 14:48:21 +02:00
Harikrishna
6edcf32c5e
Fix vsphere storage policy when compute offering uses the disk offering attached to it (#9203) 2024-06-11 14:49:58 +03:00