11233 Commits

Author SHA1 Message Date
Nicolas Vazquez
fe03e306ac
Fix attach volume error for VM - different scope for VM volume and volume disk offering (#5982)
* Fix attach volume error for VM on different storage pool

* Fix typo

* Refactor
2022-02-13 11:06:44 -03:00
davidjumani
a094dbc76e
Add option to infer boot type and mode from template (#5974)
* Add option to infer boot type and mode from template

* Rename to defaultBootType and mode
2022-02-11 15:28:59 -03:00
JoaoJandre
bcd0979a5a
Fixed possible blank domain UUID amd created new unit tests. (#5835)
Co-authored-by: Joao <JoaoJandre@gitlab.com>
2022-02-11 14:05:01 -03:00
Suresh Kumar Anaparti
212a4dedfc
Merge branch '4.16' into main 2022-02-10 20:56:52 +05:30
sureshanaparti
5297b52d98
Improve the guest OS hypervisor mappings addition on upgrade. (#5911)
- This removes the hardcoded guest os ids 305 - 329 in upgrade path 4.14.0 to 4.15.0, and 330 - 349 in upgrade path 4.15.0.0 to 4.15.1.0.
2022-02-10 14:43:03 +01:00
nvazquez
e9d8d7942d
Merge branch '4.16' 2022-02-10 09:48:49 -03:00
dahn
a6d9fa61b9
Role escalation prevention (#5879)
* prevent role access escallation

* hierarchy issue fixed

* create api list in account manager for checking new account access

* full api list check

* strange role restriction removed for BareMetal

* add role check on upfdate account as well

* more selective use of api checkers

* error msg and var name

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2022-02-10 11:50:27 +05:30
Suresh Kumar Anaparti
850714c1fc
Merge branch '4.16' into main 2022-02-10 10:57:29 +05:30
slavkap
4ffb949a58
Fix of revert RBD snapshots (#5544)
* Fix of revert RBD snapshots

If snapshot is taken only on Primary storage with the option "snapshot.backup.to.secondary" set to true, when you set this option to false the revert will fail. Added check if the snapshot is not on Secondary to check for it on Primary

* Check if snapshot is on primary storage

Will check first if the snapshot is on Primary storage, if not will
return Image as data store

* Fix unit tests

* removed unused method's params

* Formatted error message and added the snapshot ID to it

* Return to the old logic, the fix will only apply to RBD

* Formatted Exception's messages
2022-02-10 10:22:21 +05:30
Abhishek Kumar
f88f934274
api, server: fix add-remove vpn user without vpn owner (#5850)
* api, server: fix add-remove vpn user without vpn owner

Fixes #5711

ACS should not add a new user in Add state when the owner account does not have VPN access.
While removing VPN user ACS should not fail completely when owner account ahs no VPN.

* change , fixes

* remove unused method

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-02-10 10:07:23 +05:30
dahn
3fc4ef478d
replace Random with SecureRandom (#5966)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2022-02-09 21:08:33 +05:30
davidjumani
453aeb02f0
Add ID search capability to sshkeypairs (#5963) 2022-02-09 09:34:00 +05:30
Pearl Dsilva
e609aa8e9c
Skip systemVM template registration for Simulator (#5954)
* Skip systemVM template registration for Simulator

* simplify
2022-02-08 19:44:01 +01:00
Suresh Kumar Anaparti
208ae84dd7
Merge branch '4.16' into main 2022-02-08 19:01:34 +05:30
Abhishek Kumar
af58284560
server,config: respect storage.max.volume.size and make it dynamic (#5857)
* server,config: respect storage.max.volume.size and make it dynamic

Fixes #5830

* fix test

* size change

* fix check

* server: donot include ISO size while checking volume sizes

* revert size check

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2022-02-08 13:29:35 +05:30
Gabriel Beims Bräscher
2c11171517
Fix 'endpointe.url' global settings configuration typo (#5832)
* Update 'endpointe.url' global settings to 'endpoint.url'

* Add PR number on 'schema-41610to41700.sql'

* Use ApiServiceConfiguration.ApiServletPath.key() instead of "hardcoded" string
2022-02-06 12:11:45 -03:00
dahn
0f1cd6009d
add logging to deployment planners (#5859)
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2022-02-04 17:02:32 +01:00
Suresh Kumar Anaparti
e3f0e3b544
Merge branch '4.16' into main 2022-02-04 19:04:08 +05:30
Harikrishna
56a8852c4c
Remove code that reserves extra interfaces based on the global setting router.extra.public.nics in case of VMware. (#5912)
Added hot plugging of vifs whenever required in case of isolated networks. VPC networks already uses hotplugging of vifs.
2022-02-04 12:41:56 +01:00
dahn
ef7debb487
Filter usage for project (#5866)
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: José Flauzino <jose.wilson.vf@gmail.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2022-02-04 11:45:32 +01:00
Wei Zhou
194bf6ead9 Merge remote-tracking branch 'apache/4.16' 2022-02-03 20:42:34 +01:00
Wei Zhou
4a1ebb3fa5
server: allow normal users to create isolated network without source nat (#5920) 2022-02-03 16:44:11 +05:30
Abhishek Kumar
638779ca37
vm-import: fix unmanaged instance listing (#5400)
* vm-import: fix unmanaged instance listing

When the host and last host ID is not set for the VM, it may appear in the list of unmanaged instances.
This changes fixes the behaviour by filtering unmanaged instances list for host for following three criteria:
- host is set as host_id for the VM
- host is set as the last_host_id for the VM
- pod of the host is set as the pod_id for the VM and both host_id and last_host_id is NULL

* use SearchBuilder to fix query condition

* add paranthesis

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-02-03 10:27:05 +05:30
Abhishek Kumar
62b806ac13
api,server: add params for updatehypervisorcapabilities API (#5473)
* api,server: add params for updatehypervisorcapabilities API

Allows updating following capabilities for a hypervisor, version:
- Max DATA volumes limit
- Storage motion supported
- Max hosts per cluster
- VM snapshot enabled

* added test

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update test/integration/smoke/test_hypervisor_capabilities.py

Co-authored-by: dahn <daan.hoogland@gmail.com>
2022-02-03 10:21:23 +05:30
Suresh Kumar Anaparti
bf70566c2c
Merge branch '4.16' into main 2022-02-02 17:30:21 +05:30
Abhishek Kumar
8adb8df2fe
server: find suitable disk offering for volume upload (#5852)
* server: find suitable disk offering for volume upload

Fixes #5696

* fix npe check

* fixes, refactor, rename method and handle custom iops

* ui: allow offering selection

* list only disk offerings

* show name

* revert error check

* use checkaccess

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-02-02 16:35:47 +05:30
dahn
c1bba2a308
Do not restart VPC tiers with cleanup (#5873)
* do not restart VPC tiers with cleanup

* no option for cleanup for VPC tiers

* Update server/src/main/java/com/cloud/network/NetworkServiceImpl.java

* paramNames

* remove superfluent parameter

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-31 17:59:26 +05:30
Nicolas Vazquez
3e92a63155
[XenServer/XCP-ng] Pass the image store NFS version on storage commands (#5886)
* Add NFS version to mount command

* Remove extra line

* Extend NFS version to mount secondary storage

* Unused import

* Refactor NFS version to be granular

* Make use of the ConfigKey on the NFS version setting value
2022-01-31 12:21:13 +05:30
Suresh Kumar Anaparti
f610e89ea9
Merge branch '4.16' into main 2022-01-28 13:43:00 +05:30
sureshanaparti
0e150954d0
Allow direct download templates from IPv6 host address. (#5900)
* Allow templates & ISOs from IPv6 host address.

* fix checkstyle issue

* Allow only direct download templates from IPv6 address

Co-authored-by: gabriel <gabriel@apache.org>
2022-01-28 09:50:32 +05:30
Harikrishna
f15cab16da
server: Decouple service (compute) offering and disk offering (#5008)
Currently, our compute offerings and disk offerings are tightly coupled with respect to many aspects. For example, if a compute offering is created, a corresponding disk offering entry is also created with the same ID as the reference. Also creating compute offering takes few disk-related parameters which anyway goes to the corresponding disk offering only. I think this design was initially made to address compute offering for the root volume created from a template. Also changing the offering of a volume is tightly coupled with storage tags and has to be done in different APIs either migrateVolume or resizeVolume. Changing of disk offering should be seamless and should consider new storage tags, new size and place the volume in appropriate state as defined in disk offering.

more details are mentioned here https://cwiki.apache.org/confluence/display/CLOUDSTACK/Compute+offering+and+disk+offering+refactoring

* Schema changes and disk offering column change from "type" to "compute_only"

* Few more changes

* Decoupled service offering and disk offering

* Remove diskofferingid from vminstance VO

* Decouple service offering and disk offering states

* diskoffering getsize() is only for strict disk offerings

* Fix deployVM flow

* Added new API params to compute offering creation

* Add diskofferingstrictness to serviceoffering vo under quota

* Added overrideDiskOfferingId parameter in deploy VM API which will override disk offering for the root disk both in template and ISO case

Added diskSizeStrictness parameter in create Disk offering API which will decide whether to restrict resize or disk offering change of a volume

* Fix User vm response to show proper service offering and disk offerings

* Added disk size strictness in disk offering response

* Added disk offering strictness to the service offering response

* Remove comments

* Added UI changes for Disk offering strictness in add compute offering form and Disk size strictness in add disk offering form

* Added diskoffering details to the service offering response

* Added UI changes in deployvm wizard to accept override disk offering id

* Fix delete compute offering

* Fix VM deployment from custom service offering

* Move uselocalstorage column access from service offering to disk offering

* UI: Separated compute and disk releated parameters in add compute offering wizard, also added association to disk offering

* Fixed diskoffering automatic selection on add compute offering wizard

* UI: move compute only toggle button outside the box in add compute offering wizard

* Added volumeId parameter to listDiskOfferings API and the disksizestrictness flag of the current disk offering is honored while list disk offerings

* Added configuration parameter to decide whether to check volume tags on the destination storagepool during migration

* Added disk offering change checks during resize volume operation

* Added new API changeofferingforVolume API and corresponding changes

* Add UI form for changeOfferingForVolume API

* Fix UI conflicts

* Fix service offering usage as disk offering

* Fix unit test failures

* fix user_vm_view

* Addressed review comments

* Fixed service_offering_view

* Fix service offering edit flow

* Fix service offering constructor to address custom offering

* Fix domain_router_view to get proper service offering id

* Removed unused import

* Addressed review comments and fixed update service offering flow with storage tags

* Added marvin test cases for checking disk offering strictness

* review comments addressed

* Remove system_use column from disk offering join

* update volume_view to update system_use column from service offering and not disk offering

* Fix changeOfferingForVolume API for custom disk offering

* Fix global setting implementation

* Fix list volumes, after changing system_use column from disk offering to service offering in volume_view

* Changes for override root disk offering in deployvm wizard in case of custom offering

* Fix a unit test case

* Fixed recent unit test cases with new serviceofferingvo constructor

* Fix unit test in VolumeApiServiceImpl

* Added storage id for the list disk offering API and corresponding UI changes in migrateVolume and changeOfferingForVolume flow

* Rename global configuration parameter from storage.pool.tags.disk.offering.strictness to match.storage.pool.tags.with.disk.offering

* Fix smoke test failures

* Added tool tip for migrate volume UI form

* Address review comments and fix UI form of deploy VM in case of ISO.

* Fixed resize volume UI form for data disk

* UI changes to disable override root disk size when override root disk offering is enabled

* UI fix in deploy vm wizard

* Fix listdiskoffering after rebasing with main

* Fixed UI in migrate and changeofferingfor volume to handle empty disk offering list
Removed the volume's current disk offering from listDiskOffering response list

* Added custom Iops to resize volume form and removed the current disk offering during change offering for volume UI form

* Fix false response on updateDiskOffering API

* Added search field for changeofferingforvolume UI form

* Fix resize volume and migrate volume to update volume path if DRS is applied on volume in datastore cluster

* Removed DB changes from 4.16 upgrade file

* Resolving merge conflicts with main 4.17

* Added support for auto migration and auto resize of the root volume upon changing the service offering for VM.

* UI: Added automigrate checkbox in scale VM form

* Addes since attributes to new API params

* Added shrinkOK parameter to changeofferingforvolume API

* Added shrinkOk param to UI in changeOfferingforVolume form

* Added shrinkOk flag to scaleVM and changeServiceForVirtualMachines and UI form

* Removed old foreign key constraint on IDs of service offering and disk offering

* Allow resize and automigrate of root volume if required in all cases of service offering change

* Allow only resize to higher disk size from UI

* Fixing vue syntax error

* Make UI changes to provide root disk size box when the linked disk offering is of custom

* Converted from check box to toggle in scale VM, changeoffering, resize and migrate volume forms

* Fix resize volume operation to update the VM settings

* Fix migratevolume form to pick selected storage pool id in list diskofferings API
2022-01-27 15:08:42 +05:30
Daniel Augusto Veronezi Salvador
d26ce157db
Fix camel case (#5898)
Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2022-01-26 19:20:18 -03:00
Suresh Kumar Anaparti
982eef202f
Merge branch '4.16' into main 2022-01-26 12:21:24 +05:30
Nicolas Vazquez
84f5768e64
[VMware][Deploy-as-is] OVF properties not importing when template is uploaded from local (#5861)
* Fix ova upload missing details

* Refactor and cleanup

* Unused import
2022-01-26 11:28:52 +05:30
Suresh Kumar Anaparti
97d6cd50a0
Merge branch '4.16' into main 2022-01-21 19:59:35 +05:30
Wei Zhou
f3289fb596
server: fix regular user can create isolated network without sourcenat (#5844)
* server: fix regular user can create isolated network without sourcenat
2022-01-20 14:10:23 +05:30
Suresh Kumar Anaparti
42a941cc70
Merge branch '4.16' into main 2022-01-13 12:47:08 +05:30
Abhishek Kumar
d78a815442
server: fix update vm with unconstrained offering (#5828)
Fixes #5694

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2022-01-13 12:30:12 +05:30
Wei Zhou
216bea1978
ipv6: disable IPv6-only shared network with VR (#5810)
* ipv6: disable IPv6-only shared network with VR

* Update #5810: import org.apache.commons.lang3.StringUtils
* Update #5810: do not import org.apache.commons.lang3.StringUtils
* Update server/src/main/java/com/cloud/network/NetworkServiceImpl.java
* Update engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-12 13:33:35 +05:30
Suresh Kumar Anaparti
e1b56be67c
Merge branch '4.16' into main 2022-01-11 12:46:33 +05:30
Wei Zhou
9293f5bd27
server: fix enable/disable static nat if userdata is not supported (#5839)
* server: fix enable/disable static nat if userdata is not supported

* Update #5839: rename applyUserData to applyUserDataIfNeeded

* Update server/src/main/java/com/cloud/network/rules/RulesManagerImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-11 12:22:59 +05:30
Wei Zhou
fadd74aaca
network: fix vm can be deployed on L2 network of other accounts (#5784)
* Update #5769: fix domain admin can deploy vm on L2 network of other users

* test: fix test_storage_policy.py

* Update #5784: revert part of changes in #2420
2022-01-11 12:16:00 +05:30
dahn
028d338aaa
remove VmWorkJob after adding a nic to a vm (#5658)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2022-01-10 16:31:50 +01:00
SadiJr
5d7ea30f87
vmware: Remove restriction, in VMWare, to resize only SCSI disks (#5650)
Remove restriction, in VMWare, to resize only SCSI disks, because VMWare supports resize of subtypes of SCSI, like LSILOGIC, LSILOGIC SAS, etc

In VMWare, there are different types of SCSI controllers, such as BusLogic Parallel, LSI Logic Parallel, LSI Logic SAS, etc. All of these sub-types support the disk resize operation, the only exception is the IDE controller. However, in ACS, there is a logical restriction to allow only resizing of volumes of the specific SCSI type; thus, not allowing resizing of the SCSI sub-types. This PR corrects this behavior blocking the resize of volumes which uses IDE controller only.
More information can be found at:

https://www.starwindsoftware.com/blog/storage-controllers-in-vmware-vsphere
https://www.nakivo.com/blog/scsi-controller-and-other-vmware-controller-types/
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-5872D173-A076-42FE-8D0B-9DB0EB0E7362.html
https://communities.vmware.com/t5/ESXi-Discussions/Unable-to-increase-disk-Space/td-p/397823

Co-authored-by: SadiJr <sadi@scclouds.com.br>
2022-01-08 09:02:00 +05:30
Nicolas Vazquez
946f9156e5
vmware: Persist empty values instead of null (#5748)
This PR allows deploying VMware appliances missing default values on some properties
Fixes: #5747
2022-01-08 08:58:46 +05:30
Suresh Kumar Anaparti
5c02f6d507
Merge branch '4.16' into main 2022-01-06 17:47:37 +05:30
dahn
2774bc156f
use physical size instead of virtual size for migration. (#5750)
* Use Physical size to evaluate if migration is possible

* Improve logging and consider files skipped as failure in complete migration

* skipped can't be negative

* remove useless method

* group multidisk templates for secstor migration

* use enum

* Update engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/DataMigrationUtility.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Pearl d'Silva <pearl.dsilva@shapeblue.com>
2022-01-06 17:18:50 +05:30
Abhishek Kumar
51f69f7134
server: do not return inaccessible entity details to normal users (#5827)
Fixes #5534

As pre 3.x APIs allow using internal DB IDs, even normal users can use internal IDs.
This fix removes additional information in error message when the caller doesn't have access to the resource.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-01-06 16:42:57 +05:30
Suresh Kumar Anaparti
30ae9eedad
Merge branch '4.16' into main 2022-01-05 10:14:47 +05:30
Rakesh
2bd1dc1e14
Enable resetting config values to default value (#4230)
* Enable resetting config values to default value

Provide reset button to zone,cluster,domain,account,
primary and secondary storage so that config values
can be reset to default value

* fix ui issue

* Update test/integration/smoke/test_reset_configuration_settings.py

* Update test/integration/smoke/test_reset_configuration_settings.py

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2022-01-03 21:55:30 +01:00