900 Commits

Author SHA1 Message Date
Suresh Kumar Anaparti
bf70566c2c
Merge branch '4.16' into main 2022-02-02 17:30:21 +05:30
Abhishek Kumar
8adb8df2fe
server: find suitable disk offering for volume upload (#5852)
* server: find suitable disk offering for volume upload

Fixes #5696

* fix npe check

* fixes, refactor, rename method and handle custom iops

* ui: allow offering selection

* list only disk offerings

* show name

* revert error check

* use checkaccess

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-02-02 16:35:47 +05:30
dahn
c1bba2a308
Do not restart VPC tiers with cleanup (#5873)
* do not restart VPC tiers with cleanup

* no option for cleanup for VPC tiers

* Update server/src/main/java/com/cloud/network/NetworkServiceImpl.java

* paramNames

* remove superfluent parameter

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-31 17:59:26 +05:30
Suresh Kumar Anaparti
f610e89ea9
Merge branch '4.16' into main 2022-01-28 13:43:00 +05:30
sureshanaparti
0e150954d0
Allow direct download templates from IPv6 host address. (#5900)
* Allow templates & ISOs from IPv6 host address.

* fix checkstyle issue

* Allow only direct download templates from IPv6 address

Co-authored-by: gabriel <gabriel@apache.org>
2022-01-28 09:50:32 +05:30
Harikrishna
f15cab16da
server: Decouple service (compute) offering and disk offering (#5008)
Currently, our compute offerings and disk offerings are tightly coupled with respect to many aspects. For example, if a compute offering is created, a corresponding disk offering entry is also created with the same ID as the reference. Also creating compute offering takes few disk-related parameters which anyway goes to the corresponding disk offering only. I think this design was initially made to address compute offering for the root volume created from a template. Also changing the offering of a volume is tightly coupled with storage tags and has to be done in different APIs either migrateVolume or resizeVolume. Changing of disk offering should be seamless and should consider new storage tags, new size and place the volume in appropriate state as defined in disk offering.

more details are mentioned here https://cwiki.apache.org/confluence/display/CLOUDSTACK/Compute+offering+and+disk+offering+refactoring

* Schema changes and disk offering column change from "type" to "compute_only"

* Few more changes

* Decoupled service offering and disk offering

* Remove diskofferingid from vminstance VO

* Decouple service offering and disk offering states

* diskoffering getsize() is only for strict disk offerings

* Fix deployVM flow

* Added new API params to compute offering creation

* Add diskofferingstrictness to serviceoffering vo under quota

* Added overrideDiskOfferingId parameter in deploy VM API which will override disk offering for the root disk both in template and ISO case

Added diskSizeStrictness parameter in create Disk offering API which will decide whether to restrict resize or disk offering change of a volume

* Fix User vm response to show proper service offering and disk offerings

* Added disk size strictness in disk offering response

* Added disk offering strictness to the service offering response

* Remove comments

* Added UI changes for Disk offering strictness in add compute offering form and Disk size strictness in add disk offering form

* Added diskoffering details to the service offering response

* Added UI changes in deployvm wizard to accept override disk offering id

* Fix delete compute offering

* Fix VM deployment from custom service offering

* Move uselocalstorage column access from service offering to disk offering

* UI: Separated compute and disk releated parameters in add compute offering wizard, also added association to disk offering

* Fixed diskoffering automatic selection on add compute offering wizard

* UI: move compute only toggle button outside the box in add compute offering wizard

* Added volumeId parameter to listDiskOfferings API and the disksizestrictness flag of the current disk offering is honored while list disk offerings

* Added configuration parameter to decide whether to check volume tags on the destination storagepool during migration

* Added disk offering change checks during resize volume operation

* Added new API changeofferingforVolume API and corresponding changes

* Add UI form for changeOfferingForVolume API

* Fix UI conflicts

* Fix service offering usage as disk offering

* Fix unit test failures

* fix user_vm_view

* Addressed review comments

* Fixed service_offering_view

* Fix service offering edit flow

* Fix service offering constructor to address custom offering

* Fix domain_router_view to get proper service offering id

* Removed unused import

* Addressed review comments and fixed update service offering flow with storage tags

* Added marvin test cases for checking disk offering strictness

* review comments addressed

* Remove system_use column from disk offering join

* update volume_view to update system_use column from service offering and not disk offering

* Fix changeOfferingForVolume API for custom disk offering

* Fix global setting implementation

* Fix list volumes, after changing system_use column from disk offering to service offering in volume_view

* Changes for override root disk offering in deployvm wizard in case of custom offering

* Fix a unit test case

* Fixed recent unit test cases with new serviceofferingvo constructor

* Fix unit test in VolumeApiServiceImpl

* Added storage id for the list disk offering API and corresponding UI changes in migrateVolume and changeOfferingForVolume flow

* Rename global configuration parameter from storage.pool.tags.disk.offering.strictness to match.storage.pool.tags.with.disk.offering

* Fix smoke test failures

* Added tool tip for migrate volume UI form

* Address review comments and fix UI form of deploy VM in case of ISO.

* Fixed resize volume UI form for data disk

* UI changes to disable override root disk size when override root disk offering is enabled

* UI fix in deploy vm wizard

* Fix listdiskoffering after rebasing with main

* Fixed UI in migrate and changeofferingfor volume to handle empty disk offering list
Removed the volume's current disk offering from listDiskOffering response list

* Added custom Iops to resize volume form and removed the current disk offering during change offering for volume UI form

* Fix false response on updateDiskOffering API

* Added search field for changeofferingforvolume UI form

* Fix resize volume and migrate volume to update volume path if DRS is applied on volume in datastore cluster

* Removed DB changes from 4.16 upgrade file

* Resolving merge conflicts with main 4.17

* Added support for auto migration and auto resize of the root volume upon changing the service offering for VM.

* UI: Added automigrate checkbox in scale VM form

* Addes since attributes to new API params

* Added shrinkOK parameter to changeofferingforvolume API

* Added shrinkOk param to UI in changeOfferingforVolume form

* Added shrinkOk flag to scaleVM and changeServiceForVirtualMachines and UI form

* Removed old foreign key constraint on IDs of service offering and disk offering

* Allow resize and automigrate of root volume if required in all cases of service offering change

* Allow only resize to higher disk size from UI

* Fixing vue syntax error

* Make UI changes to provide root disk size box when the linked disk offering is of custom

* Converted from check box to toggle in scale VM, changeoffering, resize and migrate volume forms

* Fix resize volume operation to update the VM settings

* Fix migratevolume form to pick selected storage pool id in list diskofferings API
2022-01-27 15:08:42 +05:30
Daniel Augusto Veronezi Salvador
d26ce157db
Fix camel case (#5898)
Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2022-01-26 19:20:18 -03:00
Suresh Kumar Anaparti
982eef202f
Merge branch '4.16' into main 2022-01-26 12:21:24 +05:30
Nicolas Vazquez
84f5768e64
[VMware][Deploy-as-is] OVF properties not importing when template is uploaded from local (#5861)
* Fix ova upload missing details

* Refactor and cleanup

* Unused import
2022-01-26 11:28:52 +05:30
Suresh Kumar Anaparti
97d6cd50a0
Merge branch '4.16' into main 2022-01-21 19:59:35 +05:30
Wei Zhou
f3289fb596
server: fix regular user can create isolated network without sourcenat (#5844)
* server: fix regular user can create isolated network without sourcenat
2022-01-20 14:10:23 +05:30
Suresh Kumar Anaparti
42a941cc70
Merge branch '4.16' into main 2022-01-13 12:47:08 +05:30
Abhishek Kumar
d78a815442
server: fix update vm with unconstrained offering (#5828)
Fixes #5694

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2022-01-13 12:30:12 +05:30
Wei Zhou
216bea1978
ipv6: disable IPv6-only shared network with VR (#5810)
* ipv6: disable IPv6-only shared network with VR

* Update #5810: import org.apache.commons.lang3.StringUtils
* Update #5810: do not import org.apache.commons.lang3.StringUtils
* Update server/src/main/java/com/cloud/network/NetworkServiceImpl.java
* Update engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-12 13:33:35 +05:30
Suresh Kumar Anaparti
e1b56be67c
Merge branch '4.16' into main 2022-01-11 12:46:33 +05:30
Wei Zhou
9293f5bd27
server: fix enable/disable static nat if userdata is not supported (#5839)
* server: fix enable/disable static nat if userdata is not supported

* Update #5839: rename applyUserData to applyUserDataIfNeeded

* Update server/src/main/java/com/cloud/network/rules/RulesManagerImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-01-11 12:22:59 +05:30
Wei Zhou
fadd74aaca
network: fix vm can be deployed on L2 network of other accounts (#5784)
* Update #5769: fix domain admin can deploy vm on L2 network of other users

* test: fix test_storage_policy.py

* Update #5784: revert part of changes in #2420
2022-01-11 12:16:00 +05:30
dahn
028d338aaa
remove VmWorkJob after adding a nic to a vm (#5658)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2022-01-10 16:31:50 +01:00
SadiJr
5d7ea30f87
vmware: Remove restriction, in VMWare, to resize only SCSI disks (#5650)
Remove restriction, in VMWare, to resize only SCSI disks, because VMWare supports resize of subtypes of SCSI, like LSILOGIC, LSILOGIC SAS, etc

In VMWare, there are different types of SCSI controllers, such as BusLogic Parallel, LSI Logic Parallel, LSI Logic SAS, etc. All of these sub-types support the disk resize operation, the only exception is the IDE controller. However, in ACS, there is a logical restriction to allow only resizing of volumes of the specific SCSI type; thus, not allowing resizing of the SCSI sub-types. This PR corrects this behavior blocking the resize of volumes which uses IDE controller only.
More information can be found at:

https://www.starwindsoftware.com/blog/storage-controllers-in-vmware-vsphere
https://www.nakivo.com/blog/scsi-controller-and-other-vmware-controller-types/
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-5872D173-A076-42FE-8D0B-9DB0EB0E7362.html
https://communities.vmware.com/t5/ESXi-Discussions/Unable-to-increase-disk-Space/td-p/397823

Co-authored-by: SadiJr <sadi@scclouds.com.br>
2022-01-08 09:02:00 +05:30
Nicolas Vazquez
946f9156e5
vmware: Persist empty values instead of null (#5748)
This PR allows deploying VMware appliances missing default values on some properties
Fixes: #5747
2022-01-08 08:58:46 +05:30
Suresh Kumar Anaparti
5c02f6d507
Merge branch '4.16' into main 2022-01-06 17:47:37 +05:30
Abhishek Kumar
51f69f7134
server: do not return inaccessible entity details to normal users (#5827)
Fixes #5534

As pre 3.x APIs allow using internal DB IDs, even normal users can use internal IDs.
This fix removes additional information in error message when the caller doesn't have access to the resource.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-01-06 16:42:57 +05:30
Suresh Kumar Anaparti
30ae9eedad
Merge branch '4.16' into main 2022-01-05 10:14:47 +05:30
Rakesh
2bd1dc1e14
Enable resetting config values to default value (#4230)
* Enable resetting config values to default value

Provide reset button to zone,cluster,domain,account,
primary and secondary storage so that config values
can be reset to default value

* fix ui issue

* Update test/integration/smoke/test_reset_configuration_settings.py

* Update test/integration/smoke/test_reset_configuration_settings.py

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2022-01-03 21:55:30 +01:00
Rohit Yadav
204fd5cda8 merge: fix forward merge import issue
Fixes build failure on main branch

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-12-30 17:25:59 +05:30
Rohit Yadav
c84198d76d Merge remote-tracking branch 'origin/4.16'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-12-30 16:52:30 +05:30
SadiJr
d8d5665903
api: Allow users to list VMs by Backup Offering Id (#5679)
Co-authored-by: SadiJr <sadi@scclouds.com.br>
2021-12-30 16:20:33 +05:30
Wei Zhou
8952cd5955
kvm: add hosts using cloudstack ssh private key (ccc21 hackathon) (#5684)
This PR provides the option to add kvm hosts with empty or wrong password.
To support this, the cloudstack ssh public key needs to be added in the ~/.ssh/authorized_keys on host.

Feature work:

1. get cloudstack public key from management server (/var/lib/cloudstack/management/.ssh/id_rsa.pub)

![image](https://user-images.githubusercontent.com/57355700/141449653-85f644b5-c32e-44ca-9c6b-77570262c046.png)


2. add the key to ~/.ssh/authorized_keys on kvm hosts

![image](https://user-images.githubusercontent.com/57355700/141449722-e906eea5-74fd-4f81-a4d3-41563beeb79c.png)

3. add kvm host with empty password

<img src="https://user-images.githubusercontent.com/57355700/141449865-6ffee1f0-b0d7-4ea4-b11a-32df42e2fe91.png" height="50%" width="50%">

Tested as:
1. add host, with correct password,  works as expected.
2. put host to maitenance, stop cloudstack-agent, cancel maintenance, remove the host. all work.
3. add host, with empty password (ssh public key is added to  ~/.ssh/authorized_keys),   works as expected.
4. put host to maitenance, stop cloudstack-agent, cancel maintenance. all work as expected.
2021-12-30 16:20:04 +05:30
Jayanth Reddy
b63498d847
server: Fix a typo (#5761)
Fix a typo where the word "Security" is misspelled as "Secrity".
2021-12-30 16:17:15 +05:30
Nicolas Vazquez
c0ddda361a
[VMware] Fix service offerings listing on appliances deployment options (#5752)
* Fix list service offerings for VMware deployment options

* UI fix to allow selecting offerings
2021-12-30 16:13:41 +05:30
Marcus Sorensen
dcdcd09058
Randomize managed volume copy host (#5789)
* Randomize managed volume copy host

* Managed volume copy was always returning first host that could see storage pools

* Fix null value in logging for ScaleIOPrimaryDataStoreDriver due to if/else logic error

Signed-off-by: Marcus Sorensen <mls@apple.com>

* Use String.format for ScaleIO debug message

Signed-off-by: Marcus Sorensen <mls@apple.com>

* Update debug message for ScaleIO copy methods

Signed-off-by: Marcus Sorensen <mls@apple.com>

Co-authored-by: Marcus Sorensen <mls@apple.com>
2021-12-30 16:11:00 +05:30
Suresh Kumar Anaparti
0c0254d8f5
Merge branch '4.16' into main 2021-12-28 01:40:23 +05:30
Wei Zhou
7823868746
network: update ip in lb/pf/dnat tables when update vm nic ip (#5757)
* network: update ip in lb/pf/dnat tables when update vm nic ip

* Update: create 3 separated methods for DNAT/LB/PF update

* Update: invert if condition to reduce code indentation
2021-12-28 01:35:29 +05:30
Rakesh
8ba5b92a83
server: Fix NPE while deleting a domain (#5753)
* server: Fix NPE while deleting a domain

While deleting a domain, if vlan ip range cant be found
then this will throw NPE. Just return false if vlan ip
range cant be found

* return true if vlan is not found

* change output message

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-12-28 01:21:43 +05:30
Rakesh
9607ae9639
server: Fix NPE while adding network to VPC (#5723)
* server: Fix NPE while adding network to VPC

Sometimes adding network to VPC fails because of libirt error.
In that case the new network wont have any VR's associated with it
and hence it will return null.

* use collectionutils

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-12-25 19:57:18 +05:30
sureshanaparti
9b0a1418ec
Provision to sort ISOs from UI, and Updated Templates/ISOs API response to return in the order of sortkey (#5800) 2021-12-24 12:44:50 +05:30
Daan Hoogland
2a331b3742 Merge release branch 4.16 to main
* 4.16:
  Allow force reboot VM from user account, to start VM on the same host (#5791)
  api: Fix search cluster by name (#5782)
  Enhance log message in FirstFitPlanner (#5762)
2021-12-22 18:03:04 +01:00
sureshanaparti
39e41f6b6e
Allow force reboot VM from user account, to start VM on the same host (#5791) 2021-12-22 17:43:02 +01:00
Pearl Dsilva
936ebbb90f
api: Fix search cluster by name (#5782) 2021-12-22 10:41:06 +05:30
Gabriel Beims Bräscher
995ce06cb7
Enhance log message in FirstFitPlanner (#5762)
* Enhance log message in FirstFitPlanner

When cluster reached capacity threshold the message is:
"capacity threshold defined at each cluster/ at global value for capacity Type : 0"

Admins hardly remember the Capacity Type and it can take a while to look at which is the resource for the respective ID. This enhancement addes log message pointing to the capacity name (e.g. Memory / CPU) as well as global settings parameter name and value to be looked at.

* Change formatation in String 'warnMessageForClusterReachedCapacityThreshold'
2021-12-22 10:10:40 +05:30
Wei Zhou
51d4e5475f Merge remote-tracking branch 'apache/4.16' 2021-12-10 17:50:25 +01:00
SadiJr
9363f6490b
Fix full backup VM restore (#5680)
Co-authored-by: SadiJr <sadi@scclouds.com.br>
2021-12-10 07:16:42 +01:00
dahn
a47e53fcee
internal ref replaced by uuid (#5738)
* internal ref replaced by uuid

* remove all internal DB IDs from usage records

* remove superfluent .append(")")

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-12-07 10:09:38 +05:30
José Flauzino
28385be609
Fix metrics stats for VMs not running (#5633)
* Fix metrics stats for VMs that are not running

* Improves the way to get vmIdsToRemoveStats

* Improves test

Co-authored-by: José Flauzino <jose@scclouds.com.br>
2021-12-06 11:06:10 -03:00
Rohit Yadav
2e9c9417be Merge remote-tracking branch 'origin/4.16' 2021-12-03 15:41:56 +05:30
Wei Zhou
a277e4e08c
server: fix non-root users are able to list system networks by id (#5729) 2021-12-03 10:36:33 +01:00
Wei Zhou
4d99f84273
server: update capacity_state of host cpu core after disable/enable a host (#5701) 2021-12-03 06:45:53 +01:00
davidjumani
dfd75d7bc0
api: Fix wrong logger class in *Cmd.java (#5740)
This PR changes the incorrect class name used in the logger in ListHypervisorsCmd
2021-12-02 15:16:28 +05:30
Abhishek Kumar
2df82d8188
ui: changes in migrate vm storage and migrate volume form (#5145)
Better forms in UI for migrating VMs and volumes.

- Show option to migrate with storage while live migrating a VM
- For VM storage migration (stopped VM), allow migrating volumes to specific primary storages
- Show primary storage details in migrate volume form

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-11-30 17:07:48 +05:30
José Flauzino
80cb361b97
Handle NullPointerException when sending email alerts (#5649)
* Handle NullPointerException when sending email

* Improve the log message related to the recipient list is empty

Co-authored-by: José Flauzino <jose@scclouds.com.br>
2021-11-23 16:53:28 +05:30