11192 Commits

Author SHA1 Message Date
Pearl Dsilva
37761c610a
travis: fix consistent failures noticed on few tests (#5399) 2021-09-02 15:10:53 +05:30
Pearl Dsilva
557dc5e1a0
api: List details of template download state for stores corresponding to a zone (#5379)
* api: List details of template download state for stores corresponding to a zone

* fix test
2021-09-02 10:58:58 +05:30
Wei Zhou
4e53997ca2
server: do not remove volume from DB if fail to expunge it from primary storage or secondary storage (#5373)
* server: do not remove volume from DB if fail to expunge it from primary storage or secondary storage

* server/VolumeApiServiceImpl.java: move to method

* update #5373
2021-08-31 13:48:58 -03:00
Abhishek Kumar
73cabcd641
xcp-ng: allow passing vm boot options (#5335)
* xenserver: honor vm boot details

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: allow boot option selection for xenserver

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix case

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* host uefi capability

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add detail only if uefi supported

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update host detail

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix version comparison

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-31 16:10:12 +05:30
Abhishek Kumar
fc213ac9fd
server: improve attach volume in specific cases (#5371)
* server: improve attach volume in specific cases

When a VM is in stopped state and host for it is not found then server skips sending AttachCommand to hypervisor.
Change tries to improve this case and finds a suitable host in the VM's cluster when volume is not in a HOST scope pool.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* typo

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2021-08-30 22:27:45 -03:00
nicolas
04738d7775
Merge branch '4.15' into main 2021-08-28 09:14:39 -03:00
Wei Zhou
41f6f0e568
server: allow destroy/recover volumes which are attached to removed vms (#5364)
* server: allow destroy volumes which attach to a expunged vm

* server: recover volume which is attached to a removed vm
2021-08-28 09:10:31 -03:00
Pearl Dsilva
4acedc8308
Fix failure to scp diagnostic data file from SSVM on ubuntu based env (#5384) 2021-08-28 10:30:31 +05:30
nicolas
8860fc8143
Merge branch '4.15' into main 2021-08-28 01:11:52 -03:00
Wei Zhou
e968f3b16a
server: detach data disks before destroying vms (#5356) 2021-08-28 01:00:04 -03:00
SadiJr
0a73f5162d
Externalize config to increment or not VM metrics in memory (#5351)
Co-authored-by: SadiJr <17a0db2854@firemailbox.club>
2021-08-24 14:16:58 -03:00
DK101010
9163013683
Feat/ram reservation (#4662)
* remove hot enable cpu und memory in case of reservation

ram and cpu reservation have not relation to ram and cpu hot add

* add custom ram_reservation and it to vm details

* system vms haven't this property, for this reason add additional check

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* replace 0.0 with NumberUtils

* remove default value and remove return MinRam(seems to be not necessary)

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/guru/VmwareVmImplementer.java

Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>

* Update plugins/hypervisors/vmware/src/main/java/com/cloud/hypervisor/vmware/resource/VmwareResource.java

Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
2021-08-24 14:15:52 -03:00
Pearl Dsilva
e1e6ec107c
vmware: Bypass empty string check for username and password (#5337)
* Bypass empty string check for username and password

Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-08-24 12:45:29 +05:30
DK101010
83dd294940
remove doubles before save (#5341)
* remove doubles before save

* Update ResourceManagerImpl.java

adding final statement

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
2021-08-24 01:13:31 -03:00
Daniel Augusto Veronezi Salvador
8a16729fcf
Support vm dynamic scaling with kvm (#4878)
* Create utility to centralize byte convertions

* Add/change toString definitions

* Create Libvirt handler to ScaleVmCommand

* Enable dynamic scalling VM with KVM

* Move config from interface to class and rename it

As every variable declared in interfaces are already final,
this moving will be needed to mock tests in nexts commits

* Configure VM max memory and cpu cores

The values are according to service offering or global configs

* Extract dpdk configuration to a method and test it

* Extract OS desc config to a method and test it

* Extract guest resource def to a method and test it

Improve libvirt def

* Refactor LibvirtVMDef.GuestResourceDef

* Refactor ScaleVmCommand

* Improve VMInstaVO toString()

* Refactor upgradeRunningVirtualMachine method

* Turn int variables into long on utility

* Verify if VM is scalable on KVMGuru

* Rename some KVMGuruTest's methods

* Change vm's xml to work with max memory

* Verify if service offering is dynamic before scale

* Create methods to retrieve data from domain

* Create def to hotplug memory

* Adjust the way command was scaling the VM

* Fix database persistence before executing command

* Send more info to host to improve log

* Fix var name

* Fix missing "}"

* Undo unnecessary changes

* Address review

* Fix scale validation

* Add VM prepared for dynamic scaling validation

* Refactor LibvirtScaleVmCommandWrapper and improve unit tests

* Remove duplicated method

* Add RuntimeException check

* Remove copyright from header

* Remove copyright from header

* Remove copyright from header

* Remove copyright from header

* Remove copyright from header

* Update ByteScaleUtilsTest.java

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-08-21 09:29:02 +02:00
DK101010
1b041ee7d8
adapt condition to use the correct letter for pvlan types (#5194)
* adapt condition to use the correct letter for pvlan types

* changed to var and add lower case

* Update server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Update server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-20 09:51:05 -03:00
Rakesh
ee01522d3f
Allow users to update volume name (#4618)
Provide an api support to update volume name by all users
2021-08-19 11:56:35 -03:00
Rohit Yadav
953f5b81ed Merge remote-tracking branch 'origin/4.15' into main 2021-08-18 16:56:19 +05:30
Abhishek Kumar
7bfc7f04fe
server: use id column as secondary sort criteria with sortKey (#5320)
For tables that use sort_key field, incorrect results can be returned by the list* APIs while using pagination and with the same values for sort-key field. (Default value for sort_key field remains 0).
This change adds id column as secondary sort criteria in such cases.

Fixes #5316

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-18 16:52:25 +05:30
davidjumani
21d7854274
s2svpn: Set initial state as Connecting (#5327)
* s2svpn: Set initial state as Connecting

* Fix comments

* Adding connecting elsewhere
2021-08-18 16:50:38 +05:30
Rakesh
00e014ca58
Add sent and received bytes to listNetworks and listVirtualMachines. (#4776)
* Add sent and receivedbytes to listNetworks and listVirtualMachines.

Display the traffic data in networks and vm api response

* follow code convention

* remove final keyword

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-08-16 14:29:34 -03:00
DK101010
664a46a525
PR multi tags in compute offering [#4398] (#4399)
* [#4398] adapt code to handle multi tag string with commas

* [#4398] remove trailing spaces

* [#4398] add multi host tag support for ingest process

* [#4398] add test for multi tag support in offerings

* [#4398]  update multitag support for DeploymentPlanningManagerImpl

encapsulate multi tag check from Ingest Feature, DepolymentPlanningManager into
HostDaoImpl to prevent code duplicates

* [#4398] move logic to HostVO and add tests

* rename test method

* [#4398] Change string method to apaches StringUtils

* [#4398] modify test for multi tag support

* adapt sql for double tags

Co-authored-by: Dirk Klahre <Dirk.Klahre@Itelligence.de>
2021-08-16 12:08:40 -03:00
Pearl Dsilva
0838d79ddd
Add missing command - syncStoragePool in main branch (#5312) 2021-08-16 15:18:21 +05:30
DK101010
1d17d7cfbc
Add internal cs name to vm during the ingest (#5242)
* add internal cs name to vm during the ingest

* Update server/src/main/java/org/apache/cloudstack/vm/UnmanagedVMsManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* remove lang.StringUtils

* Update vmware-base/src/main/java/com/cloud/hypervisor/vmware/mo/HostMO.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* Update vmware-base/src/main/java/com/cloud/hypervisor/vmware/mo/HostMO.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-13 16:38:52 -03:00
Rakesh
2a4c2c2506
Global setting to select preferred storage pool (#5249)
* Global setting to select preferred storage pool

Currently all the volumes are allocated on storage pools
based on the capacity or the algorithm selected. Sometimes
we need to deploy all volumes of particular account in a
specific storage pool and in that case its not possible.

with this change, we can specify the uuid of the preferred
storage pool, so that all volumes of the account will be
deployed in this pool

* code feedback

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-08-12 00:01:15 -03:00
Wei Zhou
846efdbfe4
server: Optional destination host when migrate a vm (#4378)
* server: Optional destination host when migrate a vm

* #4378: migrate systemvms/routers with optional host

* #4378: fix mistake

* #4378: fix issue when migrate systemvm

* #4378 add autoselect to migrate api commands

* #4378: more ui change

* #4378: add label label.migrate.auto.select

* #4378: add method chooseVmMigrationDestination

* #4378: fix vm migration wih storageid on vmware

* #4378: add method to collect vm disk/network statistics

* #4378: set autoSelect to default 'true'

* #4378: use BooleanUtils.isNotFalse

Co-authored-by: Wei Zhou <weizhou@apache.org>
2021-08-10 01:25:57 -03:00
davidjumani
cb1078aa20
Merge remote-tracking branch 'apache/4.15' into main 2021-08-09 15:10:51 +05:30
davidjumani
3c8c704df3
kubernetes: Deploy kubernetes-provider when creating a cluster (#5254)
* kubernetes: Deploy kubernetes-provider when creating a cluster

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-09 14:45:20 +05:30
Pearl Dsilva
ea7d3b34d1
Cleanup volume information from db when deleted (#4551)
* Cleanup volume information from db when deleted

* reuse search builder

* revert change

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-08-09 14:21:07 +05:30
Rakesh
949cf71491
api: Display last updated time for VM (#4685)
Add a field in api response and ui to display last updated time
2021-08-09 14:17:53 +05:30
Rakesh
2f8d557f58
api: Change GET/POST request max length of VM user data to 4K/1M (#4737)
Currently we can send a default value of 4K/32K for GET/POST request of
user data field. Most new browsers and also nginx support till 1MB of
post data.

Added a new global setting `vm.userdata.max.length` with default value of
32KB which can be increased till 1MB.
2021-08-09 14:14:19 +05:30
Abhishek Kumar
1ccb42017f
server: skip zone check for PERHOST iso during attachIso (#5270)
* server: skip zone check for PERHOST iso during attachIso

Hypervisor tools ISO - vmware-toools.iso, xs-tools.iso are marked as PERHOST in DB. They are active but not downloaded to the secondary storages and hence no template-zone entry.
Skips the template-zone check for such templates.

Fixes #5265

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* inverted check

* use constants in TemplateManager

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-08-09 14:02:25 +05:30
Gabriel Beims Bräscher
95de82750e
server: Fix migration issue in UserVmManagerImpl.migrateVirtualMachineWithVolume (#5288)
I've upgraded a stage environment from an older 4.16-SNAPSHOT to the current one and found a regression bug at the VM migration.

When calling the migrateVirtualMachineWithVolume, the following InvalidParameterValueException is launched: Unsupported hypervisor: KVM for VM migration, we support XenServer/VMware/KVM only.
2021-08-09 13:39:09 +05:30
Gabriel Beims Bräscher
43c8da2d0e
API-call to declare host as Degraded (#4111)
* Declare host as dead

* Enhance DeclareHostAsDeadCmd and add CancelHostAsDeadCmd

* Stop VMs on Dead Host
- Enhance code

* Add "since" on API, enhance description; change API cmds response handling

* Replace the ResourceState and command names from 'Dead' to 'Degraded'

* Replace missing 'Dead' word to 'Degraded'

* Update API version for 4.16.0.0
2021-08-08 21:38:06 +02:00
slavkap
d6a77a72f0
Allow updating the storage/host tags of service offerings (#5043) 2021-08-02 15:48:07 +02:00
Gabriel Beims Bräscher
0d8b4de1b2
Cleaning up code and enhancing a few IP management logs (#4714)
* Cleanup unnecessary code and enhance a few log messages at IpAddressManagerImpl

* Add toString method for DataCenterVO

* line too long
2021-07-30 16:38:11 +02:00
Wei Zhou
61fd4028bc
vmware: VR health check in vmware basic zone (#5236)
This PR fixes VR heallth check error in vmware basic zone (#4158).
2021-07-30 10:57:21 +05:30
Harikrishna
98d42750cc
Merge pull request #4257 from RodrigoDLopez/Remove_tags_check_when_migrate_volume
remove the unnecessary check for tags when migrating volumes
2021-07-29 09:53:52 +05:30
Sina Kashipazha
490658c5c3
display nics deviceid and order nics by deviceid on Nics tab of instances. (#4768) 2021-07-28 15:11:43 -03:00
Daniel Augusto Veronezi Salvador
1f5ee5b3e3
Improve logs on ConsoleProxyManagerImpl and refactor a few process (#4959) 2021-07-27 19:56:14 -03:00
Rohit Yadav
c6cb30d1ea Merge remote-tracking branch 'origin/4.15' into main 2021-07-27 17:47:05 +05:30
Pearl Dsilva
826e479cf5
api: Fix pagination for list PublicIPAddresses (#5231)
* api: Fix pagination for list PublicIPAddresses

* refactor based on comment
2021-07-27 15:00:56 +05:30
Abhishek Kumar
87ee86679e
ui: vmware vm import-unmanage (#5075)
Adds UI for importing and unmanaging VMs.
A new navigation section - Tools has been added in the UI.

Doc PR: apache/cloudstack-documentation#221

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-07-27 11:12:37 +05:30
davidjumani
37f3fc30c9 Merge remote-tracking branch 'apache/4.15' into main 2021-07-27 10:23:00 +05:30
Daniel Augusto Veronezi Salvador
3c1219a5e2
Add logs to api removeVpnUser (#4616)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-07-26 23:28:06 -03:00
slavkap
446337b4cc
Fix of shrinking volumes with QCOW2 format (#5225)
* Fix of shrinking volumes with QCOW2 format

If the volumes are with QCOW2 format the shrinking will be handled on
the agents side. There are cases in some storage plugins where the
volumes' format is kept in the DB in QCOW2 but the actual format is raw.
Till the current implementation this was limiting the plugins to shrink
the volumes. Now this will be handled by the storage plugins

* Addressed @nvazquez suggested change

Will log the exception instead the exception message
2021-07-26 08:37:47 -03:00
DK101010
c23cbdff8d
Remove condition that are prevent resizing for root volumes (vmware) (#5186)
* remove condition that are prevent resizing for root volumes

* adjust tests acordingly

* enable disk resize tests for VMware

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: mib1185 <mail@mib85.de>
2021-07-21 22:33:51 -03:00
Daniel Augusto Veronezi Salvador
eb3acc334b
Externalize tls version and security protocols configuration on mail sending (#5119)
* Externalize configs to alert

* Externalize configs to project

Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2021-07-21 09:01:11 -03:00
dahn
1f8b34f5c5
add entity-type to message when no UUID is found for a DB ID (#5163)
* add entity-type to message

* Update server/src/main/java/com/cloud/uuididentity/UUIDManagerImpl.java

* update for review comments

* syntax

* Avoid NPE when entity type is null, and throw appropriate exception

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2021-07-21 08:01:08 -03:00
Pearl Dsilva
1f743e911a
api: Add 'created' field to API response (#5210)
* api: Add 'created' field to API response

* address comments
2021-07-20 18:04:13 -03:00