2834 Commits

Author SHA1 Message Date
John Bampton
9834d5168f
pre-commit: standardize more line endings to lf (#7804)
https://github.com/pre-commit/pre-commit-hooks#mixed-line-ending
2023-10-18 11:25:00 +02:00
Daan Hoogland
8eaf2648b6 Merge release branch 4.18 to main
* 4.18:
  Fix UUID for child datastores in all cases (#8057)
2023-10-18 11:22:42 +02:00
Harikrishna
76ab621a5a
Fix UUID for child datastores in all cases (#8057) 2023-10-18 13:00:55 +05:30
Daan Hoogland
72cf9740f9 Merge branch '4.18' 2023-10-06 13:50:29 +02:00
Daniel Augusto Veronezi Salvador
9b8eaeea78
Fix: Convert volume to another directory instead of copying it while taking volume snapshots on KVM (#8041) 2023-10-06 09:47:34 +02:00
Marcus Sorensen
82b981854b
KVM Agent config to reserve dom0 CPUs (#7987)
This PR allows an admin to reserve some hypervisor host CPUs for system use. Another way to think of it is limiting the number of CPUs allocatable to VMs. This can be useful if the admin wants to do other things with the hypervisor's CPU, for example reserve some cores for running hyperconverged storage processes.

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-10-06 10:50:18 +05:30
Rohit Yadav
8cd7147b25 Merge remote-tracking branch 'origin/4.18' 2023-09-28 12:15:23 +05:30
Marcus Sorensen
3694667f50
Trigger out of band VM state update via libvirt event when VM stops (#7963)
* Trigger out of band VM state update via libvirt event when VM stops

* Add License headers, refactor nested try

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-09-28 12:12:03 +05:30
Marcus Sorensen
221f863939
Use direct download timeout configs for URL check (#7948)
Signed-off-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-09-28 12:11:38 +05:30
Marcus Sorensen
28c4be1cf2
Fix style for LibvirtComputingResource variable names and its dependencies (#7991)
* Fix style for LibvirtComputingResource variable names and its dependencies

* More variable name fixes

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-09-27 12:38:25 +05:30
Vishesh
84e770bf95
Remove powermock from pom.xml (#8000) 2023-09-26 13:31:56 +02:00
John Bampton
4eb110af73
Remove unneeded duplicate words (#7850) 2023-09-18 13:16:33 +02:00
Wei Zhou
45616aaf61 Merge remote-tracking branch 'origin/4.18' 2023-09-14 14:00:01 +02:00
Marcus Sorensen
f049d4d189
Increase reserve on ScaleIO disk formatting for fragmentation (#7955)
Signed-off-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-09-14 16:43:16 +05:30
Wei Zhou
246bb24b0f Updating pom.xml version numbers for release 4.18.2.0-SNAPSHOT
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-12 17:26:53 +02:00
Wei Zhou
f6b2a58727 Merge branch '4.18' 2023-09-07 08:56:35 +02:00
Wei Zhou
4bdff06acd Updating pom.xml version numbers for release 4.18.1.0
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-07 08:50:50 +02:00
Wei Zhou
126dd5fa4c
kvm: fix live vm migration between local storage pools (#7945) 2023-09-07 08:22:37 +05:30
Nicolas Vazquez
57c61fb33c
Fix direct download https compressed qcow2 template checker (#7932)
This PR fixes an issue on direct download while registering HTTPS compressed files
Fixes: #7929
2023-09-01 08:16:03 +02:00
Daan Hoogland
40d7da6707 Merge release branch 4.18 to main
* 4.18:
  xen/xcpng: set vmr.VCPUsMax to minimum of global setting and host cpu cores (#7887)
2023-08-28 14:53:23 +02:00
Wei Zhou
439d70fd2b
xen/xcpng: set vmr.VCPUsMax to minimum of global setting and host cpu cores (#7887)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-28 14:26:20 +02:00
Daan Hoogland
24ae5aa5fa Merge branch '4.18' 2023-08-25 14:27:34 +02:00
Rohit Yadav
e964395bd4
vmware: improve solidfire storage plugin integration and fix cases (#3) (#7761)
This fixes the following cases in which Solidfire storage integration
caused issues when using Solidfire datadisks with VMware:

1. Take Volume Snapshot of Solidfire data disk
2. Delete an active Instance with Solidfire data disk attached
3. Attach used existing Solidfire data disk to a running/stopped VM
4. Stop and Start an instance with Solidfire data disks attached
5. Expand disk by resizing Solidfire data disk by providing size
6. Expand disk by changing disk offering for the Solidfire data disk

Additional changes:
- Use VMFS6 as managed datastore type if the host supports
- Refactor detection and splitting of managed storage ds name in storage
  processor
- Restrict storage rescanning for managed datastore when resizing

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-25 11:36:03 +05:30
Wei Zhou
c683de4a55
kvm: fix unit test LibvirtReplugNicCommandWrapperTest (#7908) 2023-08-24 21:41:04 +05:30
Daan Hoogland
27f148e79e Merge branch '4.18' 2023-08-23 11:15:56 +02:00
Wei Zhou
e4117e68d2
engine/schema: fix duplicated guest OSes in 4.18.0.0 (#7799)
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-08-23 09:44:34 +02:00
Daan Hoogland
ea832bce13 Merge branch '4.18' 2023-08-22 11:44:45 +02:00
Wei Zhou
c8d6e50539
VMware: add support for 8.0b (8.0.0.2), 8.0c (8.0.0.3) (#7380)
* VMware: add support for 8.0b (8.0.0.2)

* VMware 8: add new guest os mappings in VirtualMachineGuestOsIdentifier

The full list can be found at https://developer.vmware.com/apis/1355/vsphere

* VMware: get guest os mappings of parent version

* VMware8: remove guest os mappings for 8.0.0.2

* VMware8: fix code smells

* vmware: remove annotations in VmwareVmImplementerTest which caused 0.0% code coverage

* VMware8: add a unit test case

* VMware: add support for 8.0c (8.0.0.3)

* VMware8: move to CloudStackVersion.getVMwareParentVersion

* VMware: add support for 8.0u1 (8.0.1.0)

* Copy engine/schema/src/main/java/com/cloud/upgrade/GuestOsMapper.java from PR 6979

* Copy engine/schema/src/main/java/com/cloud/storage/dao/GuestOSHypervisorDao.java from PR 6979

* VMware: ignore the last number in VMware versions

* VMware: copy guest os mapping from 8.0 to 8.0.1

* VMware: add unit tests in VmwareVmImplementerTest.java

* Copy engine/schema/src/test/java/com/cloud/upgrade/GuestOsMapperTest.java from PR 6979

* VMware8: retry vm poweron if fails due to exception "File system specific implementation of Ioctl[file] failed"

This fixes a weird issue on vmware8. When power on a vm, sometimes it fails due to error

2023-04-27 07:04:43,207 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-442:ctx-cdd42b03 10.0.32.133, job-105/job-106, cmd: StartCommand) (logid:8a24a607) StartCommand failed due to [Exception: java.lang.RuntimeException
Message: File system specific implementation of Ioctl[file] failed
].
java.lang.RuntimeException: File system specific implementation of Ioctl[file] failed
        at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:426)
        at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.powerOn(VirtualMachineMO.java:288)

in vmware.log on ESXi host, it shows

2023-04-27T09:20:41.713Z In(05)+ vmx - Power on failure messages: File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to lock the file
2023-04-27T09:20:41.713Z In(05)+ vmx - Cannot open the disk '/vmfs/volumes/7b29c876-ac102328/i-2-167-VM/ROOT-167.vmdk' or one of the snapshot disks it depends on.
2023-04-27T09:20:41.713Z In(05)+ vmx - Module 'Disk' power on failed.
2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to start the virtual machine.

There is a KB article for it, but I still do not know why and how to fix it.
https://kb.vmware.com/s/article/1004232

* VMware: extract to method powerOnVM

* vmware: fix mistake in logs

* vmware8: use curl instead of wget to fix test failures

Traceback (most recent call last):
  File "/root/test_internal_lb.py", line 555, in test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
    self.execute_internallb_roundrobin_tests(vpc_offering)
  File "/root/test_internal_lb.py", line 641, in execute_internallb_roundrobin_tests
    client_vm, applb.sourceipaddress, max_http_requests)
  File "/root/test_internal_lb.py", line 497, in run_ssh_test_accross_hosts
    (e, clienthost.public_ip))
AssertionError: list index out of range: SSH failed for VM with IP Address: 10.0.52.187

and

sshClient: DEBUG: {Cmd: /usr/bin/wget -T3 -qO- --user=admin --password=password http://10.1.2.253:8081/admin?stats via Host: 10.0.52.188} {returns: ["/usr/bin/wget: '/usr/lib/libpcre.so.1' is not an ELF file", "/usr/bin/wget: can't load library 'libpcre.so.1'"]}

* VMware: correct guest OS names in hypervisor mappings for VMware 8.0

el9 and variants were introduced by https://github.com/apache/cloudstack/pull/7059
they are supported with guest os identifiers since VMware 8.0

see https://vdc-repo.vmware.com/vmwb-repository/dcr-public/c476b64b-c93c-4b21-9d76-be14da0148f9/04ca12ad-59b9-4e1c-8232-fd3d4276e52c/SDK/vsphere-ws/docs/ReferenceGuide/vim.vm.GuestOsDescriptor.GuestOsIdentifier.html

* VMware: add Ubuntu 20.04 and 22.04 support for vmware 7.0+

* PR7380: only add guest os mappings for Ubuntu 20.04

* PR7380: Correct RHEL9 guest os names and others for VMware 8.0

* PR7380: correct guest os names on 8.0.0.1 as well

* PR7380: remove Windows 12 and Windows Server 2025 which are not released yet
2023-08-17 10:42:42 +02:00
Daan Hoogland
5559668f12 Merge branch '4.18' 2023-08-15 09:15:17 +02:00
Fabricio Duarte
2c6072273b
VMware: Fix casting exception during backup NIC synchronization (#7831) 2023-08-14 10:20:33 +02:00
Daan Hoogland
7b64236469 Merge release branch 4.18 to main
* 4.18:
  server: remove registered userdata when cleanup an account (#7777)
  server: Use max secondary storage defined on the account during upload  (#7441)
  test: upgrade kubernetes versions to 1.25.0/1.26.0 (#7685)
  kvm: Added VNI Devices as normal bridge slave devs (#7836)
  noVNC: fix JP keyboard on vmware7+ which uses websocket URL (#7694)
2023-08-10 14:50:46 +02:00
Vishesh
82f9106633
Remove powermock from kvm hypervisor plugin (#7818)
* Remove powermock from kvm hypervisor plugin

* Add @RunWith

* Remove unnecessary mocks

* Remove commented code
2023-08-10 09:04:36 +02:00
fermosan
fa58f59619
kvm: Added VNI Devices as normal bridge slave devs (#7836)
This will allow for VXLAN configurations to utilize tags on the physical network of a zone
2023-08-10 08:59:28 +02:00
Daan Hoogland
eb31e3d795 Merge release branch 4.18 to main
* 4.18:
  Allow KVM overcommit to work without reducing minimum VM memory when vm ballooning is disabled (#7810)
2023-08-06 10:41:00 +02:00
Rohit Yadav
dc5e4f3ec6
Allow KVM overcommit to work without reducing minimum VM memory when vm ballooning is disabled (#7810)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-08-06 10:39:14 +02:00
Vishesh
84974a1e08
Remove powermock from vmware-base & vmware hypervisor plugin (#7798) 2023-08-04 10:40:29 +02:00
Vishesh
90443cdd43
Remove powermock from xenserver hypervisor plugin (#7806) 2023-08-04 10:25:58 +02:00
John Bampton
6f4503488b
pre-commit: apply end-of-file-fixer to all files (#7551) 2023-08-02 13:47:21 +02:00
Daan Hoogland
d51d8a4a13 Merge release branch 4.18 to main
* 4.18:
  UI: Filter templates by zone and hypervisor type when reinstall a VM (#7739)
  KVM: fix SSVM starting when overprovisioning memory (#7663)
  pom.xml: add property project.systemvm.template.location (#7706)
  cloudutils: fix adding rocky9 host failure due to missing /etc/sysconfig/libvirtd (#7779)
  server: get id from persisted object ReservationVO (#7785)
  search in (too) large result sets (#7766)
  ui: fix 404 error when list volumes of system vms (#7772)
  packaging: install tzdata-java on centos7/centos8 (#7768)
2023-07-31 09:04:44 +02:00
dahn
d127d7939d
KVM: fix SSVM starting when overprovisioning memory (#7663) 2023-07-28 11:23:30 +02:00
Rohit Yadav
62a8f4ef72 Merge remote-tracking branch 'origin/4.18' 2023-07-24 15:57:37 +05:30
Marcus Sorensen
63216425d5
Set encrypted PowerFlex disk format correctly (#7735)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-07-24 13:13:46 +05:30
Daan Hoogland
6bb95c0200 Merge release branch 4.18 to main
* 4.18:
  Storage and volumes statistics tasks for StorPool primary storage (#7404)
  proper storage construction (#6797)
  guarantee MAC uniqueness (#7634)
  server: allow migration of all VMs with local storage on KVM (#7656)
  Add L2 networks to Zones with SG (#7719)
2023-07-19 10:59:19 +02:00
dahn
0aade286f5
proper storage construction (#6797) 2023-07-19 10:27:20 +02:00
dahn
73a269e3b3
guarantee MAC uniqueness (#7634)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-19 10:25:01 +02:00
Rohit Yadav
5383bf64f4 Merge remote-tracking branch 'origin/4.18'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-07-07 23:04:44 +05:30
Abhishek Kumar
f0cc76a3a8
vmware: fix unmanaged instance listing when different name & internal CS name (#7641)
Currently, ACS can continue to show an imported instance/VM as an unmanaged instance if the name and internalCSName (custom attribute, cloud.vm.internal.name) is different for the instance/VM on vCenter. This PR while filtering managed instances from the instance list received from ESXi host also checks if the internal name for the instance is not in the managed instance names list.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-07 23:00:25 +05:30
dahn
2752c49fa7
agent: get the right controll cidr (#7580)
Fixes: #7574
2023-07-07 22:57:58 +05:30
Vishesh
d2cd3c81cb
Fix lint errors (#7727) 2023-07-07 21:02:42 +05:30
Vishesh
594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00