35947 Commits

Author SHA1 Message Date
Marcus Sorensen
a0702279aa
server Don't allow inadvertent deletion of hidden details via API (#7880)
* Don't allow inadvertent deletion of hidden details via API

* Update VM details unit test ensuring system/hidden details not removed

* Update test/integration/component/test_update_vm.py

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-24 10:45:39 +02:00
Charles Queiroz
2e1c2821a8
UI: Add central project store and watch functionality (#7900)
Added a new getter 'allProjects' and mutation 'RELOAD_ALL_PROJECTS' to centralize the tracking of available projects in the state.
This eliminates direct manipulation of the Project list in separate components and improves data consistency across the application.
A watcher in ProjectMenu.vue has been implemented to handle changes to the 'allProjects' getter.
The 'RELOAD_ALL_PROJECTS' mutation was also added where necessary to ensure projects list is updated in the state whenever changes occur.
2023-08-24 10:13:47 +02:00
Rohit Yadav
b37834fe75
ui: bump UI pkg version to match CloudStacks (#7894)
This uses the version in UI's package.json same as CloudStack's. The idea
is that static assets aren't cached per discussion on
https://github.com/apache/cloudstack/issues/7546

Fixes #7546

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-23 13:07:07 +02:00
Rohit Yadav
d87f39ad27
ui: make it clear that deletion of a volume will not delete any snapshots (#7897)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-23 16:07:17 +05:30
Wei Zhou
8cb30551d5
UI: update ip ranges of shared networks (#7896)
* UI: update ip ranges of shared networks

* UI: remove ip6gateway and ip6cidr from GuestIpRanges.vue
2023-08-23 16:06:16 +05:30
Wei Zhou
e4117e68d2
engine/schema: fix duplicated guest OSes in 4.18.0.0 (#7799)
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-08-23 09:44:34 +02:00
Rohit Yadav
0e9a19f24d
ui: Fix project theme on reload or refresh (#7893)
* css fixes from main

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* component: when in project, toggle the theme also

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-22 14:20:45 +02:00
Wei Zhou
db0e7a65af
server: check hostId when attach disk to a Stopped vm with local storage (#7886)
This fixes #7834
2023-08-22 14:52:20 +05:30
Wei Zhou
5097d2aa8b
ui: fix creating zone with vxlan if Guest physical network is not the last (#7801)
This fixes #7790
2023-08-22 14:45:30 +05:30
Wei Zhou
78bdde9e98
AutoScaling: support Managed User Data (#7769) 2023-08-22 11:07:16 +02:00
Nicolas Vazquez
405ef82aef
UI: Fix hide delete button for templates that are in Installing template state (#7882)
This PR hides the Delete Template button when a template is installing

Fixes: #7865
2023-08-22 14:32:54 +05:30
Marcus Sorensen
6cded7e010
Support GNU sed apidoc for MacOS build (#7888)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-08-22 14:31:20 +05:30
Fabricio Duarte
6d24217636
server: Allow admins to disable the 2FA of users in subdomains (#7870) 2023-08-21 15:48:33 +02:00
Nicolas Vazquez
ddc2a362a8
UI: Add missing tooltips to Register/Upload Templates/ISOs views (#7879) 2023-08-21 11:13:25 +02:00
Sina Kashipazha
d296f54c7f
Api: update command domainId/account descriptions (#7876) 2023-08-20 15:44:31 +02:00
João Jandre
f7345e861f
cpvm: Fix CPVM not releasing CPU after closing console (#7826) 2023-08-18 11:12:06 +02:00
Sina Kashipazha
add64bd7e6
security group: replace vm.getPrivateMacAddress() with nic.getMacAddress() (#7293) 2023-08-18 10:49:16 +02:00
sato03
5b33967310
Fix role escalation prevention (#7853)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-08-18 09:33:05 +02:00
Wei Zhou
87d6c4fbd5
packaging: install tzdata-java or timezone-java (#7875)
According to https://pkgs.org/download/tzdata-java
the package name is tzdata-java on some OSes, but timezone-java on some other OSes (e.g openSUSE Leap 15.4)
2023-08-18 09:20:00 +02:00
Wei Zhou
d8a5c89060
server: get only Ready ISO to mount (#7848) 2023-08-17 11:04:14 +02:00
Wei Zhou
c8d6e50539
VMware: add support for 8.0b (8.0.0.2), 8.0c (8.0.0.3) (#7380)
* VMware: add support for 8.0b (8.0.0.2)

* VMware 8: add new guest os mappings in VirtualMachineGuestOsIdentifier

The full list can be found at https://developer.vmware.com/apis/1355/vsphere

* VMware: get guest os mappings of parent version

* VMware8: remove guest os mappings for 8.0.0.2

* VMware8: fix code smells

* vmware: remove annotations in VmwareVmImplementerTest which caused 0.0% code coverage

* VMware8: add a unit test case

* VMware: add support for 8.0c (8.0.0.3)

* VMware8: move to CloudStackVersion.getVMwareParentVersion

* VMware: add support for 8.0u1 (8.0.1.0)

* Copy engine/schema/src/main/java/com/cloud/upgrade/GuestOsMapper.java from PR 6979

* Copy engine/schema/src/main/java/com/cloud/storage/dao/GuestOSHypervisorDao.java from PR 6979

* VMware: ignore the last number in VMware versions

* VMware: copy guest os mapping from 8.0 to 8.0.1

* VMware: add unit tests in VmwareVmImplementerTest.java

* Copy engine/schema/src/test/java/com/cloud/upgrade/GuestOsMapperTest.java from PR 6979

* VMware8: retry vm poweron if fails due to exception "File system specific implementation of Ioctl[file] failed"

This fixes a weird issue on vmware8. When power on a vm, sometimes it fails due to error

2023-04-27 07:04:43,207 ERROR [c.c.h.v.r.VmwareResource] (DirectAgent-442:ctx-cdd42b03 10.0.32.133, job-105/job-106, cmd: StartCommand) (logid:8a24a607) StartCommand failed due to [Exception: java.lang.RuntimeException
Message: File system specific implementation of Ioctl[file] failed
].
java.lang.RuntimeException: File system specific implementation of Ioctl[file] failed
        at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:426)
        at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.powerOn(VirtualMachineMO.java:288)

in vmware.log on ESXi host, it shows

2023-04-27T09:20:41.713Z In(05)+ vmx - Power on failure messages: File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of LookupAndOpen[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - File system specific implementation of Ioctl[file] failed
2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to lock the file
2023-04-27T09:20:41.713Z In(05)+ vmx - Cannot open the disk '/vmfs/volumes/7b29c876-ac102328/i-2-167-VM/ROOT-167.vmdk' or one of the snapshot disks it depends on.
2023-04-27T09:20:41.713Z In(05)+ vmx - Module 'Disk' power on failed.
2023-04-27T09:20:41.713Z In(05)+ vmx - Failed to start the virtual machine.

There is a KB article for it, but I still do not know why and how to fix it.
https://kb.vmware.com/s/article/1004232

* VMware: extract to method powerOnVM

* vmware: fix mistake in logs

* vmware8: use curl instead of wget to fix test failures

Traceback (most recent call last):
  File "/root/test_internal_lb.py", line 555, in test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80
    self.execute_internallb_roundrobin_tests(vpc_offering)
  File "/root/test_internal_lb.py", line 641, in execute_internallb_roundrobin_tests
    client_vm, applb.sourceipaddress, max_http_requests)
  File "/root/test_internal_lb.py", line 497, in run_ssh_test_accross_hosts
    (e, clienthost.public_ip))
AssertionError: list index out of range: SSH failed for VM with IP Address: 10.0.52.187

and

sshClient: DEBUG: {Cmd: /usr/bin/wget -T3 -qO- --user=admin --password=password http://10.1.2.253:8081/admin?stats via Host: 10.0.52.188} {returns: ["/usr/bin/wget: '/usr/lib/libpcre.so.1' is not an ELF file", "/usr/bin/wget: can't load library 'libpcre.so.1'"]}

* VMware: correct guest OS names in hypervisor mappings for VMware 8.0

el9 and variants were introduced by https://github.com/apache/cloudstack/pull/7059
they are supported with guest os identifiers since VMware 8.0

see https://vdc-repo.vmware.com/vmwb-repository/dcr-public/c476b64b-c93c-4b21-9d76-be14da0148f9/04ca12ad-59b9-4e1c-8232-fd3d4276e52c/SDK/vsphere-ws/docs/ReferenceGuide/vim.vm.GuestOsDescriptor.GuestOsIdentifier.html

* VMware: add Ubuntu 20.04 and 22.04 support for vmware 7.0+

* PR7380: only add guest os mappings for Ubuntu 20.04

* PR7380: Correct RHEL9 guest os names and others for VMware 8.0

* PR7380: correct guest os names on 8.0.0.1 as well

* PR7380: remove Windows 12 and Windows Server 2025 which are not released yet
2023-08-17 10:42:42 +02:00
Charles Queiroz
e8b491177f
server: Replace Hashtable with LinkedHashMap in createIsoResponse (#7844)
* Replace Hashtable with LinkedHashMap in createIsoResponse

This change replaces the use of Hashtable with LinkedHashMap in the `createIsoResponse` method of `ViewResponseHelper`.
 The reason for this modification is to maintain the insertion order of entries, which isn't the case with Hashtable.
  This could lead to more predictable results and behaviors in calling methods.

* Replace Hashtable with LinkedHashMap in view response creation methods

Changed Hashtable to LinkedHashMap in various response creation methods within ViewResponseHelper class.
This modification ensures an ordered iteration which is beneficial for scenarios where the insertion order of responses needs to be maintained consistently.

---------

Co-authored-by: Sina Kashipazha <soreana@users.noreply.github.com>
2023-08-16 08:58:41 +02:00
dahn
cf249f64a3
api: Force-stop descriptions (#7866) 2023-08-16 08:31:32 +02:00
dahn
19aacad46b
UI: Show iso urls (#7869)
* add url to details

* some cleanup
2023-08-15 13:51:53 +02:00
Wei Zhou
aa02d9b3c1
test: skip live storage migration on CentOS 7 (#7862)
since #7570, The detail 'Host.OS' of centos7 host is changed from 'CentOS' to 'CentOS Linux'
2023-08-15 08:39:18 +02:00
Alexandru Bagu
a47a4f4ad4
CPVM: fix cpu usage for console vm when using vnc over websockets (#6970)
Co-authored-by: Alexandru Bagu <alexandru.bagu@hotmail.com>
2023-08-14 15:45:04 +02:00
Fabricio Duarte
72e3491cef
server: Fix allocation of more public IPs than the account's limit (#7832) 2023-08-14 10:33:29 +02:00
Fabricio Duarte
2c6072273b
VMware: Fix casting exception during backup NIC synchronization (#7831) 2023-08-14 10:20:33 +02:00
Wei Zhou
bf9c71430b
CKS: check access when get kubernetescluster config (#7854) 2023-08-14 10:02:20 +02:00
Rohit Yadav
feb9509547
ui: assorted improvements (#7833)
This PR aims to polish the UI with following tweaks and changes:

- Increase resource and os-logo icons both in list view, user-menu bar and VM deployment form
- Fix css issues in VM deployment form when resource icons are on some of the templates/isos but not all
- Replace edit icon in the resource icon editting button on the infocard, in resource view
- Fix css marging/padding issue for nav bar and left-branding/logo
- Introduce a new Limits option in the user menu, to allow users to see their own limits when they log in
- Rename resource tab to limits tab for accounts, project and domains
- Introduce a new copy-label component, that can be clicked to copy strings; use in info-card and list view for entites such as IP addresses and UUIDs
- Add router-link to /zones/ in case of user-accounts (when /zone isn't routable in the UI)
- Show better list of nics and ssh keys pairs in infocard for VM resource view
- Standardise most resources to show state/status columns right after resource name (wherever applicable)
- Remove displayname column in VM list view, add cpu number and memory by default
- Add k8s version column in k8s list view
- Add size and phy size columns in case of template and ISOs list view, only for root/domain admins
- Add phy network router-link in case of guest VLAN list view; rearrange columns list for consistency
- Add snapshot phy size column in the snapshot list view; and router-link for volume in the snapshot list view; and missing/useful details in the volume snapshot details view
- Add a create and add data disk feature in Instances tab, just like we've add nic feature in the same

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 14:00:04 +05:30
Rohit Yadav
29c7b3167e
cloudstack: bump to use 4.18.1.0 systemvmtemplate (#7828)
* cloudstack: bump to use 4.18.1.0 systemvmtemplate

Use new 4.18.1.0 systemvmtemplate

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix cd ISO url

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 07:38:47 +02:00
Wei Zhou
d9319e645c
storpool: fix pom.xml which caused docker image build failures (#7845)
steps to reproduce the issue:
- git clone https://github.com/apache/cloudstack.git
- cd cloudstack
- rm -rf .git/
- run `mvn -P developer,systemvm clean install`

Without this PR, it fails with error

```
 > [ 8/10] RUN mvn -Pdeveloper -Dsimulator -DskipTests clean install:
668.1 [ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:4.9.10:revision (get-the-git-infos) on project cloud-plugin-storage-volume-storpool: .git directory is not found! Please specify a valid [dotGitDirectory] in your pom.xml -> [Help 1]
```
2023-08-10 16:28:49 +02:00
Wei Zhou
1605b2f0db
server: remove registered userdata when cleanup an account (#7777) 2023-08-10 14:33:59 +02:00
João Jandre
fdb23dae40
server: Use max secondary storage defined on the account during upload (#7441) 2023-08-10 11:39:40 +02:00
Wei Zhou
b32ba953cf
test: upgrade kubernetes versions to 1.25.0/1.26.0 (#7685)
* test: upgrade versions to 1.25.0/1.26.0

* PR7685: move settings to test_data.py
2023-08-10 09:48:08 +02:00
fermosan
fa58f59619
kvm: Added VNI Devices as normal bridge slave devs (#7836)
This will allow for VXLAN configurations to utilize tags on the physical network of a zone
2023-08-10 08:59:28 +02:00
Wei Zhou
a1af03f413
noVNC: fix JP keyboard on vmware7+ which uses websocket URL (#7694)
* noVNC: fix JP keyboard on vmware7+ which uses websocket URL

* noVNC: cleanup rfb.js

* noVNC: fix < and > on JP keyboard

* noVNC: fix Caps lock on JP keyboard
2023-08-09 11:43:16 +02:00
Sina Kashipazha
9df23f951b
Prometheus exporter fix cpu/memory usage labels (#7629) 2023-08-07 20:47:03 +02:00
dahn
dea1373480
server: clean network offerings for domain on remove (#7775) 2023-08-07 09:56:58 +02:00
Rohit Yadav
4f85c5568f
engine/schema: fix CPU cap limitation for upgraded environment (#7808)
CPU cap limitation was enabled as part of
https://github.com/apache/cloudstack/pull/6420 that changes behaviour
for existing environments. The CPU cap limitation on KVM causes
systemvms to not start or be really slow in nested and virtualised
environments.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-07 10:19:33 +05:30
mprokopchuk
ab0297ea9b
VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted (#7760)
VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted

* Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

---------

Co-authored-by: Maxim Prokopchuk <mprokopchuk@apple.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-07 10:18:17 +05:30
Rohit Yadav
dc5e4f3ec6
Allow KVM overcommit to work without reducing minimum VM memory when vm ballooning is disabled (#7810)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
2023-08-06 10:39:14 +02:00
dahn
6b21825746
make details optional for addGuestOs API (#7812) 2023-08-04 08:44:29 +02:00
kishankavala
57779a8586
Remove nic allocation txn during Vm deploy. During Vm deploy failure, this will allow cleanup of any nics that are successfully provisioned (#7809) 2023-08-04 08:42:38 +02:00
dahn
b4032d9984
include all VMs when projects selected not just for admins (#7667) 2023-08-03 16:06:11 +02:00
dahn
c86684fbdd
UI: allow new keys for VM details (#7793) 2023-08-03 04:58:22 +02:00
slavkap
da7fc72c66
Refactoring StorPool's smoke tests (#7392)
* Removed the hardcoded StorPool endpoint from tests

- removed the hardcoded enpoint of StorPool primary storage from tests
- added the git commit information into the maven build

* Convert indents to spaces

* update git-commit-id-plugin version
2023-08-02 14:37:37 +08:00
Wei Zhou
87f218c68c
UI: decode userdata in EditVM dialog (#7796) 2023-08-01 09:10:26 +02:00
dahn
60ab494c25
packaging: unalias cp before package upgrade (#7722) 2023-08-01 01:34:05 +02:00
dahn
2bb079189c
make NoopDbUpgrade do a systemvm template check (#7564) 2023-07-31 16:08:14 +02:00