36701 Commits

Author SHA1 Message Date
Bryan Lima
00fe25ab01
Fix allocation of VMs with multiple clusters (#8611)
* Fix allocation of VMs with multiple clusters

* Readd debug guard
2024-06-14 13:54:01 +03:00
dahn
ed86dc973b
protect against missing service offering (#9235)
* protect agains missing service offering

* search removed before assuming none

* import

* javadoc
2024-06-14 15:25:46 +05:30
Wei Zhou
8d02e5f808
test: fix test/integration/smoke/test_register_userdata.py which caused networks not to be deleted (#9244)
by moving the test under new domain/account instead of ROOT account
2024-06-14 09:05:57 +02:00
Abhishek Kumar
ce9b2c52f3
cks: fix events (#9070)
Fixes #8043

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-14 12:22:39 +05:30
Abhishek Kumar
df5c546559
server: remove username required condition for listVmsForImport (#9124)
* server: remove username required condition for listVmsForImport

Fixes #8414

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* revert

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-14 12:14:59 +05:30
Rohit Yadav
6ce2a58f9b
ui: improve sort utility for metrics (#9247)
This improves the UI sorting utility to check for metrics data and sort
after cleaning the string data.

Fixes #8663

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-06-14 12:13:02 +05:30
Rohit Yadav
2ca0857bd5
api: listVM API improvement followup, change returning of stats detail (#9177)
- Changes behaviour of details param handling via global setting:
  - listVirtualMachines API: when the details param is not provided, it returns whether stats are returned controlled by a new global setting `list.vm.default.details.stats`
  - listVirtualMachinesMetrics API: when the details param is not provided, it uses `all` details including `stats`
- Users who are affected slow performance of the listVirtualMachines API response time can set `list.vm.default.details.stats` to `false`
- Remove ConfigKey vm.stats.increment.metrics.in.memory which was renamed to `vm.stats.increment.metrics` in #5984 and also remove unused/unnecessary global settings via upgrade path
- Changes default value of VM stats accumulation setting `vm.stats.increment.metrics` to false until a better solution emerges. Since #5984, this is true and during the execution of listVM APIs the stats are clubbed/calculated which can immensely slow down list VM API calls. Any costly operations such as summing of stats shouldn't be done during the course of a synchronous API, such as the list VM API.
- Fix UI that uses listVirtualMachinesMetrics to not call `stats` detail when in list view without metrics selected.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-06-14 11:03:14 +05:30
Vishesh
74f5e52e6e
Fix unit test failure (#9238) 2024-06-13 16:06:35 +05:30
Rene Peinthor
37f4398c80
linstor: Support VM-Instance Disk snapshots (#8796)
* linstor: update to java-linstor 0.5.1

* linstor: Support VM-Instance Disk snapshots

This adds VM-Instance disk snapshot support for
Linstor primary storage. Instance snapshots are stored on
the used Linstor storage pool backend and can be converted
into regular volume snapshots and also reverted.

Instance VM snapshots are not fully atomic but with the
create multi snapshot feature as good as it gets.
Snapshots are done over multiple volumes in the same devicemanager run.
2024-06-13 15:26:33 +05:30
Rene Peinthor
034a5c8466
linstor: ui show Server and resource-group fields with custom protocol (#9138) 2024-06-13 15:19:03 +05:30
Abhishek Kumar
19e9020c9b
ui: fix dashboard retrievals based on permissions (#9237)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-13 10:23:44 +02:00
Abhishek Kumar
abbc61c01e
engine-orchestration: expunge destroyed system vm volume (#9197)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-13 10:00:22 +02:00
Abhishek Kumar
2fef0a32bc
cks: fix list apis response count (#8701)
* cks: fix list apis count

Fixes count value in listKubernetesClusters and
listSupportedKubernetesVersions APIs response.
2024-06-13 13:08:19 +05:30
Harikrishna
517cddcb15
Fix error message if specific host does not have capacity (#9218) 2024-06-13 12:38:51 +05:30
dahn
b3c3f91718
api: add to cpu speed parameter a description of the cgroup2 case (#9191)
This PR adds to the cpuSpeed parameter of CreateServiceOfferingCmd a description how it will be interpreted in the case cgroups are being used on KVM.

Fixes: #6743

Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>

---------

Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2024-06-13 12:36:20 +05:30
Rohit Yadav
0f8a839a1d
engine/schema: force index in user_vm_view to speed up VM instance listing (#9198)
The user_vm_view can end up not picking the right index to join against
the user_ip_address table causing full table scan on the user_ip_address
table. This could be related to a MySQL bug
https://bugs.mysql.com/bug.php?id=41220

In a test environment with 20k shared networks and over 20M IPs, the
listVirtualMachines API was found to take over 17s to return list of
just 10 VMs. However, with this fix it would now take under 200ms to
return the list.

MySQL slow query logging showed ~nearly 20M table scans of the IP
address table:
```
# User@Host: cloud[cloud] @ localhost [127.0.0.1]  Id:    39
# Query_time: 8.227541  Lock_time: 0.000014 Rows_sent: 12  Rows_examined: 19,667,235
SET timestamp=1715410270;
SELECT user_vm_view.id, user_vm_view.name /*snipped*/ FROM user_vm_view
WHERE user_vm_view.id IN (4,6,7,8,9,10,11,12,13,14,15,16);
```

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-06-13 12:29:11 +05:30
Abhishek Kumar
f45267174a
ui: list only accessible networks during import (#9194)
* ui: list only accessible networks during import

Fixes #8612

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* space

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-13 09:28:36 +03:00
Rohit Yadav
78ace3a750
saml: introduce saml2.check.signature (#9219)
Adminstrators should ensure that IDP configuration has signing
certificate for the actual signature check to be performed. In addition
to this, this change introduces a new global setting
`saml2.check.signature` which can deliberately fail a SAML login attempt
when the SAML response has missing signature.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-06-13 11:30:33 +05:30
Vishesh
8221be3a8c
Fix marvin package version while building packages (#9230) 2024-06-13 11:29:51 +05:30
Wei Zhou
b2ef53b8a2
kvm: replace ISO path in vm XML configuration during vm migration (#9212)
* kvm: replace ISO path in vm XML configuration during vm migration

* Update 9212: address comments

* kvm: fix vm migration if there are multiple image stores
2024-06-12 16:01:23 +02:00
Abhishek Kumar
5c749eced2
logging: httpupload do not warn with exception trace (#9220)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2024-06-12 13:57:41 +02:00
Abhishek Kumar
e2e46e3108
ui: fix dashboard on saml user login (#9097)
* ui: fix dashboard on saml user login

Fixes #8950
Fixes #9045

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update ui/src/permission.js

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-12 17:19:21 +05:30
Abhishek Kumar
10dc5ad774
ui; fix admin dashboard vue warnings (#9140)
Fixes warnings on admin dashboard in console.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-12 17:01:52 +05:30
Vishesh
c756e4370a
List volumes by service offering id (#9211)
* Allow listing of volumes by service offering id

* Address comments
2024-06-12 16:13:56 +05:30
dahn
4de975ff25
list by isEncrypted (#8643)
* list by isEncrypted

* use filter on VO and cleanup

* add encryption type to volume response

* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ListVolumesCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-12 15:11:00 +05:30
Suresh Kumar Anaparti
503ae64a3d
Upload volume format check with the supported hypervisor, fail if supported hypervisor not found on zone (#9224) 2024-06-12 11:40:29 +02:00
Pearl Dsilva
0f770194df
Configure config drive ISO with password file on reinstallation of VM with password-enabled template (#9113)
* Configure config drive ISO with password file on reinstallation of VM with password-enabled template

* address comment
2024-06-12 12:00:08 +03:00
Suresh Kumar Anaparti
4ec0f823cf
ScaleIO volume live migration - use usable bytes from source disk to format the destination disk (#9174) 2024-06-12 13:58:10 +05:30
Abhishek Kumar
1577218999
server,event: fix events for storage maintenance (#9056)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-12 09:43:13 +02:00
Abhishek Kumar
ca59720015
vpc,event: fix events for createVpc (#9055)
Fixes #8496

* vpc,event: fix events for createVpc
2024-06-12 12:56:08 +05:30
Wei Zhou
edf7394919
server: search template name and display text if keyword is passed (#9215) 2024-06-11 19:35:43 +05:30
Suresh Kumar Anaparti
2e3f76ec03
Improve error messaging / logs when listing VMs on the remote KVM host (for import) (#9204) 2024-06-11 14:48:21 +02:00
Harikrishna
6edcf32c5e
Fix vsphere storage policy when compute offering uses the disk offering attached to it (#9203) 2024-06-11 14:49:58 +03:00
Harikrishna
acae5c5b9e
kvm: Update the java doc for the method disconnectPhysicalDiskByPath (#9210)
This PR addresses the issue #8789

The original issue is disconnectPhysicalDiskByPath() implementation in FibreChannelAdaptor always returns true irrespective of the success of the operation. This was already fixed in the PR #8889 .

Ideally this method has to be called after choosing the right adapter based on the storage pool type of the volume path, but currently it is just called in a loop.
05b9b6e2e7/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java (L200-L212)

while trying to fix the case of running into the loop of all adapters by somehow passing the storage pool type to that caller cleanup() method but this is touching all over the code (which I fear it creates other regressions), instead I feel we can keep it the current way only since Fibrechannel adapter has already fixed.

In this PR I've added the java doc explaining the method and situation.
2024-06-11 14:44:46 +05:30
Abhishek Kumar
43ab8a9367
cks,ui: fix npe and check for disable zone (#9105)
Fixes #8962
2024-06-11 14:36:11 +05:30
Abhishek Kumar
4bd0b1c24f
ui: refactor global createmenu (#9139)
Refactors global create menu code to make it visible only when any menu
is accessible.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-11 09:33:37 +02:00
Abhishek Kumar
7aacbcb559
api: listApis should return params based on caller (#8973) 2024-06-11 11:28:08 +05:30
Abhishek Kumar
fcca3e8f39
api,server: encryptformat in volume response (#9106)
Fixes #9014

* api,server: encryptformat in volume response

* Apply suggestions from code review

* remove passphrase db change
2024-06-11 11:20:50 +05:30
Vishesh
3321971adc
UI: Update AddAccountOrUserToProject to show a drop down for account & user (#9072) 2024-06-11 11:18:03 +05:30
Rohit Yadav
3f2761ec7c Merge remote-tracking branch 'origin/4.18' into 4.19 2024-06-10 15:26:31 +05:30
Oleg Chuev
0b54fbb539
ui: Updated units names (#8543)
* Updated units names

* Update ui/public/locales/en.json

* Update ui/public/locales/en.json

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
2024-06-10 15:20:52 +05:30
Abhishek Kumar
10f4de0318
kvm: consider provisioning type for local data volumes (#9141)
Fixes #8644

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-10 11:38:31 +03:00
Daan Hoogland
c779b1c616 Merge branch '4.18' into 4.19 2024-06-06 11:24:09 +02:00
Abhishek Kumar
91c7bc722f
server,cks: check if vm is cks node during vm destroy (#9057)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-06 10:54:02 +02:00
Rene Peinthor
2339412f73
linstor: make getDevicePath more robust (#9143) 2024-06-06 09:49:03 +02:00
João Jandre
631d6ad09b
Do not retrieve VM's stats on normal VM listing (#8782)
* Do not retrieve VM's stats on normal VM listing

* Add config to control the behavior

* address reviews
2024-06-05 17:45:28 +05:30
Vishesh
68a231aaee
Show site to site VPN connections in UI (#9066) 2024-06-04 14:43:41 +02:00
Wei Zhou
875cd14919
server: fix fail to list public ips of shared networks sometimes (#8624) 2024-06-04 09:28:39 +02:00
Abhishek Kumar
d77998c2d4
utils,ui: obfuscate sensitive log info, use POST for configureOutOfBandManagement (#9126)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-06-03 11:57:44 +02:00
Vishesh
87b55af197
Fixup response code on incorrect credentials (#8671) 2024-05-30 08:48:53 +02:00