4740 Commits

Author SHA1 Message Date
Daan Hoogland
fadb39ece7 Merge release branch 4.20 to main
* 4.20:
  merge errors fixed
  Restrict the migration of volumes attached to VMs in Starting state (#9725)
  server, plugin: enhance storage stats for IOPS (#10034)
  Introducing granular command timeouts global setting (#9659)
  Improve logging to include more identifiable information (#9873)
2025-01-08 14:01:19 +01:00
Abhishek Kumar
bd488c4bba
server, plugin: enhance storage stats for IOPS (#10034)
Adds framework layer change to allow retrieving and storing IOPS stats for storage pools. Custom `PrimaryStoreDriver` can implement method - `getStorageIopsStats` for returning IOPS stats. Existing method `getUsedIops` can also be overridden by such plugins when only used IOPS is returned.
For testing purpose, implementation has been added for simulator hypervisor plugin to return capacity and used IOPS for a pool.
For local storage pool, implementation has been added using iostat to return currently used IOPS.
StoragePoolResponse class has been updated to return IOPS values which allows showing IOPS values in UI for different storage pool related views and APIs.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-07 17:17:12 +05:30
dahn
d1cf45365a
Revert "pre-commit: add hook check-yaml (#9133)" (#10161)
This reverts commit 57867dc6b031b996147ccfc32deba3624227fbce.
2025-01-06 12:21:26 +01:00
Vishesh
a4224e58cc
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin

* Update logging for scaleio plugin

* Improve logging to include more identifiable information for default volume storage plugin

* Improve logging to include more identifiable information for agent managers

* Improve logging to include more identifiable information for Listeners

* Replace ids with objects or uuids


* Improve logging to include more identifiable information for engine

* Improve logging to include more identifiable information for server

* Fixups in engine

* Improve logging to include more identifiable information for plugins

* Improve logging to include more identifiable information for Cmd classes

* Fix toString method for StorageFilterTO.java
2025-01-06 16:42:37 +05:30
John Bampton
5bae1188ff
pre-commit fix mixed line endings in XML files (#10148)
https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#mixed-line-ending
2025-01-03 09:42:09 +01:00
John Bampton
0f3ea8ceb2
Add pre-commit hook fix-byte-order-marker (#10147)
https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#fix-byte-order-marker
2025-01-03 09:40:43 +01:00
John Bampton
fd24509ba7
pre-commit: fix .cs files with the trailing-whitespace hook (#9794)
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-12-31 13:17:10 +01:00
John Bampton
57867dc6b0
pre-commit: add hook check-yaml (#9133)
https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#check-yaml
2024-12-31 09:32:32 +01:00
John Bampton
6a2c7b0220
pre-commit: enable markdownlint rule MD012 (#9364)
MD012 no-multiple-blanks - Multiple consecutive blank lines

https://github.com/DavidAnson/markdownlint/blob/main/doc/md012.md
2024-12-27 10:25:11 +01:00
John Bampton
ee893d1202
Clean up Python code; Python 2 to 3 (#8993) 2024-12-23 10:11:51 +01:00
Daan Hoogland
9295a1624d Merge release branch 4.20 to main
* 4.20:
  VR: apply iptables rules when add/remove static routes (#10064)
  Certificate and VM hostname validation improvements (#10051)
  set ulimit for server according to redhat spec (#10040)
  kvm-storage: provide isVMMigrate information to storage plugins (#10093)
  Allow config drive deletion of migrated VM, on host maintenance (#10045)
  linstor: improve heartbeat check with also asking linstor (#10105)
  server: simplify role change validation (#9173)
  UI: create VPC network offering with conserve mode (#10082)
  server: fix typo removeaccessvpn in VirtualRouterElement (#10086)
  UI: remove duplicated Instance Name in Public IP details page (#10087)
  UI: Fixes in the Usage UI (#10000)
  SAML2: add cookie with HttpOnly too #10013 (#10047)
  ui: Allow font-awesome icon usage and optimise icon size inconsistency (#9744)
2024-12-20 14:37:49 +01:00
Daan Hoogland
b7f0aac519 Merge branch '4.19' into 4.20 2024-12-20 14:34:39 +01:00
Suresh Kumar Anaparti
188eacd9eb
Certificate and VM hostname validation improvements (#10051)
* Certificate and VM hostname validation improvements

* Improve certificate name validation and some code/log improvements
2024-12-20 10:18:11 +05:30
Rene Peinthor
a9587bfd2e
kvm-storage: provide isVMMigrate information to storage plugins (#10093)
Particular Linstor needs can use this information to only allow
dual volume access for live migration and not enable it in general,
which can and will lead to data corruption if for some reason
2 VMs get started on 2 different hosts.
2024-12-18 09:13:41 +01:00
Suresh Kumar Anaparti
b4ad04badf
Allow config drive deletion of migrated VM, on host maintenance (#10045) 2024-12-18 09:12:28 +01:00
Rene Peinthor
a2f2e87c12
linstor: improve heartbeat check with also asking linstor (#10105)
If a node doesn't have a DRBD connection to another node,
additionally ask Linstor-Controller if the node is alive.
Otherwise we would have simply said no and the node might still be alive.
This is always the case in a non hyperconverged setup.
2024-12-16 09:59:57 +01:00
Wei Zhou
14460f49e1
Merge remote-tracking branch 'apache/4.19' into 4.20 2024-12-12 11:55:39 +01:00
Wei Zhou
f9b176744e
SAML2: add cookie with HttpOnly too #10013 (#10047) 2024-12-09 16:45:21 +01:00
Henrique Sato
4ac4d9cf29
API to validate Quota activation rule (#9605)
* API to validate Quota activation rule

* Apply suggestions from code review

Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>

* Use constants

---------

Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2024-12-03 14:41:25 -03:00
Daan Hoogland
9b6f9b5f7d Merge release branch 4.20 to main
* 4.20:
  UI: Tooltip on the host information card to display the CPU speed in MHz and the memory value in MB (to 3 decimal places) (#9971)
  UI: Allow accounts of the `User` type to add other accounts or users to projects through UI (#9927)
  enable to create VPC portfowarding rules with source cidr (#7081)
  Add new column `last_id` to the table volumes (#9759)
  Allow VMWare import via another host (#9787)
  Linstor: add support for ISO block devices and direct download (#9792)
  get expunged VM data for job result (#9949)
  fix section divider display on auth page (#9966)
2024-12-03 16:33:51 +01:00
Daan Hoogland
da54234585 Merge branch '4.19' into 4.20.merge 2024-12-03 16:32:15 +01:00
Wei Zhou
da94ae2c1c
Merge remote-tracking branch 'apache/4.20' 2024-12-03 09:44:42 +01:00
Abhisar Sinha
d17de834a5
Disable API Key Access for users, accounts and domains (#9741)
* cli changes to update user/account, list by apikeyaccess, domain level setting

* UI changes for updating user/account and searchfilter in listview

* make the api parameters and setting accessible only to root admin

* revert changes to ui/package-lock.json

* minor changes to description strings

* UT for ApiServer and AccountManagerImpl classes

* fix pre-commit failure

* Added a constant for the string System

* UT for searchForUsers and searchForAccounts

* Fix marvin test error

* Update schema to use idempotent add column

* Fix `updateTemplatePermission` when the UI is set to a language other than English (#9766)

* Fix updateTemplatePermission UI in non-english language

* Improve fix

---------

Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>

* Added user name uuid to logging

* Add events when api key access is changed via api or config setting

* fix the userid for api key access update event

* Fix ut failure after event logging

* Convert drop down to radio-button in edit user and account

* Add ApiKeyAccess status in User InfoCard for Users if Api key is generated

* Return apiKeyAccess in user and account response only for Root Admin

* fixed noredist build failure

* Show apikeyaccess on the left panel in the user view for root admins as well

* don't show divider if apiKeyAccess is not shown to user

* Fix events generated to set Username, Account and Domain of the caller correctly

* cli changes to update user/account, list by apikeyaccess, domain level setting

* UI changes for updating user/account and searchfilter in listview

* make the api parameters and setting accessible only to root admin

* revert changes to ui/package-lock.json

* minor changes to description strings

* UT for ApiServer and AccountManagerImpl classes

* fix pre-commit failure

* Added a constant for the string System

* UT for searchForUsers and searchForAccounts

* Fix marvin test error

* Update schema to use idempotent add column

* Added user name uuid to logging

* Add events when api key access is changed via api or config setting

* fix the userid for api key access update event

* Fix ut failure after event logging

* Convert drop down to radio-button in edit user and account

* Add ApiKeyAccess status in User InfoCard for Users if Api key is generated

* Return apiKeyAccess in user and account response only for Root Admin

* fixed noredist build failure

* Show apikeyaccess on the left panel in the user view for root admins as well

* don't show divider if apiKeyAccess is not shown to user

* Fix events generated to set Username, Account and Domain of the caller correctly

* Added DB upgrade path from 42000 to 42010

---------

Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Lucas Martins <56271185+lucas-a-martins@users.noreply.github.com>
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-12-03 12:10:54 +05:30
Rodrigo D. Lopez
4189bac8e0
enable to create VPC portfowarding rules with source cidr (#7081)
Co-authored-by: Lopez <rodrigo@scclouds.com.br>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2024-11-28 17:53:07 +01:00
Vishesh
37eec06166
Allow VMWare import via another host (#9787)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-11-28 17:50:08 +01:00
Rene Peinthor
d54b105a03
Linstor: add support for ISO block devices and direct download (#9792) 2024-11-28 17:47:47 +01:00
João Jandre
d9774a8462 Updating pom.xml version numbers for release 4.21.0.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:47:06 -03:00
João Jandre
c63c7ee63e Updating pom.xml version numbers for release 4.20.1.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:40:45 -03:00
João Jandre
2fe3fcef7c Updating pom.xml version numbers for release 4.20.0.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-19 08:54:07 -03:00
Daan Hoogland
a82a2429f4 Merge branch '4.19' 2024-11-14 15:59:35 +01:00
Daan Hoogland
a2690e9864 imports 2024-11-14 15:28:32 +01:00
Daan Hoogland
24c28ad460 Merge branch '4.18' into 4.19 2024-11-14 14:30:06 +01:00
Daan Hoogland
6ef2eaca14 Merge tag '4.18.2.5' into 4.18 2024-11-14 13:59:25 +01:00
dahn
b75742da43
enum name changed for logger update reverted (#9922) 2024-11-14 08:38:42 -03:00
João Jandre
b38ee63c48 Merge branch '4.19' 2024-11-13 10:47:24 -03:00
Rene Peinthor
dfe4a67859
kvm: ref-count secondary storage pool usage (#9498)
If a secondary storage pool is used by e.g.
2 concurrent snapshot->template actions,
if the first action finished it removed the netfs mount
point for the other action.
Now the storage pools are usage ref-counted and will only
deleted if there are no more users.
2024-11-13 10:32:46 -03:00
Daan Hoogland
f7b7339bcf Merge branch '4.19' 2024-11-07 11:33:50 +01:00
Rohit Yadav
5b7c86aa45 Merge remote-tracking branch 'origin/4.18' into 4.19
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-11-07 14:51:43 +05:30
Rene Peinthor
371e244375
linstor: fix live migrate on non-hyperconverged setups (#9832)
In non-hyperconverged setups, diskless nodes don't have a connection
to each other, so setting properties there had no effect.
Now it is checked if a connection exists,
between the live migration nodes and if not,
it will set the allow-two-primaries on resource-definition level.
2024-11-07 14:46:32 +05:30
Wei Zhou
006db90f75
kvm: fix ovs network creation issue (#9869)
This fixes the issue when create a ovs network
```
2024-10-29 16:02:45,089 WARN  [resource.wrapper.LibvirtOvsFetchInterfaceCommandWrapper] (agentRequest-Handler-2:null) (logid:e716722e) Network interface: ''cloudbr1'' not found
```

This is a regression of a previous security release
see "framework/cluster: improve cluster service, integration API server"

since we now use NetworkInterface.getByName to get network interface, we should NOT add single quotes before/after the label.
2024-11-07 14:36:53 +05:30
Daniel Augusto Veronezi Salvador
84538da27c Updating pom.xml version numbers for release 4.18.2.5
Signed-off-by: Daniel Augusto Veronezi Salvador <gutoveronezi@apache.org>
2024-11-05 00:54:55 -03:00
Daniel Augusto Veronezi Salvador
966b75d0b9 Verify QCOW2 features on direct download of template 2024-11-05 00:26:19 -03:00
João Jandre
8af08ddafb Merge branch '4.19' 2024-11-04 08:58:48 -03:00
slavkap
be247334a7
StorPool: fix of delete snapshot (#9855)
* StorPool: fix of delete snapshot

Mark the DB record as destroyed when a snapshot is deleted

* Addressed reviews

* addressed review

* addressed review
2024-11-04 08:52:02 -03:00
Rene Peinthor
a6cef7a78d
linstor/kvm: Workaround a qemu bug and IDE bus discard enabled. (#9859)
qemu has a bug versions prior 7.0 with discard enabled and using the IDE bus.
It would crash the qemu process and kill the virtual machine,
this is most noticeable on installing a windows guest from the
Windows ISO installer.
2024-11-04 08:46:40 -03:00
Rohit Yadav
20901c7443 Merge remote-tracking branch 'origin/4.19' 2024-10-17 14:53:26 +05:30
Rene Peinthor
7fbd0a9fb4
linstor: enable discard for Linstor storage pools (#9770)
* linstor: enable discard for Linstor storage pools

All Linstor storage backends support discard, so it can be safely enabled.

* linstor: enable discard for Linstor storage pools CHANGELOG.md
2024-10-17 14:51:47 +05:30
Rohit Yadav
46201ee043 Merge remote-tracking branch 'origin/4.19' 2024-10-16 10:50:23 +05:30
dahn
44597724f5
make saml auth request option forceauthn configurable (#9756) 2024-10-16 10:45:18 +05:30
João Jandre
554ea220f0 Fix merge errors 2024-10-11 14:12:18 -03:00