12173 Commits

Author SHA1 Message Date
Bernardo De Marco Gonçalves
70776b067a
fix listing of VMs by network (#10204) 2025-01-21 10:33:55 +01:00
Suresh Kumar Anaparti
c5e8f63452
Fix NPE issues during host rolling maintenance, due to host tags and custom constrained/unconstrained service offering (#9844) 2025-01-20 13:20:17 +01:00
Daan Hoogland
ecd60a4e46 Merge release branch 4.20 to main
* 4.20:
  Maintenance mode: Add host to deployment planner avoid list to fix local storage vm migration (#9892)
  Add project-user association normalization script to 4.20.1 upgrade (#10116)
  fix slider component for global settings of the range type (#10187)
  Clean up network permissions on account deletion (#10176)
2025-01-20 11:58:28 +01:00
BartJM
a163831b7e
Maintenance mode: Add host to deployment planner avoid list to fix local storage vm migration (#9892) 2025-01-20 11:54:30 +01:00
Daan Hoogland
9f594c9699 Merge branch '4.19' into 4.20 2025-01-20 11:53:32 +01:00
Daan Hoogland
0c13ded943 Merge release branch 4.20 to main
* 4.20:
  Rollback of changes with errors during the VM assign (#7061)
  [VMware] Consider CD/DVD drive when calculating next free unit number for volume attachment over IDE controller (#9644)
  consider a valid ipv4 address as a validish ipv4 /32 cidr (#10174)
2025-01-15 17:52:07 +01:00
Bernardo De Marco Gonçalves
796bd4f72c
Clean up network permissions on account deletion (#10176) 2025-01-15 17:38:20 +01:00
Stephan Krug
d9a7b6e511
Rollback of changes with errors during the VM assign (#7061)
Co-authored-by: Stephan Krug <stephan.krug@scclouds.com.br>
Co-authored-by: Gabriel <gabriel.fernandes@scclouds.com.br>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-01-15 16:10:49 +01:00
Daan Hoogland
bd874eaa44 Merge release branch 4.20 to main
* 4.20:
  systemvm: fix keystore is reset when patch a systemvm (#9900)
  no retrieval of null hosts (#10175)
  upgrade: consider multiple hypervisors and secondary storages (#10046)
  CheckOnHostCommand: add missing timeout setting (#9677)
2025-01-13 11:23:51 +01:00
Daan Hoogland
e2cfddb1b6 Merge branch '4.19' into 4.20 2025-01-13 11:23:14 +01:00
dahn
fce77733a5
no retrieval of null hosts (#10175) 2025-01-13 09:28:47 +01:00
Wei Zhou
7adc732992
upgrade: consider multiple hypervisors and secondary storages (#10046) 2025-01-13 09:28:18 +01:00
Daan Hoogland
fadb39ece7 Merge release branch 4.20 to main
* 4.20:
  merge errors fixed
  Restrict the migration of volumes attached to VMs in Starting state (#9725)
  server, plugin: enhance storage stats for IOPS (#10034)
  Introducing granular command timeouts global setting (#9659)
  Improve logging to include more identifiable information (#9873)
2025-01-08 14:01:19 +01:00
Daan Hoogland
ab76d3c9cc merge errors fixed 2025-01-08 14:00:41 +01:00
Daan Hoogland
4f9c3492ec Merge release branch 4.19 to 4.20
* 4.19:
  Restrict the migration of volumes attached to VMs in Starting state (#9725)
2025-01-08 13:23:42 +01:00
Felipe
21416cd403
Restrict the migration of volumes attached to VMs in Starting state (#9725)
Co-authored-by: Bernardo De Marco Gonçalves <bernardomg2004@gmail.com>
2025-01-08 08:49:21 +01:00
Abhishek Kumar
bd488c4bba
server, plugin: enhance storage stats for IOPS (#10034)
Adds framework layer change to allow retrieving and storing IOPS stats for storage pools. Custom `PrimaryStoreDriver` can implement method - `getStorageIopsStats` for returning IOPS stats. Existing method `getUsedIops` can also be overridden by such plugins when only used IOPS is returned.
For testing purpose, implementation has been added for simulator hypervisor plugin to return capacity and used IOPS for a pool.
For local storage pool, implementation has been added using iostat to return currently used IOPS.
StoragePoolResponse class has been updated to return IOPS values which allows showing IOPS values in UI for different storage pool related views and APIs.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-07 17:17:12 +05:30
Harikrishna
9bc283e5c2
Introducing granular command timeouts global setting (#9659)
* Introducing granular command timeouts global setting

* fix marvin tests

* Fixed log messages

* some more log message fix

* Fix empty value setting

* Converted the global setting to non-dynamic

* set wait on command only when granular wait is defined. This is to keep the backward compatibility

* Improve error logging
2025-01-07 17:06:32 +05:30
Vishesh
a4224e58cc
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin

* Update logging for scaleio plugin

* Improve logging to include more identifiable information for default volume storage plugin

* Improve logging to include more identifiable information for agent managers

* Improve logging to include more identifiable information for Listeners

* Replace ids with objects or uuids


* Improve logging to include more identifiable information for engine

* Improve logging to include more identifiable information for server

* Fixups in engine

* Improve logging to include more identifiable information for plugins

* Improve logging to include more identifiable information for Cmd classes

* Fix toString method for StorageFilterTO.java
2025-01-06 16:42:37 +05:30
Daan Hoogland
30b2588c06 Merge release branch 4.20 to main
* 4.20:
  log name change after merge forward
  check tags while fetching storage pool for importing vm (#9764)
  UI: Add cluster arch type to the zone creation wizard (#10080)
2025-01-03 10:17:28 +01:00
Daan Hoogland
cfafcaeb01 log name change after merge forward 2025-01-03 10:16:44 +01:00
Daan Hoogland
79316450d1 Merge release branch 4.19 to 4.20
* 4.19:
  check tags while fetching storage pool for importing vm (#9764)
2025-01-03 09:46:58 +01:00
John Bampton
5bae1188ff
pre-commit fix mixed line endings in XML files (#10148)
https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#mixed-line-ending
2025-01-03 09:42:09 +01:00
Vishesh
d8c321d57a
check tags while fetching storage pool for importing vm (#9764) 2025-01-03 09:05:54 +01:00
Daan Hoogland
2daffa34f2 Merge release branch 4.20 to main
* 4.20:
  VR: fix site-2-site VPN if split connections is enabled (#10067)
  UI: fix cannot open 'Edit tags' modal for static routes (#10065)
  Update ownership selection component to be language independent (#10052)
  Support to enable/disable VM High Availability manager and related alerts (#10118)
2024-12-30 13:35:30 +01:00
Suresh Kumar Anaparti
330ed25a6c
Support to enable/disable VM High Availability manager and related alerts (#10118)
- Adds new config 'vm.ha.enabled'  with Zone scope, to enable/disable VM High Availability manager. This is enable by default (for backward compatibilty).
  When enabled, the VM HA WorkItems (for VM Stop, Restart, Migration, Destroy) can be created and the scheduled items are executed.
  When disabled, new VM HA WorkItems are not allowed and the scheduled items are retried until max retries configured at 'vm.ha.migration.max.retries' (executed in case HA is re-enabled during retry attempts), and then purged after 'time.between.failures' by the cleanup thread that runs regularly at 'time.between.cleanup'.
- Adds new config 'vm.ha.alerts.enabled' with Zone scope, to enable/disable alerts for the VM HA operations. This is enabled by default.
2024-12-26 17:45:32 +05:30
Daan Hoogland
9295a1624d Merge release branch 4.20 to main
* 4.20:
  VR: apply iptables rules when add/remove static routes (#10064)
  Certificate and VM hostname validation improvements (#10051)
  set ulimit for server according to redhat spec (#10040)
  kvm-storage: provide isVMMigrate information to storage plugins (#10093)
  Allow config drive deletion of migrated VM, on host maintenance (#10045)
  linstor: improve heartbeat check with also asking linstor (#10105)
  server: simplify role change validation (#9173)
  UI: create VPC network offering with conserve mode (#10082)
  server: fix typo removeaccessvpn in VirtualRouterElement (#10086)
  UI: remove duplicated Instance Name in Public IP details page (#10087)
  UI: Fixes in the Usage UI (#10000)
  SAML2: add cookie with HttpOnly too #10013 (#10047)
  ui: Allow font-awesome icon usage and optimise icon size inconsistency (#9744)
2024-12-20 14:37:49 +01:00
Daan Hoogland
b7f0aac519 Merge branch '4.19' into 4.20 2024-12-20 14:34:39 +01:00
Suresh Kumar Anaparti
188eacd9eb
Certificate and VM hostname validation improvements (#10051)
* Certificate and VM hostname validation improvements

* Improve certificate name validation and some code/log improvements
2024-12-20 10:18:11 +05:30
Bernardo De Marco Gonçalves
73c3339bf1
List users by their authentication source (#10115) 2024-12-19 14:12:55 +01:00
Suresh Kumar Anaparti
b4ad04badf
Allow config drive deletion of migrated VM, on host maintenance (#10045) 2024-12-18 09:12:28 +01:00
Abhishek Kumar
8639ba8b01
server: simplify role change validation (#9173)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2024-12-14 20:26:32 +01:00
Ruben Bosch
54bc150140
fix typo in SnapshotManager.java (#10104) 2024-12-14 17:56:40 +01:00
Wei Zhou
a278849507
server: fix typo removeaccessvpn in VirtualRouterElement (#10086) 2024-12-13 09:10:54 +01:00
Bernardo De Marco Gonçalves
f75a194c09
Persist IP addresses related to VM access via CPVM (#9534) 2024-12-10 11:43:17 +01:00
Rohit Yadav
cb530c1749 Merge remote-tracking branch 'origin/4.20' 2024-12-09 11:36:40 +05:30
Felipe
810c4108c7
engine/schema: Add access modifier to VolumeVO (#9394)
* added private access modifiers to VolumeVO and fixed an occurrence where a field was being accessed without a getter

* renamed the field hostip to hostIp and removed duplicated methods
https://github.com/apache/cloudstack/pull/9394#issuecomment-2264816580
2024-12-06 08:51:42 +01:00
Daan Hoogland
205ebfb8b5 Merge release branch 4.20 to main
* 4.20:
  UI: Fix userdata and load balancer selection (#10016)
  Prevent password updates for SAML and LDAP users (#9999)
  cloudstack-migrate-databases: sql AND added (#10033)
  engine/schema: move SQLs to 4.20.0 to 4.20.1 upgrade (#10018)
  Remove user from project before deletion (#10008)
  Simplify validation for creating volume templates via UI (#9828)
2024-12-04 13:05:32 +01:00
Daan Hoogland
5ee61c226a Merge branch '4.19' into 4.20 2024-12-04 13:04:18 +01:00
Bernardo De Marco Gonçalves
52584d93dc
Prevent password updates for SAML and LDAP users (#9999) 2024-12-04 11:17:27 +01:00
Fabricio Duarte
ef1a58d837
Remove user from project before deletion (#10008)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-12-04 08:06:22 +01:00
Henrique Sato
4ac4d9cf29
API to validate Quota activation rule (#9605)
* API to validate Quota activation rule

* Apply suggestions from code review

Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>

* Use constants

---------

Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2024-12-03 14:41:25 -03:00
Daan Hoogland
9b6f9b5f7d Merge release branch 4.20 to main
* 4.20:
  UI: Tooltip on the host information card to display the CPU speed in MHz and the memory value in MB (to 3 decimal places) (#9971)
  UI: Allow accounts of the `User` type to add other accounts or users to projects through UI (#9927)
  enable to create VPC portfowarding rules with source cidr (#7081)
  Add new column `last_id` to the table volumes (#9759)
  Allow VMWare import via another host (#9787)
  Linstor: add support for ISO block devices and direct download (#9792)
  get expunged VM data for job result (#9949)
  fix section divider display on auth page (#9966)
2024-12-03 16:33:51 +01:00
Daan Hoogland
da54234585 Merge branch '4.19' into 4.20.merge 2024-12-03 16:32:15 +01:00
Wei Zhou
da94ae2c1c
Merge remote-tracking branch 'apache/4.20' 2024-12-03 09:44:42 +01:00
Vishesh
52e7b41a54
Add support for functionality in hypervisor class (#9736)
* Add support for functionality in hypervisor class

* Address comments

* address comments
2024-12-03 12:20:59 +05:30
Abhisar Sinha
d17de834a5
Disable API Key Access for users, accounts and domains (#9741)
* cli changes to update user/account, list by apikeyaccess, domain level setting

* UI changes for updating user/account and searchfilter in listview

* make the api parameters and setting accessible only to root admin

* revert changes to ui/package-lock.json

* minor changes to description strings

* UT for ApiServer and AccountManagerImpl classes

* fix pre-commit failure

* Added a constant for the string System

* UT for searchForUsers and searchForAccounts

* Fix marvin test error

* Update schema to use idempotent add column

* Fix `updateTemplatePermission` when the UI is set to a language other than English (#9766)

* Fix updateTemplatePermission UI in non-english language

* Improve fix

---------

Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>

* Added user name uuid to logging

* Add events when api key access is changed via api or config setting

* fix the userid for api key access update event

* Fix ut failure after event logging

* Convert drop down to radio-button in edit user and account

* Add ApiKeyAccess status in User InfoCard for Users if Api key is generated

* Return apiKeyAccess in user and account response only for Root Admin

* fixed noredist build failure

* Show apikeyaccess on the left panel in the user view for root admins as well

* don't show divider if apiKeyAccess is not shown to user

* Fix events generated to set Username, Account and Domain of the caller correctly

* cli changes to update user/account, list by apikeyaccess, domain level setting

* UI changes for updating user/account and searchfilter in listview

* make the api parameters and setting accessible only to root admin

* revert changes to ui/package-lock.json

* minor changes to description strings

* UT for ApiServer and AccountManagerImpl classes

* fix pre-commit failure

* Added a constant for the string System

* UT for searchForUsers and searchForAccounts

* Fix marvin test error

* Update schema to use idempotent add column

* Added user name uuid to logging

* Add events when api key access is changed via api or config setting

* fix the userid for api key access update event

* Fix ut failure after event logging

* Convert drop down to radio-button in edit user and account

* Add ApiKeyAccess status in User InfoCard for Users if Api key is generated

* Return apiKeyAccess in user and account response only for Root Admin

* fixed noredist build failure

* Show apikeyaccess on the left panel in the user view for root admins as well

* don't show divider if apiKeyAccess is not shown to user

* Fix events generated to set Username, Account and Domain of the caller correctly

* Added DB upgrade path from 42000 to 42010

---------

Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Lucas Martins <56271185+lucas-a-martins@users.noreply.github.com>
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-12-03 12:10:54 +05:30
Abhisar Sinha
ef6c0c443d
Prepend VPC name to VPC network tier name (#9780)
* Fix `updateTemplatePermission` when the UI is set to a language other than English (#9766)

* Fix updateTemplatePermission UI in non-english language

* Improve fix

---------

Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>

* Prepend vpc name to vpc tier network name based on global setting

* Added UT for createVpcGuestNetwork

* rename connector to delimiter and add configKey.Category.Network

* Move setting the name to a new method

---------

Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Lucas Martins <56271185+lucas-a-martins@users.noreply.github.com>
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-12-03 12:06:00 +05:30
Wei Zhou
8a1da3804c
Resize volume: add pool capacity disablethreshold for resize and allow volume auto migration (#9761)
* server: add global settings for volume resize

* resizeVolume: support automigrate

* Address Suresh's comments

* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ResizeVolumeCmd.java

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>

* address Suresh's comments

* UI: add autoMigrate to resizeVolume

* resizevolume: add unit tests

* resizevolume: add unit test for Allocated volume

---------

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-12-02 10:28:14 +05:30
Wei Zhou
34056d956c
Improvement: management server peer states (#9885)
* Improvement: management server peer states

* Update pr9885: consider new mgmt server node which has msId=managementServerNodeId

* Update pr9885: update global config description

* Update pr9885: update label on UI

* framework: Do not update mshost_peer when mgmt server is Up as it will be updated by status update

* mgmt: Update state to Up when mgmt server writes heartbeat to db

* mgmt: change Service IP to Management IP

---------

Co-authored-by: Boris Stoyanov - a.k.a Bobby <bss.stoyanov@gmail.com>
2024-12-02 10:26:20 +05:30