3579 Commits

Author SHA1 Message Date
Abhishek Kumar
ae2ffbe40b Merge remote-tracking branch 'apache/4.19' into 4.20 2025-01-31 16:54:22 +05:30
Fabricio Duarte
4201a2dc11
Fix Usage job not executing successfully (#10244)
* Add `api_key_access` to Usage account table

* use the procedure in cloud_usage instead
2025-01-30 22:07:50 +01:00
Bernardo De Marco Gonçalves
a335feab6a
fix allocation of vmfs storage pools (#10201) 2025-01-30 18:44:48 +01:00
Bernardo De Marco Gonçalves
d9af9bdb36
fix SQL syntax erros and target 419 (#10273) 2025-01-30 15:51:55 +01:00
Abhishek Kumar
b93589b5bd
server: reset 2fa user configuration on incomplete setup (#10247)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-30 15:51:04 +01:00
Daan Hoogland
717ce981d4 Merge release branch 4.19 to 4.20
* 4.19:
  extra null guard (#10264)
2025-01-28 14:33:49 +01:00
Abhishek Kumar
33a37da9ec
server: investigate pending HA work when executing in new MS session (#10167)
For HA work items that are created for host state change, checks must be
done when execution is called in a new management server session.

A new column, reason, has been added in cloud.op_ha_work table to track
the reason for HA work.

When HighAvailabilityManager starts it finds and puts all pending HA
work items in Investigating state. During execution of the HA work if it
is found in investigating state, checks are done to verify if the work
is still valid. If the jobs is found to be invalid it is cancelled.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-28 14:39:31 +05:30
dahn
f652ad0d98
extra null guard (#10264) 2025-01-27 14:14:31 +01:00
Daan Hoogland
34d2a3bc86 Merge branch '4.19' into 4.20 2025-01-24 17:01:42 +01:00
dahn
0a77eb7f85
deal with NPE during host reconnect (#10158)
* log to see what command is being processed

* exception names
2025-01-24 15:39:56 +05:30
Daan Hoogland
9f594c9699 Merge branch '4.19' into 4.20 2025-01-20 11:53:32 +01:00
Fabricio Duarte
2300c1caa3
Add project-user association normalization script to 4.20.1 upgrade (#10116) 2025-01-16 16:18:30 +01:00
Bernardo De Marco Gonçalves
796bd4f72c
Clean up network permissions on account deletion (#10176) 2025-01-15 17:38:20 +01:00
Daan Hoogland
e2cfddb1b6 Merge branch '4.19' into 4.20 2025-01-13 11:23:14 +01:00
Wei Zhou
7adc732992
upgrade: consider multiple hypervisors and secondary storages (#10046) 2025-01-13 09:28:18 +01:00
Abhishek Kumar
bd488c4bba
server, plugin: enhance storage stats for IOPS (#10034)
Adds framework layer change to allow retrieving and storing IOPS stats for storage pools. Custom `PrimaryStoreDriver` can implement method - `getStorageIopsStats` for returning IOPS stats. Existing method `getUsedIops` can also be overridden by such plugins when only used IOPS is returned.
For testing purpose, implementation has been added for simulator hypervisor plugin to return capacity and used IOPS for a pool.
For local storage pool, implementation has been added using iostat to return currently used IOPS.
StoragePoolResponse class has been updated to return IOPS values which allows showing IOPS values in UI for different storage pool related views and APIs.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-07 17:17:12 +05:30
Harikrishna
9bc283e5c2
Introducing granular command timeouts global setting (#9659)
* Introducing granular command timeouts global setting

* fix marvin tests

* Fixed log messages

* some more log message fix

* Fix empty value setting

* Converted the global setting to non-dynamic

* set wait on command only when granular wait is defined. This is to keep the backward compatibility

* Improve error logging
2025-01-07 17:06:32 +05:30
Vishesh
a4224e58cc
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin

* Update logging for scaleio plugin

* Improve logging to include more identifiable information for default volume storage plugin

* Improve logging to include more identifiable information for agent managers

* Improve logging to include more identifiable information for Listeners

* Replace ids with objects or uuids


* Improve logging to include more identifiable information for engine

* Improve logging to include more identifiable information for server

* Fixups in engine

* Improve logging to include more identifiable information for plugins

* Improve logging to include more identifiable information for Cmd classes

* Fix toString method for StorageFilterTO.java
2025-01-06 16:42:37 +05:30
Suresh Kumar Anaparti
330ed25a6c
Support to enable/disable VM High Availability manager and related alerts (#10118)
- Adds new config 'vm.ha.enabled'  with Zone scope, to enable/disable VM High Availability manager. This is enable by default (for backward compatibilty).
  When enabled, the VM HA WorkItems (for VM Stop, Restart, Migration, Destroy) can be created and the scheduled items are executed.
  When disabled, new VM HA WorkItems are not allowed and the scheduled items are retried until max retries configured at 'vm.ha.migration.max.retries' (executed in case HA is re-enabled during retry attempts), and then purged after 'time.between.failures' by the cleanup thread that runs regularly at 'time.between.cleanup'.
- Adds new config 'vm.ha.alerts.enabled' with Zone scope, to enable/disable alerts for the VM HA operations. This is enabled by default.
2024-12-26 17:45:32 +05:30
Daan Hoogland
b7f0aac519 Merge branch '4.19' into 4.20 2024-12-20 14:34:39 +01:00
Suresh Kumar Anaparti
b4ad04badf
Allow config drive deletion of migrated VM, on host maintenance (#10045) 2024-12-18 09:12:28 +01:00
Felipe
810c4108c7
engine/schema: Add access modifier to VolumeVO (#9394)
* added private access modifiers to VolumeVO and fixed an occurrence where a field was being accessed without a getter

* renamed the field hostip to hostIp and removed duplicated methods
https://github.com/apache/cloudstack/pull/9394#issuecomment-2264816580
2024-12-06 08:51:42 +01:00
dahn
0eb94a1396
move upgrade code to the right file (#10036)
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2024-12-04 14:30:39 +01:00
Daan Hoogland
8b9db0869f Merge release branch 4.19 to 4.20
* 4.19:
  move sql code to the right file (#10035)
2024-12-04 14:29:03 +01:00
dahn
762d5493dd
move sql code to the right file (#10035)
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2024-12-04 14:28:04 +01:00
Daan Hoogland
5ee61c226a Merge branch '4.19' into 4.20 2024-12-04 13:04:18 +01:00
Wei Zhou
b3dc402aa8
engine/schema: move SQLs to 4.20.0 to 4.20.1 upgrade (#10018) 2024-12-04 08:11:28 +01:00
Fabricio Duarte
ef1a58d837
Remove user from project before deletion (#10008)
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-12-04 08:06:22 +01:00
Daan Hoogland
da54234585 Merge branch '4.19' into 4.20.merge 2024-12-03 16:32:15 +01:00
Abhisar Sinha
d17de834a5
Disable API Key Access for users, accounts and domains (#9741)
* cli changes to update user/account, list by apikeyaccess, domain level setting

* UI changes for updating user/account and searchfilter in listview

* make the api parameters and setting accessible only to root admin

* revert changes to ui/package-lock.json

* minor changes to description strings

* UT for ApiServer and AccountManagerImpl classes

* fix pre-commit failure

* Added a constant for the string System

* UT for searchForUsers and searchForAccounts

* Fix marvin test error

* Update schema to use idempotent add column

* Fix `updateTemplatePermission` when the UI is set to a language other than English (#9766)

* Fix updateTemplatePermission UI in non-english language

* Improve fix

---------

Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>

* Added user name uuid to logging

* Add events when api key access is changed via api or config setting

* fix the userid for api key access update event

* Fix ut failure after event logging

* Convert drop down to radio-button in edit user and account

* Add ApiKeyAccess status in User InfoCard for Users if Api key is generated

* Return apiKeyAccess in user and account response only for Root Admin

* fixed noredist build failure

* Show apikeyaccess on the left panel in the user view for root admins as well

* don't show divider if apiKeyAccess is not shown to user

* Fix events generated to set Username, Account and Domain of the caller correctly

* cli changes to update user/account, list by apikeyaccess, domain level setting

* UI changes for updating user/account and searchfilter in listview

* make the api parameters and setting accessible only to root admin

* revert changes to ui/package-lock.json

* minor changes to description strings

* UT for ApiServer and AccountManagerImpl classes

* fix pre-commit failure

* Added a constant for the string System

* UT for searchForUsers and searchForAccounts

* Fix marvin test error

* Update schema to use idempotent add column

* Added user name uuid to logging

* Add events when api key access is changed via api or config setting

* fix the userid for api key access update event

* Fix ut failure after event logging

* Convert drop down to radio-button in edit user and account

* Add ApiKeyAccess status in User InfoCard for Users if Api key is generated

* Return apiKeyAccess in user and account response only for Root Admin

* fixed noredist build failure

* Show apikeyaccess on the left panel in the user view for root admins as well

* don't show divider if apiKeyAccess is not shown to user

* Fix events generated to set Username, Account and Domain of the caller correctly

* Added DB upgrade path from 42000 to 42010

---------

Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Lucas Martins <56271185+lucas-a-martins@users.noreply.github.com>
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-12-03 12:10:54 +05:30
Wei Zhou
8a1da3804c
Resize volume: add pool capacity disablethreshold for resize and allow volume auto migration (#9761)
* server: add global settings for volume resize

* resizeVolume: support automigrate

* Address Suresh's comments

* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ResizeVolumeCmd.java

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>

* address Suresh's comments

* UI: add autoMigrate to resizeVolume

* resizevolume: add unit tests

* resizevolume: add unit test for Allocated volume

---------

Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-12-02 10:28:14 +05:30
Wei Zhou
34056d956c
Improvement: management server peer states (#9885)
* Improvement: management server peer states

* Update pr9885: consider new mgmt server node which has msId=managementServerNodeId

* Update pr9885: update global config description

* Update pr9885: update label on UI

* framework: Do not update mshost_peer when mgmt server is Up as it will be updated by status update

* mgmt: Update state to Up when mgmt server writes heartbeat to db

* mgmt: change Service IP to Management IP

---------

Co-authored-by: Boris Stoyanov - a.k.a Bobby <bss.stoyanov@gmail.com>
2024-12-02 10:26:20 +05:30
Lucas Martins
5886780240
Change vmsnapshot.max config to be dynamic (#9883)
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-11-28 14:49:05 -03:00
Lucas Martins
dc0ebe985c
Remove info about the environment from exception when revert VM snapshot (#9944)
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-11-28 14:42:34 -03:00
Rodrigo D. Lopez
4189bac8e0
enable to create VPC portfowarding rules with source cidr (#7081)
Co-authored-by: Lopez <rodrigo@scclouds.com.br>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2024-11-28 17:53:07 +01:00
Felipe
a8cb7abca3
Add new column last_id to the table volumes (#9759)
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-28 17:51:53 +01:00
João Jandre
c63c7ee63e Updating pom.xml version numbers for release 4.20.1.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-27 11:40:45 -03:00
João Jandre
2fe3fcef7c Updating pom.xml version numbers for release 4.20.0.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-11-19 08:54:07 -03:00
João Jandre
b38ee63c48 Merge branch '4.19' 2024-11-13 10:47:24 -03:00
Wei Zhou
adbf370909
Revert "storage: fix private templates are not copied to new image store (#9206)" (#9839)
This reverts commit e06f80e899127abe45c10ff2d38420a770ceb076.
2024-11-13 10:31:31 -03:00
Daan Hoogland
f7b7339bcf Merge branch '4.19' 2024-11-07 11:33:50 +01:00
Abhisar Sinha
22c6f08f77
engine-storage: Set SecretConsumerDetail for VM live migration with storage on shared NFS (#9222)
* engine-storage-datamotion: Set SecretConsumerDetail for VM live migration with storage on shared NFS

* VM live migration - powerflex encrypted volume

* rename isPowerFlex
2024-11-07 14:45:04 +05:30
Wei Zhou
019f2c60ce
server: fix vm_stats records are not removed if batchsize is 0 (#9818) 2024-10-18 14:52:46 +05:30
Wei Zhou
f10d9fecbe
fix forward-merge errors 2024-10-15 09:21:33 +02:00
João Jandre
554ea220f0 Fix merge errors 2024-10-11 14:12:18 -03:00
Daan Hoogland
dbfc7f23a7 Merge branch '4.19' 2024-10-11 17:59:46 +02:00
João Jandre
e666dca403
linked clone for file based storage (#8911) 2024-10-02 13:15:29 -03:00
João Jandre
bf2cedea79 Merge branch 4.19 to main 2024-09-26 09:01:22 -03:00
Abhishek Kumar
53aa92199b
server: fix nfs version option during mount (#9559)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-09-25 16:47:16 +02:00
Layon
a506bf1f2e
refactor schema-41910to42000.sql (#9680)
Co-authored-by: layon.oliveira <layon.oliveira@scclouds.com.br>
2024-09-17 10:15:56 -03:00