141 Commits

Author SHA1 Message Date
vanzan01
e6c7a71d40
Add yamllint pre-commit hook for YAML file standardization (#11017)
* Add yamllint pre-commit hook for YAML file standardization

Add yamllint hook to .pre-commit-config.yaml to standardize YAML files
across the project. Created .yamllint.yml configuration with relaxed
rules to accommodate existing files. Fixed colon spacing in .asf.yaml
to pass yamllint checks.

Closes #10974

Signed-off-by: vanzan01 <vanzan.1250@gmail.com>

* Add extra eol to .yamllint.yml

Address review comment from DaanHoogland

Signed-off-by: vanzan01 <vanzan.1250@gmail.com>

* Move .yamllint.yml to .github/linters/ and improve file pattern

- Move .yamllint.yml to .github/linters/ directory for better organization
- Update pre-commit config to point to new location
- Change file pattern to \.ya?ml$ as suggested

Address review feedback from jbampton

Signed-off-by: vanzan01 <vanzan.1250@gmail.com>

* Update .pre-commit-config.yaml

Co-authored-by: John Bampton <jbampton@users.noreply.github.com>

* Make yamllint more forgiving and exclude only k8s templates

- Remove extensive file exclusions, keep only k8s templates
- Increase line length limit to 400 chars (GitHub Actions friendly)
- Allow flexible colon spacing (max-spaces-after: -1)
- Disable document-start requirement

This provides linting value while handling infrastructure patterns.

Signed-off-by: vanzan01 <vanzan.1250@gmail.com>

* Update .pre-commit-config.yaml

Co-authored-by: John Bampton <jbampton@users.noreply.github.com>

---------

Signed-off-by: vanzan01 <vanzan.1250@gmail.com>
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
2025-09-20 03:16:00 +10:00
John Bampton
6eb27da5e4
Fix spelling in Java and Python files; update the ignored words list codespell.txt (#11285)
Co-authored-by: dahn <daan.hoogland@gmail.com>
2025-09-17 17:24:04 +02:00
John Bampton
0d9c5065de
Add CodeQL Analysis for GitHub Actions (#11300) 2025-09-17 12:33:08 +02:00
John Bampton
9317a46513
Add all workflow buttons to README (#11406) 2025-09-15 09:38:47 +02:00
Wei Zhou
574ed7857b
.github: Update to JDK 17 in ci.yml and build.yml (#11030)
* .github: Update to JDK 17 in build.yml

* .github: update JDK in ci.yml

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-09-01 10:14:09 +02:00
Vishesh
f6ad184ea2
Feature: Add support for GPU with KVM hosts (#11143)
This PR allows attaching of GPU devices via PCI, mdev or VF to an Instance for KVM.

It allows the operator to discover the GPU devices on the KVM host and create a Compute Offering with GPU support based on the available GPU devices on the host. Once the operator has created the Compute offering, it can be used by users to launch Instances with GPU devices.
2025-07-29 13:46:24 +05:30
Harikrishna
cca8b2fef9
Extensions Framework & Orchestrate Anything (#9752)
The Extensions Framework in Apache CloudStack is designed to provide a flexible and standardised mechanism for integrating external systems and custom workflows into CloudStack’s orchestration process. By defining structured hook points during key operations—such as virtual machine deployment, resource preparation, and lifecycle events—the framework allows administrators and developers to extend CloudStack’s behaviour without modifying its core codebase.
2025-07-28 10:41:17 +05:30
John Bampton
fb3bb3fd56
pre-commit: upgrade markdownlint to the latest version (#11066) 2025-07-23 17:23:20 +02:00
Daan Hoogland
0dbd761fbb Merge branch '4.20' 2025-07-14 11:17:08 +02:00
Daan Hoogland
61e74e086b Merge branch '4.19' into 4.20 2025-07-14 10:37:06 +02:00
Wei Zhou
67a1ea35f4
.github: restrict codecov in UI build to apache/cloudstack repo (#11158) 2025-07-10 03:44:46 +02:00
Manoj Kumar
7632814cd2
Instance lease: Allow deployment of instances with lease duration and leaseexpiry action (#10560)
* FR-248: Instance lease, WIP commit

* insert lease expiry into db and use that to filter exiring vms, add asyncjobmanager

* Add leaseDuration and leaseExpiryAction in Service offering create flow

* Update listVM cmd to allow listing only leased instances

* Add methods to fetch instances for which lease is expiring in next days

* Changes included:
config key setup and configured for alert email
lease options in create and update vm screen
handle delete protection, edit vm, create vm
validated stop and detroy, delete protection

* Update UI screens for leased properties coming from config and service offering

* use global lock before running scheduler

* Unit tests

* Flow changes done in UI based on discussion

* Include view changes in schema upgrade files and use feature in various UI elements

* Added integration test for vm deployment, UI enhancements for user persona, bug fixes

* validate integration tests, minor ui changes and log messages

* fix build: moving configkey from setup to test itself

* Disable testAlert to unblock build and trim whitespaces in integration tests

* Address review comments

* Minor changes in EditVM screen

* Use ExecutorService instead of Timer and TimerTask

* Additional review comments

* Incorporate following changes:
1. Execute lease action once on the instance
2. Cancel lease on instance when feature is disabled
3. Relevant events when lease gets disabled, cancelled, executed
4. Disable associating lease after deployment
5. UI elements and flow changes
6. Changes based on feedback from demo

* Handle pr review comments

* address review comments

* move instance.lease.enabled config to VMLeaseManager interface

* bug fix in edit instance flow and reject api request for invalid values

* max allowed lease is for 100 years

* log instance ids for expired instance

* Fix config validation for value range and code coverage improvement

* fix lease expiry request failures in async

* dont use forced: true for StopVmCmd

* Update server/src/main/java/org/apache/cloudstack/vm/lease/VMLeaseManager.java

Co-authored-by: Vishesh <vishesh92@gmail.com>

* handle review comments

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2025-05-28 17:40:09 +05:30
Wei Zhou
f2a6a2ff13
.github: fix sonar checks (#10894)
* .github: fix sonar check

* .github: fix main sonar check

* add more pom.xml files
2025-05-22 12:07:00 +02:00
Wei Zhou
97c96a55ba
.github: fix sonar checks (#10894)
* .github: fix sonar check

* .github: fix main sonar check

* add more pom.xml files
2025-05-20 15:26:54 +02:00
Pearl Dsilva
1e5d133033 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-05-12 13:12:09 +05:30
Pearl Dsilva
f6d0590424 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-05-05 10:14:46 +05:30
Pearl Dsilva
d697cff898
Update dependency required for test_outofbandmanagement.py (#10805) 2025-05-02 17:34:17 +02:00
Pearl Dsilva
193d6ef52f Merge branch '4.20' of https://github.com/apache/cloudstack 2025-04-18 18:29:44 -04:00
Pearl Dsilva
a95e9d7f77 Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-04-18 18:29:16 -04:00
Wei Zhou
f055268fe2
.github: fix simulator CI caused by imcompatibility between python3.10 and nosetests (#10753) 2025-04-18 13:27:32 +02:00
dahn
1c626c884c
Update bug.yml 2025-01-20 17:06:59 +01:00
dahn
0ed6895db4
feature and bug templates (#10186) 2025-01-17 11:41:25 +01:00
John Bampton
0128046429
pre-commit update markdownlint; enable rules MD009,MD019; fix license header (#10145)
https://github.com/DavidAnson/markdownlint/blob/main/doc/md009.md

https://github.com/DavidAnson/markdownlint/blob/main/doc/md019.md
2024-12-31 13:02:16 +01:00
John Bampton
b48de4e023
pre-commit: enable markdownlint rule MD018 (#9134)
https://github.com/DavidAnson/markdownlint/blob/main/doc/md018.md

Co-authored-by: dahn <daan@onecht.net>
2024-12-27 12:28:24 +01:00
John Bampton
6a2c7b0220
pre-commit: enable markdownlint rule MD012 (#9364)
MD012 no-multiple-blanks - Multiple consecutive blank lines

https://github.com/DavidAnson/markdownlint/blob/main/doc/md012.md
2024-12-27 10:25:11 +01:00
Wei Zhou
47f60190a9
.github: fix test_certauthority_root in 4.19 (#10037) 2024-12-04 14:34:20 +01:00
Wei Zhou
d7ca05e5ba
Build: drop EL7 support, support JRE17 for packages and sonar check (#8609)
This PR fixes the issue with sonar check

```
Error:  Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184:sonar (default-cli) on project cloudstack:
Error:
Error:  The version of Java (11.0.22) used to run this analysis is deprecated, and SonarCloud no longer supports it. Please upgrade to Java 17 or later.
Error:  You can find more information here: https://docs.sonarsource.com/sonarcloud/appendices/scanner-environment/
```

main changes
- Support build/packaging using JDK17
- Still supports JDK11 for building
- Support JRE17 for use in production installation
- Drop EL7 support

The community packages will be still packaged using JDK11. 
If uses want, they can build by JDK17 as well.

Signed-off-by: Wei Zhou <wei.zhou@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-27 16:09:10 +05:30
nvazquez
b73f634ea6
Merge branch '4.19' 2024-08-06 12:39:13 -03:00
nvazquez
d541e9014d
Merge branch '4.18' into 4.19 2024-08-06 12:29:18 -03:00
Abhishek Kumar
2e0024e216
server, api: account and api entity access improvements
Fixes domain-admin access check to prevent unauthorized access.
Introduces a new non-dynamic global setting - api.allow.internal.db.ids
to control whether to allow using internal DB IDs as API parameters or
not. Default value for the global setting is false.

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-08-06 12:24:19 -03:00
John Bampton
2919cac643
pre-commit: add codespell to check spelling (#8197) 2024-07-12 09:44:14 +02:00
John Bampton
806692bcea
pre-commit: enable Python flake8 rule E272 (#9361)
There should be only one space before a keyword.
2024-07-10 09:17:25 +02:00
John Bampton
c6d3f6e623
pre-commit: enable Python flake8 rule E271 (#9358) 2024-07-09 10:17:53 +02:00
John Bampton
a1d8bab43f
Bump actions/cache to v4 (#9353)
https://github.com/actions/cache
2024-07-09 13:47:11 +05:30
John Bampton
b69cc0272d
pre-commit: enable Python Flake8 rule E713 (#9016)
Co-authored-by: Vishesh <vishesh92@gmail.com>
2024-07-08 09:00:18 +02:00
Vishesh
f6ceeab3b3
server: Enforce strict host tag check (#9017)
Documentation PR: https://github.com/apache/cloudstack-documentation/pull/398

Currently, an administrator can break host tag compatibility for a VM administrator by certain operations:
* deploy/start VM on a specific host
* migrate VM
* restore VM
* scale VM

This PR allows the user to specify tags which must be checked during these operations.

Global Settings
1. `vm.strict.host.tags` - A comma-separated list of tags for strict host check (Default - empty)
2. `vm.strict.resource.limit.host.tag.check` - Determines whether the resource limits tags are considered strict or not (Default - true)

During the above operations, we now check and throw an error if host tags compatibility is being broken for tags specified in `vm.strict.host.tags`. If `vm.strict.resource.limit.host.tag.check` is set to `true`, tags set in `resource.limit.host.tags` are also checked during these operations.
2024-06-25 14:42:17 +05:30
Abhishek Kumar
3e6900ac1a
api,server: purge expunged resources (#8999)
This PR introduces the functionality of purging removed DB entries for CloudStack entities (currently only for VirtualMachine). There would be three mechanisms for purging removed resources:

    Background task - CloudStack will run a background task which runs at a defined interval. Other parameters for this task can be controlled with new global settings.
    API - New admin-only API purgeExpungedResources. It will allow passing the following parameters - resourcetype, batchsize, startdate, enddate. Currently, API is not supported in the UI.
    Config for service offering. Service offerings can be created with purgeresources parameter which would allow purging resources immediately on expunge.

Following new global settings have been added:

    expunged.resources.purge.enabled: Default: false. Whether to run a background task to purge the expunged resources
    expunged.resources.purge.resources: Default: (empty). A comma-separated list of resource types that will be considered by the background task to purge the expunged resources. Currently only VirtualMachine is supported. An empty "value will result in considering all resource types for purging
    expunged.resources.purge.interval: Default: 86400. Interval (in seconds) for the background task to purge the expunged resources
    expunged.resources.purge.delay: Default: 300. Initial delay (in seconds) to start the background task to purge the expunged resources task.
    expunged.resources.purge.batch.size: Default: 50. Batch size to be used during expunged resources purging.
    expunged.resources.purge.start.time: Default: (empty). Start time to be used by the background task to purge the expunged resources. Use format yyyy-MM-dd or yyyy-MM-dd HH:mm:ss.
    expunged.resources.purge.keep.past.days: Default: 30. The number of days in the past from the execution time of the background task to purge the expunged resources for which the expunged resources must not be purged. To enable purging expunged resource till the execution of the background task, set the value to zero.
    expunged.resource.purge.job.delay: Default: 180. Delay (in seconds) to execute the purging of an expunged resource initiated by the configuration in the offering. Minimum value should be 180 seconds and if a lower value is set then the minimum value will be used.

Documentation PR: apache/cloudstack-documentation#397

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-06-20 11:34:44 +05:30
Abhishek Kumar
be552fdce9
feature: webhooks (#8674)
* api,server,ui: weebhoks feature

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* registry of message busses

* test bus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix and refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for webhook dispatch history

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes, initial ui

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for account webhook cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix remaining event bus usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for testing webhook dispatch

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* make element

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* buid fix

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for project delete check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add collapse in create

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui fix and refactor for eventditributor publish

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update org.json and add json validation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* schema fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wordings

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: improve progress button

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unrelated change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* search and count

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add payloadurl in info

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* positive progress

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix hmac key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* create webhook form fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor, address feedback

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* indentation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix filters

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove test eventbus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* default scope be Local

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add lifecycle smoke test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add test for webhook deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor - losgs and others

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* build fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* smoke test fix, log refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* get bean from all components

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: missing label

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add some more tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename setting

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0

* fix test delivery layout

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix webhook secret display

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add http to payloadurl when no scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* allow removing secretkey for webhook

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix update sslverification

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* disallow same payload url for same account

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix delivery with url w/o scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* api: listApis should return params based on caller

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/schema/src/main/resources/META-INF/db/schema-41900to42000.sql

* remove unique constraint for now

Constraint is present in Java code validations

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: add option to delete multiple deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add filter for deliveries, delete api start/endtime support

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* do not throw error when no deliveries removed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix deliveries table column sorting, time filter cancel

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove isDebugEnabled wrapping

* merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-06-10 10:40:12 +05:30
John Bampton
e159a593f1
Add markdownlint with pre-commit (#9078)
https://github.com/DavidAnson/markdownlint/tree/main?tab=readme-ov-file#rules--aliases

https://github.com/igorshubovych/markdownlint-cli?tab=readme-ov-file#use-with-pre-commit
2024-05-28 07:40:52 +02:00
Daan Hoogland
e61f3bae4d Merge branch '4.19' 2024-04-29 11:37:40 +02:00
Vishesh
08132acaa2
Fix restore VM with allocated root disk (#8977)
* Fix restore VM with allocated root disk

* Add e2e test for restore vm

* Add more checks for e2e test
2024-04-29 12:19:05 +05:30
Rohit Yadav
836ef9ec70 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

 Conflicts:
	systemvm/debian/opt/cloud/bin/cs/CsHelper.py
2024-04-25 09:55:09 +05:30
Rohit Yadav
4c27574570 .github: fix codecov action configuration
`token` is documented to work with codecov-action@v4

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-25 09:46:37 +05:30
Rohit Yadav
5745674da5 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-04-25 09:45:08 +05:30
Rohit Yadav
77df9ad4d7 codecov: fix Github action workflow
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-25 09:43:50 +05:30
Daan Hoogland
65eef59e0b Merge release branch 4.19 to main
* 4.19:
  UI: ignore error when list public ips for CKS clusters on Shared network (#8489)
  Infra25725 add codecov token to workflow (#8960)
2024-04-23 15:49:53 +02:00
dahn
0de469ab73
Infra25725 add codecov token to workflow (#8960) 2024-04-23 13:28:09 +02:00
Abhishek Kumar
b29ec2bf12 Merge remote-tracking branch 'apache/4.19' 2024-03-01 17:40:58 +05:30
Abhishek Kumar
9fd410be36 Merge remote-tracking branch 'apache/4.18' into 4.19 2024-03-01 17:34:27 +05:30
Vishesh
bbe56ae283
Update actions/setup-java from v3 to v4 (#8709) 2024-02-27 11:46:20 +05:30