37782 Commits

Author SHA1 Message Date
Nicolas Vazquez
16c60c7528
Support Direct Download on Ceph (#11069) 2025-06-24 14:19:01 +02:00
Abhishek Kumar
0bd35a5403
ui: fix missing changes from #10814 (#11060)
Some changes from PR #10814 are missing in DeployVM and DeployVnf
wizards causing warnings in teh browser console.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-23 17:48:46 +02:00
Suyang(Dawson) Chen
5b7c3b475e
docs: Update INSTALL.md for frontend build instructions (#11057)
Co-authored-by: Bernardo De Marco Gonçalves <bernardomg2004@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2025-06-19 10:17:34 +02:00
Nicolas Vazquez
6adfda2818
CKS Enhancements (#9102)
CKS Enhancements:

* Ability to specify different compute or service offerings for different types of CKS cluster nodes – worker, master or etcd

* Ability to use CKS ready custom templates for CKS cluster nodes

* Add and Remove external nodes to and from a kubernetes cluster

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Update remove node timeout global setting

* CKS/NSX : Missing variables in worker nodes

* CKS: Fix ISO attach logic

* CKS: Fix ISO attach logic

* address comment

* Fix Port - Node mapping when cluster is scaled in the presence of external node(s)

* CKS: Externalize control and worker node setup wait time and installation attempts

* Fix logger

* Add missing headers and fix end of line on files

* CKS Mark Nodes for Manual Upgrade and Filter Nodes to add to CKS cluster from the same network

* Add support to deploy CKS cluster nodes on hosts dedicated to a domain

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* Support unstacked ETCD

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix CKS cluster scaling and minor UI improvement

* Reuse k8s cluster public IP for etcd nodes and rename etcd nodes

* Fix DNS resolver issue

* Update UDP active monitor to ICMP

* Add hypervisor type to CKS cluster creation to fix CKS cluster creation when External hosts added

* Fix build

* Fix logger

* Modify hypervisor param description in the create CKS cluster API

* CKS delete fails when external nodes are present

* CKS delete fails when external nodes are present

* address comment

* Improve network rules cleanup on failure adding external nodes to CKS cluster

* UI: Fix etcd template was not honoured

* UI: Fix etcd template was not honoured

* Refactor

* CKS: Exclude etcd nodes when calculating port numbers

* Fix network cleanup in case of CKS cluster failure

* Externalize retries and inverval for NSX segment deletion

* Fix CKS scaling when external node(s) present in the cluster

* CKS: Fix port numbers displayed against ETCD nodes

* Add node version details to every node of k8s cluster - as we now support manual upgrade

* Add node version details to every node of k8s cluster - as we now support manual upgrade

* update column name

* CKS: Exclude etcd nodes when calculating port numbers

* update param name

* update param

* UI: Fix CKS cluster creation templates listing for non admins

* CKS: Prevent etcd node start port number to coincide with k8s cluster start port numbers

* CKS: Set default kubernetes cluster node version to the kubernetes cluster version on upgrade

* CKS: Set default kubernetes cluster node version to the kubernetes cluster version on upgrade

* consolidate query

* Fix upgrade logic

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix CKS cluster version upgrade

* CKS: Fix etcd port numbers being skipped

* Fix CKS cluster with etcd nodes on VPC

* Move schema and upgrade for 4.20

* Fix logger

* Fix after rebasing

* Add support for using different CNI plugins with CKS

* Add support for using different CNI plugins with CKS

* remove unused import

* Add UI support and list cni config API

* necessary UI changes

* add license

* changes to support external cni

* UI changes

* Fix NPE on restarting VPC with additional public IPs

* fix merge conflict

* add asnumber to create k8s svc layer

* support cni framework to use as-numbers

* update code

* condition to ignore undefined jinja template variables

* CKS: Do not pass AS number when network ID is passed

* Fix deletion of Userdata / CNI Configuration in projects

* CKS: Add CNI configuration details to the response and UI

* Explicit events for registering cni configuration

* Add Delete cni configuration API

* Fix CKS deployment when using VPC tiers with custom ACLs

* Fix DNS list on VR

* CKS: Use Network offering of the network passed during CKS cluster creation to get the AS number

* CKS cluster with guest IP

* Fix: Use control node guest IP as join IP for external nodes addition

* Fix DNS resolver issue

* Improve etcd indexing - start from 1

* CKS: Add external node to a CKS cluster deployed with etcd node(s) successfully

* CKS: Add external node to a CKS cluster deployed with etcd node(s) successfully

* simplify logic

* Tweak setup-kube-system script for baremetal external nodes

* Consider cordoned nodes while getting ready nodes

* Fix CKS cluster scale calculations

* Set token TTL to 0 (no expire) for external etcd

* Fix missing quotes

* Fix build

* Revert PR 9133

* Add calico commands for ens35 interface

* Address review comments: plan CKS cluster deployment based on the node type

* Add qemu-guest-agent dependency for kvm based templates

* Add marvin test for CKS clusters with different offerings per node type

* Remove test tag

* Add marvin test and fix update template for cks and since annotations

* Fix marvin test for adding and removing external nodes

* Fix since version on API params

* Address review comments

* Fix unit test

* Address review comments

* UI: Make CKS public templates visible to non-admins on CKS cluster creation

* Fix linter

* Fix merge error

* Fix positional parameters on the create kubernetes ISO script and make the ETCD version optional

* fix etcd port displayed

* Further improvements to CKS  (#118)

* Multiple nics support on Ubuntu template

* Multiple nics support on Ubuntu template

* supports allocating IP to the nic when VM is added to another network - no delay

* Add option to select DNS or VR IP as resolver on VPC creation

* Add API param and UI to select option

* Add column on vpc and pass the value on the databags for CsDhcp.py to fix accordingly

* Externalize the CKS Configuration, so that end users can tweak the configuration before deploying the cluster

* Add new directory to c8 packaging for CKS config

* Remove k8s configuration from resources and make it configurable

* Revert "Remove k8s configuration from resources and make it configurable"

This reverts commit d5997033ebe4ba559e6478a64578b894f8e7d3db.

* copy conf to mgmt server and consume them from there

* Remove node from cluster

* Add missing /opt/bin directory requrired by external nodes

* Login to a specific Project view

* add indents

* Fix CKS HA clusters

* Fix build

---------

Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>

* Add missing headers

* Fix linter

* Address more review comments

* Fix unit test

* Fix scaling case for the same offering

* Revert "Login to a specific Project view"

This reverts commit 95e37563f48573780b07a038a7f48c0bc04e9b64.

* Revert "Fix CKS HA clusters" (#120)

This reverts commit 8dac16aa359faa6500ea1e1ce548169cfd08331a.

* Apply suggestions from code review about user data

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/userdata/BaseRegisterUserDataCmd.java

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Refactor column names and schema path

* Fix scaling for non existing previous offering per node type

* Update node offering entry if there was an existing offering but a global service offering has been provided on scale

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2025-06-19 11:00:42 +05:30
Davi Torres
4662ffc424
Changes baseurl for downloading kubectl (#11040) 2025-06-17 16:40:28 +02:00
Felipe
f8c4121ea3
Add access modifiers to VirtualMachineTO (#9277)
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2025-06-17 08:35:04 +02:00
Suyang(Dawson) Chen
6aaaa838b2
docs: revise INSTALL.md with updated Maven setup (#11025) 2025-06-16 14:47:20 +02:00
Wei Zhou
ea7919b0c1
Remove test/selenium/ test/src-not-used/ (#11013) 2025-06-16 09:02:35 +02:00
julien-vaz
e83a347ace
Improves StoragePoolAllocators' logs (#10475)
Co-authored-by: Julien Hervot de Mattos Vaz <julien.vaz@scclouds.com.br>
2025-06-14 11:23:41 +02:00
slavkap
28ff19b751
enabled discard option (#10077)
Enable the discard option for virtio-blk and virtio-scsi devices for
volumes on StorPool storage
2025-06-14 11:20:47 +02:00
slavkap
685ee9e78f
StorPool: support for direct download (#9833) 2025-06-14 11:19:37 +02:00
João Jandre
7f13beb36a
Check Qcow2 version before using --bitmaps (#10896)
* check version before using --bitmaps

* use cloudruntimeexception
2025-06-13 20:30:46 +02:00
Daniel Augusto Veronezi Salvador
380884ed50
Fix data being replicated on VM's metadata file in the VR (#10987) 2025-06-13 20:13:09 +02:00
Lucas Martins
675ee2ab39
Add parameter to not create additional users on cloudstack-setup-databases (#9969)
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2025-06-13 13:35:42 +02:00
Abhishek Kumar
28e2411110
engine-schema: fix naming for AlmaLinux (#11011)
In PR #10773, 'AlmaLinux' was incorrectly written as 'Alma Linux' in the guest OS category name and OS classification. This PR corrects the naming to 'AlmaLinux'.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-13 12:09:24 +05:30
Suyang(Dawson) Chen
4a672fd9b3
docs: fix outdated Maven subtitle in INSTALL.md (#11012) 2025-06-12 20:18:56 +02:00
Abhishek Kumar
2d669dbe54
engine-schema: fix duplicate statements in upgrade path (#11001)
Some statements to add new columns to cloud.guest_os_category have been
added twice by mistake as part of #10773. These statements are idempotent and won't affect
upgrade as such but better to cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-12 11:34:43 +05:30
Abhishek Kumar
40b0e7cd63
ui: fix build after forward merge (#11010)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-12 11:33:59 +05:30
slavkap
5c0346ea86
Adding device ID to a StorPool volume (#10587) 2025-06-11 19:39:51 +02:00
Pearl Dsilva
a13728360b Fix issue with forward merge 2025-06-11 18:12:18 +05:30
Pearl Dsilva
875223c5b4 Merge branch '4.20' of https://github.com/apache/cloudstack 2025-06-11 17:11:41 +05:30
Pearl Dsilva
41de0b9dff Merge branch '4.19' of https://github.com/apache/cloudstack into 4.20 2025-06-11 17:10:54 +05:30
Abhishek Kumar
bce17b627d
ui,api,server: template categorization based on os (#10773)
Adds new interface for image selection (template/iso) for an instance in UI.
Old interface can still be used and it can be configured using UI configuration (config.json)

OS categories/Guest OS categories have been improved with ability to create new categories, delete an existing category, and marking a category as featured to allow it to show up in the UI in the image selection interface.

New APIs added:
- addOsCategory
- deleteOsCategory
- updateOsCategory

APIs updated:
- updateOsType
- listTemplates
- listOsCategories

Several improvements in UI especially related to forms - DeloyVM, ReinstallVM, CreateVnfAppliance, AddAutoscaleGroup.

DeployVM form can now be opened from template/ISO details view with query params.

Reorganized (removed and added some) OS categories to the following (in the same order):
```
1. Ubuntu
2. Debian
3. Fedora
4. CentOS
5. Rocky Linux
6. Alma Linux
7. Oracle
8. RedHat
9. SUSE
10. Windows
11. Other
```

Documentation PR: https://github.com/apache/cloudstack-documentation/pull/500

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-10 15:25:51 +05:30
Abhishek Kumar
0b841fefcb
ui: show deploy/create button on right info pane (#10814)
* add deploy button on info card

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* info height

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* hide cancel button

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-10 15:22:16 +05:30
Pearl Dsilva
10fc1f3a1c update version 2025-06-06 18:24:14 +05:30
Pearl Dsilva
0d5a0ea681 update 4.19.4.0-SNAPSHOT version 2025-06-06 18:22:42 +05:30
Pearl Dsilva
379ee07d88 Updating pom.xml version numbers for release 4.19.4.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 18:00:09 +05:30
Pearl Dsilva
c4ba601a63 Merge branch '4.19.3.0-RC20250530T0908' into 4.19 2025-06-06 17:59:05 +05:30
Pearl Dsilva
c86b2f1da2 update 4.20.2.0-SNAPSHOT version 2025-06-06 17:53:39 +05:30
Pearl Dsilva
b5e2c181f9 Updating pom.xml version numbers for release 4.20.2.0-SNAPSHOT
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
2025-06-06 15:38:12 +05:30
Abhishek Kumar
41b4f0afd5
ui: missing changes from #10115 (#10949)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-06-04 20:13:43 +02:00
Manoj Kumar
fa85a75bc8
Log previous and new value of configuration when reset/update API is called (#10769) 2025-06-04 12:06:25 +02:00
Pearl Dsilva
c61a5eb430 Updating pom.xml version numbers for release 4.20.1.0
Signed-off-by: Pearl Dsilva <pearl1594@gmail.com>
4.20.1.0
2025-05-30 12:43:00 +05:30
Daan Hoogland
0c7d47138d Updating pom.xml version numbers for release 4.19.3.0
Signed-off-by: Daan Hoogland <daan@onecht.net>
4.19.3.0
2025-05-30 09:08:58 +02:00
Rohit Yadav
823080cc9c
systemvmtemplate: fix Debian 12.11.0 ISO url (#10927)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2025-05-30 11:29:06 +05:30
Manoj Kumar
7632814cd2
Instance lease: Allow deployment of instances with lease duration and leaseexpiry action (#10560)
* FR-248: Instance lease, WIP commit

* insert lease expiry into db and use that to filter exiring vms, add asyncjobmanager

* Add leaseDuration and leaseExpiryAction in Service offering create flow

* Update listVM cmd to allow listing only leased instances

* Add methods to fetch instances for which lease is expiring in next days

* Changes included:
config key setup and configured for alert email
lease options in create and update vm screen
handle delete protection, edit vm, create vm
validated stop and detroy, delete protection

* Update UI screens for leased properties coming from config and service offering

* use global lock before running scheduler

* Unit tests

* Flow changes done in UI based on discussion

* Include view changes in schema upgrade files and use feature in various UI elements

* Added integration test for vm deployment, UI enhancements for user persona, bug fixes

* validate integration tests, minor ui changes and log messages

* fix build: moving configkey from setup to test itself

* Disable testAlert to unblock build and trim whitespaces in integration tests

* Address review comments

* Minor changes in EditVM screen

* Use ExecutorService instead of Timer and TimerTask

* Additional review comments

* Incorporate following changes:
1. Execute lease action once on the instance
2. Cancel lease on instance when feature is disabled
3. Relevant events when lease gets disabled, cancelled, executed
4. Disable associating lease after deployment
5. UI elements and flow changes
6. Changes based on feedback from demo

* Handle pr review comments

* address review comments

* move instance.lease.enabled config to VMLeaseManager interface

* bug fix in edit instance flow and reject api request for invalid values

* max allowed lease is for 100 years

* log instance ids for expired instance

* Fix config validation for value range and code coverage improvement

* fix lease expiry request failures in async

* dont use forced: true for StopVmCmd

* Update server/src/main/java/org/apache/cloudstack/vm/lease/VMLeaseManager.java

Co-authored-by: Vishesh <vishesh92@gmail.com>

* handle review comments

---------

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
2025-05-28 17:40:09 +05:30
Pearl Dsilva
3c949365be Merge branch '4.19' into 4.20 2025-05-28 16:33:48 +05:30
Pearl Dsilva
dd066e0e0e Merge branch '4.19' of https://github.com/apache/cloudstack into 4.19 2025-05-28 16:33:24 +05:30
Harikrishna Patnala
5be6b7980d Skip verification of privileges on project accounts 2025-05-28 13:02:50 +02:00
Harikrishna Patnala
bef86aff82 Skip verification of privileges on project accounts 2025-05-28 16:13:01 +05:30
Harikrishna Patnala
864327b836 Skip verification of privileges on project accounts 2025-05-28 14:56:20 +05:30
Pearl Dsilva
62363c1f4b remove duplicate definitions 2025-05-27 23:20:38 +05:30
Daan Hoogland
650b5ec3da Merge branch '4.20' 2025-05-27 18:18:39 +02:00
Pearl Dsilva
6bb6fe7b41 Merge branch '4.19' into 4.20 2025-05-27 20:10:58 +05:30
Nicolas Vazquez
d5fbd07b9f Adding privilege checks on user and account operations
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
2025-05-27 20:08:16 +05:30
Abhishek Kumar
19d6b979af cks: create separate service account in project
A separate service account will be created and added in the project, if
not exist already, when a Kubernetes cluster is deployed in a project.
This account will have a role with limited API access.

Cleanup clusters on owner account cleanup, delete service account
if needed

When the owner account of k8s clusters is deleted, while its node VMs
get expunged, the cluster entry in DB remain present. This fixes the
issue by cleaning up all clusters for the account deleted.

Project k8s service account will be deleted on account cleanup or when
there is no active k8s cluster remaining

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-05-27 20:08:12 +05:30
Suresh Kumar Anaparti
9cf9966dfe Keep same/consistent auth time for valid & invalid users 2025-05-27 20:06:56 +05:30
Fabricio Duarte
c3c6d34589 Add access validation to Quota email APIs 2025-05-27 20:03:55 +05:30
nvazquez
64d83ce9d1 Fix access to template/ISO list for domain/resource admins
In Apache CloudStack, while using the listTemplates and listIsos APIs, Domain Admins and Resource Admins can retrieve templates and ISOs outside their intended scope.

Co-authored-by: bernardodemarco <bernardomg2004@gmail.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2025-05-27 20:03:13 +05:30
nvazquez
a0080a04fe Adding privilege checks on user and account operations
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
2025-05-27 16:24:29 +02:00