This feature adds the ability to create a new instance from a VM backup for dummy, NAS and Veeam backup providers. It works even if the original instance used to create the backup was expunged or unmanaged. There are two parts to this functionality:
Saving all configuration details that the VM had at the time of taking the backup. And using them to create an instance from backup.
Enabling a user to expunge/unmanage an instance that has backups.
This PR allows attaching of GPU devices via PCI, mdev or VF to an Instance for KVM.
It allows the operator to discover the GPU devices on the KVM host and create a Compute Offering with GPU support based on the available GPU devices on the host. Once the operator has created the Compute offering, it can be used by users to launch Instances with GPU devices.
The Extensions Framework in Apache CloudStack is designed to provide a flexible and standardised mechanism for integrating external systems and custom workflows into CloudStack’s orchestration process. By defining structured hook points during key operations—such as virtual machine deployment, resource preparation, and lifecycle events—the framework allows administrators and developers to extend CloudStack’s behaviour without modifying its core codebase.
The Netris Plugin introduces Netris as a network service provider in CloudStack to be able to create and manage Virtual Private Clouds (VPCs) in CloudStack, being able to orchestrate the following network functionalities:
- Network segmentation with Netris-VXLAN isolation method
- Routing between "public" IP and network segments with an ACS ROUTED mode offering
- SourceNAT, DNAT, 1:1 NAT between "public" IP and network segments with an ACS NATTED mode offering
- Routing between VPC network segments (tiers in ACS nomenclature)
- Access Lists (ACLs) between VPC tiers and "public" network (TCP, UDP, ICMP) both as global egress rules and "public" IP specific ingress rules.
- ACLs between VPC network tiers (TCP, UDP, ICMP)
- External load balancing – between VPC network tiers and "public" IP
- Internal load balancing – between VPC network tiers
- CloudStack Virtual Router services (DHCP, DNS, UserData, Password Injection, etc…)
* Option to deploy a VM with existing volume/snapshot
* smoke test changes
check if the hypervisor is KVM
check if the primary storage's scope is ZONE wide
* skip all tests if the storage isn't Zone-Wide and the hypervisor isn't KVM
* support StorPool tags
add StorPool tags to a volume created from snapshot or to a volume which
will be attached as a ROOT to a new VM
* Add StorPool tags on the new ROOT volume
* Add the StorPool's tags when volume is created from a snapshot or a
volume is attached as a ROOT to a VM
* Addressed review
CKS Enhancements:
* Ability to specify different compute or service offerings for different types of CKS cluster nodes – worker, master or etcd
* Ability to use CKS ready custom templates for CKS cluster nodes
* Add and Remove external nodes to and from a kubernetes cluster
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Update remove node timeout global setting
* CKS/NSX : Missing variables in worker nodes
* CKS: Fix ISO attach logic
* CKS: Fix ISO attach logic
* address comment
* Fix Port - Node mapping when cluster is scaled in the presence of external node(s)
* CKS: Externalize control and worker node setup wait time and installation attempts
* Fix logger
* Add missing headers and fix end of line on files
* CKS Mark Nodes for Manual Upgrade and Filter Nodes to add to CKS cluster from the same network
* Add support to deploy CKS cluster nodes on hosts dedicated to a domain
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
* Support unstacked ETCD
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Fix CKS cluster scaling and minor UI improvement
* Reuse k8s cluster public IP for etcd nodes and rename etcd nodes
* Fix DNS resolver issue
* Update UDP active monitor to ICMP
* Add hypervisor type to CKS cluster creation to fix CKS cluster creation when External hosts added
* Fix build
* Fix logger
* Modify hypervisor param description in the create CKS cluster API
* CKS delete fails when external nodes are present
* CKS delete fails when external nodes are present
* address comment
* Improve network rules cleanup on failure adding external nodes to CKS cluster
* UI: Fix etcd template was not honoured
* UI: Fix etcd template was not honoured
* Refactor
* CKS: Exclude etcd nodes when calculating port numbers
* Fix network cleanup in case of CKS cluster failure
* Externalize retries and inverval for NSX segment deletion
* Fix CKS scaling when external node(s) present in the cluster
* CKS: Fix port numbers displayed against ETCD nodes
* Add node version details to every node of k8s cluster - as we now support manual upgrade
* Add node version details to every node of k8s cluster - as we now support manual upgrade
* update column name
* CKS: Exclude etcd nodes when calculating port numbers
* update param name
* update param
* UI: Fix CKS cluster creation templates listing for non admins
* CKS: Prevent etcd node start port number to coincide with k8s cluster start port numbers
* CKS: Set default kubernetes cluster node version to the kubernetes cluster version on upgrade
* CKS: Set default kubernetes cluster node version to the kubernetes cluster version on upgrade
* consolidate query
* Fix upgrade logic
---------
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
* Fix CKS cluster version upgrade
* CKS: Fix etcd port numbers being skipped
* Fix CKS cluster with etcd nodes on VPC
* Move schema and upgrade for 4.20
* Fix logger
* Fix after rebasing
* Add support for using different CNI plugins with CKS
* Add support for using different CNI plugins with CKS
* remove unused import
* Add UI support and list cni config API
* necessary UI changes
* add license
* changes to support external cni
* UI changes
* Fix NPE on restarting VPC with additional public IPs
* fix merge conflict
* add asnumber to create k8s svc layer
* support cni framework to use as-numbers
* update code
* condition to ignore undefined jinja template variables
* CKS: Do not pass AS number when network ID is passed
* Fix deletion of Userdata / CNI Configuration in projects
* CKS: Add CNI configuration details to the response and UI
* Explicit events for registering cni configuration
* Add Delete cni configuration API
* Fix CKS deployment when using VPC tiers with custom ACLs
* Fix DNS list on VR
* CKS: Use Network offering of the network passed during CKS cluster creation to get the AS number
* CKS cluster with guest IP
* Fix: Use control node guest IP as join IP for external nodes addition
* Fix DNS resolver issue
* Improve etcd indexing - start from 1
* CKS: Add external node to a CKS cluster deployed with etcd node(s) successfully
* CKS: Add external node to a CKS cluster deployed with etcd node(s) successfully
* simplify logic
* Tweak setup-kube-system script for baremetal external nodes
* Consider cordoned nodes while getting ready nodes
* Fix CKS cluster scale calculations
* Set token TTL to 0 (no expire) for external etcd
* Fix missing quotes
* Fix build
* Revert PR 9133
* Add calico commands for ens35 interface
* Address review comments: plan CKS cluster deployment based on the node type
* Add qemu-guest-agent dependency for kvm based templates
* Add marvin test for CKS clusters with different offerings per node type
* Remove test tag
* Add marvin test and fix update template for cks and since annotations
* Fix marvin test for adding and removing external nodes
* Fix since version on API params
* Address review comments
* Fix unit test
* Address review comments
* UI: Make CKS public templates visible to non-admins on CKS cluster creation
* Fix linter
* Fix merge error
* Fix positional parameters on the create kubernetes ISO script and make the ETCD version optional
* fix etcd port displayed
* Further improvements to CKS (#118)
* Multiple nics support on Ubuntu template
* Multiple nics support on Ubuntu template
* supports allocating IP to the nic when VM is added to another network - no delay
* Add option to select DNS or VR IP as resolver on VPC creation
* Add API param and UI to select option
* Add column on vpc and pass the value on the databags for CsDhcp.py to fix accordingly
* Externalize the CKS Configuration, so that end users can tweak the configuration before deploying the cluster
* Add new directory to c8 packaging for CKS config
* Remove k8s configuration from resources and make it configurable
* Revert "Remove k8s configuration from resources and make it configurable"
This reverts commit d5997033ebe4ba559e6478a64578b894f8e7d3db.
* copy conf to mgmt server and consume them from there
* Remove node from cluster
* Add missing /opt/bin directory requrired by external nodes
* Login to a specific Project view
* add indents
* Fix CKS HA clusters
* Fix build
---------
Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
* Add missing headers
* Fix linter
* Address more review comments
* Fix unit test
* Fix scaling case for the same offering
* Revert "Login to a specific Project view"
This reverts commit 95e37563f48573780b07a038a7f48c0bc04e9b64.
* Revert "Fix CKS HA clusters" (#120)
This reverts commit 8dac16aa359faa6500ea1e1ce548169cfd08331a.
* Apply suggestions from code review about user data
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* Update api/src/main/java/org/apache/cloudstack/api/command/user/userdata/BaseRegisterUserDataCmd.java
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* Refactor column names and schema path
* Fix scaling for non existing previous offering per node type
* Update node offering entry if there was an existing offering but a global service offering has been provided on scale
---------
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
* FR-248: Instance lease, WIP commit
* insert lease expiry into db and use that to filter exiring vms, add asyncjobmanager
* Add leaseDuration and leaseExpiryAction in Service offering create flow
* Update listVM cmd to allow listing only leased instances
* Add methods to fetch instances for which lease is expiring in next days
* Changes included:
config key setup and configured for alert email
lease options in create and update vm screen
handle delete protection, edit vm, create vm
validated stop and detroy, delete protection
* Update UI screens for leased properties coming from config and service offering
* use global lock before running scheduler
* Unit tests
* Flow changes done in UI based on discussion
* Include view changes in schema upgrade files and use feature in various UI elements
* Added integration test for vm deployment, UI enhancements for user persona, bug fixes
* validate integration tests, minor ui changes and log messages
* fix build: moving configkey from setup to test itself
* Disable testAlert to unblock build and trim whitespaces in integration tests
* Address review comments
* Minor changes in EditVM screen
* Use ExecutorService instead of Timer and TimerTask
* Additional review comments
* Incorporate following changes:
1. Execute lease action once on the instance
2. Cancel lease on instance when feature is disabled
3. Relevant events when lease gets disabled, cancelled, executed
4. Disable associating lease after deployment
5. UI elements and flow changes
6. Changes based on feedback from demo
* Handle pr review comments
* address review comments
* move instance.lease.enabled config to VMLeaseManager interface
* bug fix in edit instance flow and reject api request for invalid values
* max allowed lease is for 100 years
* log instance ids for expired instance
* Fix config validation for value range and code coverage improvement
* fix lease expiry request failures in async
* dont use forced: true for StopVmCmd
* Update server/src/main/java/org/apache/cloudstack/vm/lease/VMLeaseManager.java
Co-authored-by: Vishesh <vishesh92@gmail.com>
* handle review comments
---------
Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
Co-authored-by: Vishesh <vishesh92@gmail.com>
* VMware - Ignore disk not found error on cleanup when the VM disk doesn't exists
* VMware - Retry powerOn on lock issues
* addressed comments
* Update CPVM reboot tests - wait for the agent to Disconnect and back Up
* Retry moveDatastoreFile when any file access issue while creating volume from snapshot
* Update full clone flag when restoring vm using root disk offering with more size than the template size
* refactored (mainly,for diskInfo - causing NPE in some cases)
* Retry moveDatastoreFile when there is any file access issue
* Fix unit tests due to change in behavior of restore VM
* update numbering in comments
* revert delete operations
* fix placement of start vm after refactoring
it does not work with python3
```
2025-04-18T10:43:58.5235913Z 2025-04-18 10:32:20,503 - CRITICAL - EXCEPTION: Failure:: ['Traceback (most recent call last):\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 59, in testPartExecutor\n yield\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 591, in run\n self._callTestMethod(testMethod)\n', ' File "/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/unittest/case.py", line 549, in _callTestMethod\n method()\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/failure.py", line 35, in runTest\n raise self.exc_val.with_traceback(self.tb)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/loader.py", line 335, in loadTestsFromName\n module = self.importer.importFromPath(\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 162, in importFromPath\n return self.importFromDir(dir_path, fqname)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 198, in importFromDir\n mod = load_module(part_fqname, fh, filename, desc)\n', ' File "/home/runner/.local/lib/python3.10/site-packages/nose/importer.py", line 128, in load_module\n spec.loader.exec_module(mod)\n', ' File "<frozen importlib._bootstrap_external>", line 883, in exec_module\n', ' File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed\n', ' File "/home/runner/work/cloudstack/cloudstack/test/integration/smoke/test_certauthority_root.py", line 27, in <module>\n from OpenSSL.crypto import FILETYPE_PEM, verify, X509\n', "ImportError: cannot import name 'verify' from 'OpenSSL.crypto' (unknown location)\n"]
```
make tests easier to run and without modifying the test to match the
correct system. Also add tests for volume and vm instance snapshots.
Improve runtime by ~6 minutes.
