4366 Commits

Author SHA1 Message Date
Pearl Dsilva
b288e935dc Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-12-07 16:47:44 -05:00
Pearl Dsilva
389dbe895a
NSX: Add CKS Support & Firewall rules for Isolated Networks (#8189)
* NSX: Add ALL LB IP to the list of route advertisements in tier1

* NSX: Support Source NAT on NSX Isolated networks

* NSX: Cks Support

* NSX: Create segment group on segment creation

* Add unit tests

* Remove group for segment before removing segment

* Create Distributed Firewall rules

* Remove distributed firewall policy on segment deletion

* Fix policy rule ID and add more unit tests

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add support for routed NSX Isolated networks \n and non RFC 1918 compliant IPs

* Add Firewall rules

* build failure - fix unit test

* fix npes

* Add support to delete firewall rules

* update nsx cks offering

* add license

* update order of ports in PF & FW rules

* fix filter for getting transport zones

* CKS support changed - MTU updated, etc

* add LB for CKS on VPC

* address comments

* adapt upstream cks logic for vpc

* rever mtu hack

* update UI changes as per upstream fix

* change display test for CKS n/w offerings for isolated and VPC tiers

* add extra line for linter

* address comment

* revert list change

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2023-12-07 16:43:56 -03:00
Nicolas Vazquez
371ad9f55b
New Feature: Import VMware VMs into KVM (#7881)
This PR adds the capability in CloudStack to convert VMware Instances disk(s) to KVM using virt-v2v and import them as CloudStack instances. It enables CloudStack operators to import VMware instances from vSphere into a KVM cluster managed by CloudStack. vSphere/VMware setup might be managed by CloudStack or be a standalone setup.

    CloudStack will let the administrator select a VM from an existing VMware vCenter in the CloudStack environment or external vCenter requesting vCenter IP, Datacenter name and credentials.
    The migrated VM will be imported as a KVM instance
    The migration is done through virt-v2v: https://access.redhat.com/articles/1351473, https://www.ovirt.org/develop/release-management/features/virt/virt-v2v-integration.html
    The migration process timeout can be set by the setting convert.instance.process.timeout
    Before attempting the virt-v2v migration, CloudStack will create a clone of the source VM on VMware. The clone VM will be removed after the registration process finishes.
    CloudStack will delegate the migration action to a KVM host and the host will attempt to migrate the VM invoking virt-v2v. In case the guest OS is not supported then CloudStack will handle the error operation as a failure
    The migration process using virt-v2v may not be a fast process
    CloudStack will not perform any check about the guest OS compatibility for the virt-v2v library as indicated on: https://access.redhat.com/articles/1351473.
2023-12-07 12:59:56 +05:30
nvazquez
edd66c31a3
Merge branch 'main' into nsx-integration 2023-12-05 08:14:00 -03:00
Daan Hoogland
d3cad4266a Merge release branch 4.18 to main
* 4.18:
  server: Initial new vpnuser state (#8268)
  UI: Removed redundant IP Address Column when create Port forwarding rules (#8275)
  UI: Removed ICMP input fields for protocol number from ACL List rules modal (#8253)
  server: check if there are active nics before network GC (#8204)
2023-12-04 10:44:43 +01:00
Bryan Lima
b0910fc61d
Add dynamic secondary storage selection (#7659) 2023-12-04 09:52:32 +01:00
Abhishek Kumar
108651ad40
api: make displaytext form upload template/iso optional (#8289)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-12-04 08:48:49 +01:00
nvazquez
cb2e2da0b9
Merge branch 'main' into nsx-integration 2023-12-03 15:37:36 -03:00
kishankavala
5651eab49c
ObjectStore Framework with MinIO and Simulator plugins (#7752)
This PR adds Object Storage feature to CloudStack.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+CloudStack+Object+Store
2023-12-01 17:51:00 +05:30
dahn
724394682c
server: Initial new vpnuser state (#8268) 2023-12-01 09:51:54 +01:00
nvazquez
33b494aebf
Merge branch 'main' into nsx-integration 2023-11-30 22:58:43 -03:00
Bryan Lima
cb62ce6767
Global ACL for VPCs (#7150) 2023-11-30 14:51:43 +01:00
João Jandre
26b01f6f3b
Flexible tags for hosts and storage pools (#7489)
Co-authored-by: João Jandre <joao@scclouds.com.br>
2023-11-30 09:36:47 +01:00
Vishesh
a31f211628
Merge remote-tracking branch 'remote/4.18' 2023-11-29 16:12:51 +05:30
Rodrigo D. Lopez
956efb27d9
allow filtering of listDiskOffering and listServiceOffering APIs by account or project (#7082) 2023-11-28 14:50:37 +01:00
Abhisar Sinha
5c7e4b7edc
api: add ipaddress argument to disassociateIPAddress (#8222)
This PR adds argument 'ipadress' to the disassociateIpAddress api. IP address can be disassociated by directly giving the address instead of ID.

Fixes: #8125
2023-11-19 11:50:57 +05:30
Daan Hoogland
98d643efe6 Merge release branch 4.18 to main
* 4.18:
  Fixed spelling and added missing states to response (#8248)
  Let Prometheus exporter plugin support utf8 characters (#8228)
2023-11-18 18:41:31 +01:00
Oleg Chuev
1b56a8e873
Fixed spelling and added missing states to response (#8248) 2023-11-18 18:36:49 +01:00
Wei Zhou
eaa4123068
VNF: fix UI bugs and polish (#8227) 2023-11-18 12:25:18 +01:00
DK101010
6001772335
multi local storage handling for kvm (#6699)
Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2023-11-16 16:43:42 +01:00
Manoj Kumar
0735b91037
api: introduce domainid and account parameter in createTemplate API (#8210)
Introduces domainid and account as optional parameter for createTemplate API. It will allow admin to create templates for specified domain belonging to specific account.
2023-11-16 17:39:38 +05:30
Bryan Lima
1f29f6f040
Public IP quarantine feature (#7378) 2023-11-15 10:29:22 +01:00
slavkap
04061f12e5
storagetype API param in list service/disk offerings (#8215) 2023-11-14 14:03:24 +01:00
Daan Hoogland
05b9b6e2e7 Merge branch '4.18' into main 2023-11-13 11:36:51 +01:00
nvazquez
053521077c
Merge branch 'main' into nsx-integration 2023-11-10 13:50:26 -03:00
Abhishek Kumar
d0f3233fda
edge-zone,kvm,iso,cks: allow k8s deployment with direct-download iso (#8142)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-11-10 13:56:05 +01:00
Daan Hoogland
e7bb89c4b2 Merge release branch 4.18 to main
* 4.18:
  updated the api documentation for createNetworkOffering and updateNetworkOffering (#7545)
2023-11-09 18:03:14 +01:00
kiranchavala
86444809c3
updated the api documentation for createNetworkOffering and updateNetworkOffering (#7545)
Co-authored-by: Kiran Chavala <kiranchavala@Kirans-MacBook-Air.local>
Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-11-09 18:01:57 +01:00
nvazquez
ab994dbfcf
Merge branch 'main' into nsx-integration 2023-11-06 08:15:33 -03:00
Nicolas Vazquez
ecd32d5632
[NSX] Fix update source NAT IP (#8176)
* [NSX] Fix update source NAT IP

* Fix startup

* Fix API result
2023-11-06 08:15:00 -03:00
slavkap
2bb182c3e1
KVM Host HA enhancement for StorPool storage (#8045)
Extending the current functionality of KVM Host HA for the StorPool storage plugin and the option for easy integration for the rest of the storage plugins to support Host HA

This extension works like the current NFS storage implementation. It allows it to be used simultaneously with NFS and StorPool storage or only with StorPool primary storage.

If it is used with different primary storages like NFS and StorPool, and one of the health checks fails for storage, there is an option to report the failure to the management with the global config kvm.ha.fence.on.storage.heartbeat.failure. By default this option is disabled when enabled the Host HA service will continue with the checks on the host and eventually will fence the host
2023-11-04 12:35:37 +05:30
nvazquez
42300aa80e
Merge branch 'main' into nsx-integration 2023-11-03 12:08:39 -03:00
Pierre-Luc Dion
a3565fc8db
Update GPU support for V100,T4,A5500. (#6639) 2023-11-02 10:46:10 +01:00
John Bampton
f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
sato03
9c4d18f9b1
role ispublic param on UI (#7973)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-11-02 09:22:18 +01:00
Pearl Dsilva
548f0408c5 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-11-01 15:23:15 -04:00
Pearl Dsilva
a36f355877
NSX: Router Public nic to get IP from systemVM Ip range (#8172)
* NSX: Router Public nic to get IP from systemVM Ip range

* Fix VR IP address and setSourceNatIp command

* NSX: hide systemVM reserved IP range SourceNAT

* fix test

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
2023-11-01 16:14:33 -03:00
Vishesh
5362bad442
Storage Management (#7949) 2023-11-01 10:46:22 +01:00
Pearl Dsilva
dc90d0b9dd
NSX: fix LB member addition and deletion and add defensive checks (#8167) 2023-10-31 21:35:02 -03:00
Pearl Dsilva
20cb9f56f3 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-10-31 06:10:52 -04:00
Harikrishna
235e4fe190
Oauth2 integration with CloudStack (#7996)
OAuth2, the industry-standard authorization or authentication framework, simplifies the process of
granting access to resources. CloudStack supports OAuth2 authentication wherein users can login into
CloudStack without using a username and password. Support for Google and Github providers has been added.
Other OAuth2 providers can be easily integrated with CloudStack using its plugin framework.

The login page will show provider options when the OAuth2 is enabled and corresponding providers are configured.

"OAuth configuration" sub-section is present under "Configuration" where admins can register the corresponding
OAuth providers.
2023-10-31 13:25:28 +05:30
Nicolas Vazquez
a4be3eb380
[NSX] Add SNAT support (#8100)
* In progress add source NAT

* Fix after merge

* Fix tests

* Fix NPE on isolated network deletion

* Reserve source NAT IP when its not passed for NSX VPC

* Create source NAT rule on VR NIC allocation

* Fix update VPC and remove VPC to update and remove SNAT rule

* Fix packaging

* Address review comment

* Fix build

* fix build - unused import

* Add defensive checks

* Add missing design to NSX public guru

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2023-10-31 00:54:27 -03:00
Pearl Dsilva
c6d6463e20 Merge branch 'main' of https://github.com/apache/cloudstack into nsx-integration 2023-10-30 20:14:21 -04:00
Pearl Dsilva
ce1659e8fc
NSX: Create and Delete static NAT & Port forward rules (#8131)
* NSX: Create and delete NSX Static Nat rules

* fix issues with static nat

* add static nat

* Support to add and delete Port forward rules

* add license

* fix adding multiple pf rules

* cleanup

* fix lint check

* fix smoke tests

* fix smoke tests
2023-10-30 19:37:12 -04:00
Wei Zhou
bd52fa8a12
New feature: VNF templates and appliances integration (#8022) 2023-10-27 10:23:00 +02:00
Daan Hoogland
587d1d7dba Merge remote-tracking branch 'apache/4.18' into main 2023-10-26 09:37:38 +02:00
slavkap
6ae3b73ca2
Create snapshot from VM snapshot without memory for NFS/Local storage (#8117) 2023-10-26 08:46:14 +02:00
Vishesh
ea90848429
Feature: Add support for DRS in a Cluster (#7723)
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.

Implementation
There are three top level components:

    Scheduler
    A timer task which:

    Generate DRS plan for clusters
    Process DRS plan
    Remove old DRS plan records

    DRS Execution
    We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.

    Algorithm
    Every algorithms implements two methods:
        needsDrs - to check if drs is required for cluster
        getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.

Algorithms

    Condensed - Packs all the VMs on minimum number of hosts in the cluster.
    Balanced - Distributes the VMs evenly across hosts in the cluster.
    Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.

APIs Added

listClusterDrsPlan

    id - ID of the DRS plan to list
    clusterid - to list plans for a cluster id

generateClusterDrsPlan

    id - cluster id
    iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.

executeClusterDrsPlan

    id - ID of the cluster for which DRS plan is to be executed.
    migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.

Config Keys Added

    ClusterDrsPlanExpireInterval
    Key drs.plan.expire.interval
    Scope Global
    Default Value 30 days
    Description The interval in days after which old DRS records will be cleaned up.

    ClusterDrsEnabled
    Key drs.automatic.enable
    Scope Cluster
    Default Value false
    Description Enable/disable automatic DRS on a cluster.

    ClusterDrsInterval
    Key drs.automatic.interval
    Scope Cluster
    Default Value 60 minutes
    Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.

    ClusterDrsIterations
    Key drs.max.migrations
    Scope Cluster
    Default Value 50
    Description Maximum number of live migrations in a DRS execution.

    ClusterDrsAlgorithm
    Key drs.algorithm
    Scope Cluster
    Default Value condensed
    Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.

    ClusterDrsLevel
    Key drs.imbalance
    Scope Cluster
    Default Value 0.5
    Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
    is allowed and 0.0 means imbalance is allowed.

    ClusterDrsMetric
    Key drs.imbalance.metric
    Scope Cluster
    Default Value memory
    Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
2023-10-26 11:48:18 +05:30
Pearl Dsilva
714a0a22d4
NSX: Add Step for Adding Public traffic network for NSX During zone creation (#8126)
* NSX: Add Step for Adding Public traffic network for NSX

* address comments and cleanup

* address comment

* remove indent
2023-10-25 11:09:05 -03:00
Aakash Sinha
8fe13f91a1
OSType response: isuserdefined as string not bool #8127 (#8128)
Co-authored-by: Aakash Sinha <mailtomeaakash@gmail.com>
2023-10-25 11:02:57 +02:00