mirror of
https://github.com/apache/cloudstack.git
synced 2025-10-26 08:42:29 +01:00
604137624d
* Add Netris module and Add netris provider * Fix * Add Netris Provider to the zone creation wizard * add steps to zone wizard for adding netris controller and public traffic * cleanup * Add missing config key * Add routed mode offerings for Netris VPC (#3) * Add routed mode offerings for Netris VPC * update vpc offering name * generalize the offering creation method for network providers * log * remove debug log * fix failing build * Add dependency and Netris API client (#4) * Add dependency and first approach to Netris API client * Fix authentication and create Netris API client, in progress sites listing * Fix get sites * Support for listing VPCs (#5) * List tenants * Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack (#7) * Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack * Pass default auth scheme for now * Support adding netris provider to CloudStack and Netris VPC Creation (#6) * Support adding netris provider to CloudStack * revert marvin change * add license and perform session check when provider is added * add license and remove unused import * fix build failure - uunused imports * address comments * fix provider name * add Netris network element * add license * Add netris management APIs and netris service provider * add license * revert change * remove other network elements from Netris element * fix api name in doc generator * remove logs * move session alive check to CheckHealthCommand exec * Fix zone creation wizard to configure netris provider * Upgrade GSON version - from PR 8756 * Add additional parametes to the add Netris provider API * add netris as a host * add additional params to the resoponse and update UI * Rename site to site_name * Create Netris VPC (#8) * Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack (#7) * Delegate API classes creation to the SDK and simply invoke the desired API class through CloudStack * Pass default auth scheme for now * Drop for_nsx and for_tungten columns in favour of checking the provider on the ntwserviceprovider map table * Remove missing setForTungsten occurrence * Remove forNsx from VPC offerings * Create Netris VPC * Fix VPC offerings listing and remove unused dao * Create VPC fixes * Upgrade GSON version - from PR 8756 * Fix VPC creation response by using the latest SDK code * Fix unit test * Remove unused import * Fix NSX unit tests after refactoring * Add Netris key to the VLAN Details table (#10) * Add Netris key to the VLAN Details table * update for_<provider> column to be generic * Fix VPC and add IPAM allocation for the VPC CIDR (#9) * Fix VPC and add IPAM allocation for the VPC CIDR * Remove VPC logic * Use zoneId accountId and domainId on resources creation * Fix naming * Fix VR public nic issue * Fix Netris Public IP for VPC source NAT allocation * Add Netris VPC Subnets and vNets (#11) * Add Netris VPC Subnets and vNets * fix compilation errors * Add netris subnet * refactor naming convention to differentiate between VPC tiers and Isolated networks * revert marvin change * fix constructor - build failure * Add support to filter netris offerings, delete netris provider when zone is being deleted * Fix build * Fix VPC creation * Fix vnet creation * unnecesary log --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Fix unit tests * Add support to delete VNets and Subnets (#13) * Add support to delete VNets and Subnets * Add support to delete vnet resources * Add support to delete vnet resources * extract code to method --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Add missing suffix return (#14) * Set up Netris Public range on new zone addition (#15) * Set up Netris Public range on new zone addition * Add dependency to calculate subnet containing a start and end IP * Remove unused import * Move dependency to the netris module * Rename Netris IP range * Refactor logic * Revert "Refactor logic" This reverts commit 7ec36a81320444c37e7bb914dd895060b663411b. * Fix setup range after adding Netris Provider * Fix VXLAN range adding on zone creation * Pass VXLAN ID during creation of Netris vNets (#16) * add zone params to accepts management vnet * Release vxlan associated to the netris broadcast domain type * handle update network broadcast uri * Update Subnet purpose for Netris Public Traffic (#17) * Update Subnet purpose for Netris Public Traffic * search for existing subnet of common purpose type * Fix VR Public IP address (#20) * Fix VR Public IP address * Do not set the Public IP range on Netris side that is not part of the Netris IP Public Pool * Leave only systemvms tag for the first element * Fix NSX compatibility * Pass network gateway instead of network CIDR for Netris vNet creation (#21) * Run moodifyvxlan script if broadcast domain type is Netris (#18) * Add support to create Netris VPC / Network offerings (#22) * Add support to create Netris VPC / Network offerings * fix support services for netris provider type * Phase4 - Add support for Source NAT, Static NAT and Port Forwarding (#19) * Run moodifyvxlan script if broadcast domain type is Netris * Add Netris NAT offerings * Add support to add Source nat rules for Natted offering * fix api params while creating Netris source NAT rule * Add support to add and delete source nat rule on netris * Add support to create /32 NAT subnet * Add support to add and delete Static NAT rules in Netris (#23) * Add support to add and delete Static NAT rules in Netris * fix static nat creation on netris & removal of subnet on deletion of static nat rule * remove nat subnet after deltion of the static nat rule * add check to see if subnet already exists and add license header * Add port forwarding rules as DNAT rules in Netris (#24) * Add port forwarding rules as DNAT rules in Netris * Fixes * Allow removing DNAT rules * Fixes * Fix subnet search * Fix update SNAT only for SNAT rules * Address comments * Fix * Fix netris pom xml * Fix SNAT rule creation * Fix IP and port placements (#27) * Fix IP and port placements * fix dnat to IP for PF rules * change dnatport --------- Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com> * List only Netris Public IPs for NAT operations (#26) * List only Netris Public IPs for NAT operations * rename getter and change type * fix failing unit tests * list all IPs if forProvider is not passed * fix list public IPs for external providers with additional IP range * filter provider Ips in a zone with external provider setup * Prevent acquiring IP that is not from the external provider range * formating --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Support to pass provider when creating public ip range and create IPAM on Netris (#28) * UI: support to pass provider when creating public ip range * prevent adding public ip range for a provider that isnt supported in zone * Create public range on Netris when created on CloudStack --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Revert UI filtration for public IPs (#29) * Fix issue with pagination of public addresses listed after filtering for external providers * Revert UI filteration for public IPs for external network provider enabled zones * Fix unit tests (#30) * Add Netris Tag parameter to the Network provider and fix zone creation wizard (#33) * Add Netris Tag parameter to the Network provider * remove unused import * Fix public IP ranges creation on zone creation (#34) * use single quotes --------- Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com> * Fix SourceAddress for SNAt to VPC cidr (#35) * Fix VPC/network offering service list for external network providers in Routed mode (#32) * Fix network offering service list for external network providers in Routed mode * filter out unsupported services based on network mode * fix supported services list for vpc offering for external providers in Routed mode * Add support to add and delete and update static routes on Netris (#37) * Add support to add static routes in Netris * support to delete static routes on netris * add defensive check for nextHop * Add support to update static routes * add state * pass empty list for switched to avoid timeout * Netris: search static route by name and next hop if exists --------- Co-authored-by: Wei Zhou <weizhou@apache.org> * Netris FR1b: Support Remote Access VPN and Site-to-Site VPN in VPC VR (#41) * Static Routes: support nexthop * Update api/src/main/java/org/apache/cloudstack/api/command/user/vpc/CreateStaticRouteCmd.java Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> * PR#10064 VR: apply iptables rules when add/remove static routes * PR#10065 UI: fix cannot open 'Edit tags' modal for static routes * PR#10066 Static Routes: fix check on wrong global configuration * PR#10067 VR: fix site-2-site VPN if split connections is enabled * PR#10081 server: do not allocate nic on public network for NSX VPC VR * PR#10082 UI: create VPC network offering with conserve mode * PR#10083 VR: allow outgoing traffic from RAS/VPN clients * PR#10086 server: fix typo removeaccessvpn in VirtualRouterElement * server: Add check on Public IP for remote access VPN * Revert "PR#10083 VR: allow outgoing traffic from RAS/VPN clients" This reverts commit 2f9b9f428947cac91de322fbdf4a980902a1c0a0. * VPC: fetch same used IP for domain router if VR is not Source NAT * VR: pass has_public_network to VR and configure RA/S2S VPN left peers * Revert "PR#10081 server: do not allocate nic on public network for NSX VPC VR" This reverts commit 809e269ed6b361d9df1fcef6537762c5612863e0. * VPC: fetch same used IP for domain router if VR is not Source NAT (v2) * VR: fix /etc/hosts and nameservers in dnsmasq.conf if VPC VR is not guest gateway prior to this PR ``` root@r-1167-VM:~# cat /etc/hosts 127.0.0.1 localhost 127.0.1.1 r-1167-VM ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters 172.21.1.33 dummy-vpc-vpn-001 172.21.1.1 r-1167-VM data-server root@r-1167-VM:~# cat /etc/dnsmasq.d/cloud.conf dhcp-hostsfile=/etc/dhcphosts.txt listen-address=127.0.0.1,172.21.1.234 dhcp-range=set:interface-eth1-0,172.21.1.234,static dhcp-option=tag:interface-eth1-0,15,cs2cloud.internal dhcp-option=tag:interface-eth1-0,6,172.21.1.1,10.0.32.1,8.8.8.8 dhcp-option=tag:interface-eth1-0,3,172.21.1.1 dhcp-option=eth1,26,1500 dhcp-option=tag:interface-eth1-0,1,255.255.255.0 ``` the lines should be ``` 172.21.1.234 r-1167-VM data-server dhcp-option=tag:interface-eth1-0,6,10.0.32.1,8.8.8.8 ``` * server: Enable static NAT for Domain router if it is not Source NAT * server: Enable static NAT for Domain router on UI * server: assign Public IP to VPC VR and enable static nat if VR is not Source NAT * server: configure dns1 if VR is not Source NAT * server: remove check on Firewall service when list network service providers * UI: remove dot from message.enabled.vpn * systemvm: add default route via first guest gateway if VR does not have public IP/interface * VR: add fw_dhcpserver for shared network * VR: pass has_public_network to VR and configure RA/S2S VPN left peers (v2) * UI: fix request error when create a VPC tier in a non-Netris/NSX env * systemvm: add default route via first guest gateway (v2) * VR: configure iptables rules for S2S vpn on first guest interface * VR: allow FORWARD to guest interfaces if VR is not Public * VR: configure remote access vpn on first guest interface if not public * VR: fix error 789 in RA VPN client when both RA and S2S are configured * server: Apply Static Route for RA/S2S VPN in VPC VR * VR: do not set mark for Public interface when VR is not really public * VPN: do not disable static nat if it is used by a RA/S2S VPN * server: skip check on network conserve mode if disable/enable RA VPN on Router IP * server: set forRouter to false when release a IP * VR: diable IP spoofing protection on default guest network * VR: fix iptables rules only when only S2S vpn is enabled * UI: show 'VPN Connections' section * VPC: new methods to configure/reconfigure Static NAT for VPC VR * API: set Type in ip address response to DomainRouter if it is used by VR * server: do not allow IP release if it is used by RA or S2S VPN gateway * VR: check if interface is added * VR: add default route only when ip is associated to first guest interface * VR: fix ipsec conf for l2tp and s2s vpn * server: save placeholder IP for VPC VR to fix the new VR IP when vpc tier is auto-shutdown * server: get non-placeholder NIC for VPC VR * VR: wait 15 seconds after starting password server * server: fix unable to configure static nat due to 'invalid virtual machine id' * UI: fix link of router in info card * VPC: apply static route for VPC VPN if needed (refactoring) * server: fix VR IP of first VPC tier is the VM gateway * server: update or remove all existing static routes when shutdown a network * server: update ipaddress after disabling static nat to fix vpc deletion issue * servr: disable remote access VPN as part of VPC dstroy * server: apply static routes when implement a vpc tier * server: apply static routes even if next hop is null * server: fix Cannot invoke "com.cloud.vm.NicProfile.getRequestedIPv4()" because "requested" is null * Netris: Update Vpn provider to VpcVirtualRouter * Netris: Add Vpn service to network offerings and networks * server: fix CIDR of VPN ip range * server: set isVrGuestGateway by SoureNat/Gateway service with Provider.VPCVirtualRouter * VR: password server takes 10-15 seconds to start if VR IP is not configured in /etc/hosts * Netris: add back routesPutBody.setStateStatus * engine/schema: remove SQL changes in schema-41910to42000.sql --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> * Add support for Gateway service for Netris VPC and network offerings (#39) * Add support for Gateway service for Netris VPC and network offerings * Restore UserData service * add gateway only to vpc service * Add support for gateway service for external network providers for networks in routed mode * add support for gateway svc * Revert "add support for gateway svc" This reverts commit 06645cd1c6d08a81ede5d1431497ea3f2efdc5dc. * Fix VPC offering creation * Fix VR public NIC after Gateway service is set to Netris --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> * Netris VPN: Fix s2s vpn status update and isolated network implementation (#42) * server: fix NPE when deploy vm on isolated network * vpn: fix s2s vpn status is not updated Prior to this fix ``` java.lang.IllegalArgumentException: Class com.cloud.agent.api.CheckS2SVpnConnectionsAnswer declares multiple JSON fields named 'details'; conflict is caused by fields com.cloud.agent.api.CheckS2SVpnConnectionsAnswer#details and com.cloud.agent.api.Answer#details at com.cloud.agent.transport.ResponseTest.testCheckS2SVpnConnectionsAnswer(ResponseTest.java:42) ``` * test: fix test_01_vpn_usage as now it is only possible to create VPN on Source NAT if it uses VR * VR: fix unable to create remote access VPN on regular isolated network the error is ``` File "/opt/cloud/bin/configure.py", line 1242, in process self.remoteaccessvpn_iptables(self.dbag['public_interface'], public_ip, self.dbag[public_ip]) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ KeyError: 'public_interface' ``` * Release NAT IP subnet when VPC is removed or IP is released (#44) * Release NAT IP subnet when VPC is removed or IP is released * add license * Add support to add IPv6 Public IP range as IPAM Allocation / Subnet on Netris (#36) * Add support to add IPv6 Public IP range as IPAM Allocation / Subnet on Netris * Add ipam alloc and subnet for the ipv6 subnet associated to the vpc tier network * remove commented code * Phase5 - Support for ACLs in Netris (#31) * Add support for Netris ACLs * acl support * Make acl api call to netris to create the rule * refactor add acl rule to populate the right fields * support icmp type acl rule * acl rule creation - move netrisnetworkRule * Update ACL naming on Netris * Add support for Deletion of netris acls * Add support to delete and re-order ACL rules * support creation of default acl rules and replacing acl rules * fix NSXNetworkRule * Add global routing flag on subnet creation (#45) * Support change snat ip (#46) * Support updating VPC Source NAT IP * Optimize code * Update source NAT IP * Fix naming convention for NAT subnets to follow other resources (#47) * Fix naming convention for NAT subnets to follow other resources * Use vpc ID for nat subnets * Use new nat subnet name for deletion of static nat rule * fix naming convevntion for nat subnet * Keep Vpn service to default VPC offering with Natted mode only (#50) * Add Vpn service to default VPC offering with Routed mode * Revert change on VPC offering and fix VPN service only for Netris NAT mode * Validate if given CIDR belongs to a bigger allocation in Netris before creating the zone-level allocation (#48) * Validate if given CIDR belongs to a bigger allocation in Netris before creating * rename method * Phase5 - Support for LB - create, delete and Update operations (#49) * Add support for Netris ACLs * acl support * Make acl api call to netris to create the rule * refactor add acl rule to populate the right fields * support icmp type acl rule * acl rule creation - move netrisnetworkRule * Update ACL naming on Netris * Add support for Deletion of netris acls * Add support to delete and re-order ACL rules * support creation of default acl rules and replacing acl rules * fix NSXNetworkRule * Fix naming convention for NAT subnets to follow other resources * Use vpc ID for nat subnets * Phase5 - Support for LB - create, delete and Update operations * Use new nat subnet name for deletion of static nat rule * add support to add netris lb rule * support deletion of LB rule on Netris * add checks when editing unsupported fields of LB rule for Netris and hide columns on the UI * fix test failure * fix imports * add license * address comments * Enable Autoscaling on Netris for CPU and memory (#51) * Enable Autoscaling on Netris for CPU and memory * Fix monitor autoscale group and cleanup * Rename autoscaling group method * Integrate Autoscaling by allowing to update LB rules * Refactor according to the SDK changes * Fix the test failures noticed on #44 (#52) * Increase code coverage (#54) * Increase code coverage * More unit tests * Remove credentials and mock api client * NetrisResource tests * Fix unit test * Add support to add and remove ACL rules when CIDR list is passed when creating LB rules (#53) * Add support to add and remove ACL rules when CIDR list is passed when creating LB rules * add deny all rule * delete the deny rule as well * Fix build (#57) * Prevent Index Out of Bounds exception when naming IPAM subnets (#58) * Prevent Index Out of Bounds exception when naming IPAM subnets * fix linter * Delete netris IPv6 subnet (#59) * Netris VPN: add static route when update a non-existent static route (#60) * Fix VPC tier creation failure - prevent creating IPv6 IPAM allocation if it already exists (#61) * Update netris VPC and tier name (#56) * Update netris VPC and tier name * add support to update vpc tier name * add license * support editing names of dual stack VPCs * VR/server: configure default gateway and RA/S2S VPN on the IP/interface with minimum network_id (#43) * server: fix NPE when deploy vm on isolated network * vpn: fix s2s vpn status is not updated Prior to this fix ``` java.lang.IllegalArgumentException: Class com.cloud.agent.api.CheckS2SVpnConnectionsAnswer declares multiple JSON fields named 'details'; conflict is caused by fields com.cloud.agent.api.CheckS2SVpnConnectionsAnswer#details and com.cloud.agent.api.Answer#details at com.cloud.agent.transport.ResponseTest.testCheckS2SVpnConnectionsAnswer(ResponseTest.java:42) ``` * test: fix test_01_vpn_usage as now it is only possible to create VPN on Source NAT if it uses VR * VR: fix unable to create remote access VPN on regular isolated network the error is ``` File "/opt/cloud/bin/configure.py", line 1242, in process self.remoteaccessvpn_iptables(self.dbag['public_interface'], public_ip, self.dbag[public_ip]) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ KeyError: 'public_interface' ``` * VR/server: configure default gateway and RA/S2S VPN on the IP/interface with minimum network_id * Don't add deny rule if no CIDR list is passed (#62) * Hide the Stickiness Configure button for Netris Load Balancers (#72) * Update IPAM subnet purpose to nat before NAT operations if its different (#71) * Netris VPN: create vpc gateway with specified IP (#63) * Netris: fix UnsupportedOperationException when create VPC offering with NATTED mode (#75) fixes ``` 2025-03-21T10:42:55,039 ERROR [c.c.a.ApiServer] (qtp1513608173-21:[ctx-f9c7f002, ctx-bcfe846d]) (logid:e12e798f) unhandled exception executing api command: [Ljava.lang.String;@3a1416cd java.lang.UnsupportedOperationException at java.base/java.util.AbstractList.add(AbstractList.java:153) at java.base/java.util.AbstractList.add(AbstractList.java:111) at org.apache.cloudstack.api.command.admin.vpc.CreateVPCOfferingCmd.getServiceProviderMapForExternalProvider(CreateVPCOfferingCmd.java:248) ``` * [UI] Zone wizard creation improvements - rename hostname to url and remove port for Netris Provider (#77) * [UI] Zone wizard creation improvements - rename hostname to url and remove port for Netris Provider * Fix schema column for url instead of hostname * Fix Static NAT rules naming (#83) * Netris: create VPN gateway with specified public IP on UI (#82) * Netris vpn: apply static routes when start or delete a VPN connection (#85) * Netris VPN: apply static routes when start S2S VPN * Netris: list static routes and revoke the routes which are not needed * Netris: use route name (x.x.x.0/x) instead of prefix (x.x.x.0) and get clean cidr list * Netris VPN: fix NPE when list static routes * Update plugins/network-elements/netris/src/main/java/org/apache/cloudstack/service/NetrisApiClientImpl.java * Delete IPv6 allocation after tier removal on VPC with dual stack offering (#86) * Netris pass v6 gateway (#87) * pass v6 gateway to netris * pass v6 gateway to netris * refactor to address comments * remove imports * [VR] Fix IPv6 NIC IP on the VR (#89) * Add support to edit ACL rules (#74) * Add support to edit ACL rules * add support to update acl rules * remove test file * VR: advertise SLAAC prefix only if VR is gateway (#91) * Make reorder ACL items invoke Netris controller (#90) * VR: fix radvd misconfiguration for non-netris env (#92) * [VR] Fix object comparisson to string comparisson on python (#93) * Fix unit tests for ACL (#94) * Use the previously assigned vNet for Netris Network when it transitions to Implemented state after gc (#88) * Use the previously assigned vNet for Netris Network when it transitions to Implemented state after gc * Fix unit tests --------- Co-authored-by: nvazquez <nicovazquez90@gmail.com> --------- Co-authored-by: Pearl Dsilva <pearl1594@gmail.com> Co-authored-by: Wei Zhou <weizhou@apache.org>