153 Commits

Author SHA1 Message Date
Lucas Martins
0a92cc0349
UI: Add VM state to Volume list view (#10341)
* Add vm state to volume list view

* Change size column order

---------

Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2025-02-12 11:50:58 -03:00
Abhishek Kumar
7abda3b963 Merge remote-tracking branch 'apache/4.20' 2025-01-31 16:55:38 +05:30
Abhishek Kumar
ae2ffbe40b Merge remote-tracking branch 'apache/4.19' into 4.20 2025-01-31 16:54:22 +05:30
Abhishek Kumar
97be6f2e5d
ui: fix column filter for templates, isos (#10288)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2025-01-31 10:23:26 +01:00
Suresh Kumar Anaparti
3b108b968f
Support for Management Server Maintenance Mode (#9854)
* Support for Management Server Maintenance

- New APIs: prepareForMaintenance and cancelMaintenance, with required parameter - managementserverid.

- New management server states for maintenance: PreparingForMaintenance, Maintenance.

- listHosts API with optional parameter – managementserverid, to list the hosts connected to the management server.

- Support management server maintenance when more than one active management servers available.

- Triggers transfer agents to other available management servers for maintenance, new agent command MigrateAgentConnectionCommand to initiate transfer of indirect agents.

- New global config 'management.server.maintenance.timeout', to set the timeout (in mins) for the management server maintenance window, default: 60 mins.

- UI changes: Prepare and Cancel Maintenance in Management Server section, Connected Agents tab, New fields for hosts and management servers.

* Updated pending jobs check timer task with ScheduledExecutorService

* keep maintenance state on trigger shutdown call when ms is in maintenance

* add pending jobs count to ms response

* during ms heartbeat, update state to up only when it's down

* allow vm work jobs of async job created before prepare for maintenance

* Revert "keep maintenance state on trigger shutdown call when ms is in maintenance"

This reverts commit 607e13364679eac897f4d146bb3325ea7a61ba17.

* skip maintenance test when multiple management servers are not available, and not configured in host setting for kvm
2025-01-29 13:31:15 +05:30
Daan Hoogland
98f5663954 Merge branch '4.20' 2025-01-24 17:10:43 +01:00
Daan Hoogland
34d2a3bc86 Merge branch '4.19' into 4.20 2025-01-24 17:01:42 +01:00
Harikrishna
4d9fd1b73f
Added displaynetwork option in filters for listnetwork only for admin (#10209) 2025-01-23 09:36:30 +01:00
Wei Zhou
91f1adab86
UI: set redundant state as N/A for non-redundant routers (#10227) 2025-01-21 14:29:37 +01:00
Daan Hoogland
9295a1624d Merge release branch 4.20 to main
* 4.20:
  VR: apply iptables rules when add/remove static routes (#10064)
  Certificate and VM hostname validation improvements (#10051)
  set ulimit for server according to redhat spec (#10040)
  kvm-storage: provide isVMMigrate information to storage plugins (#10093)
  Allow config drive deletion of migrated VM, on host maintenance (#10045)
  linstor: improve heartbeat check with also asking linstor (#10105)
  server: simplify role change validation (#9173)
  UI: create VPC network offering with conserve mode (#10082)
  server: fix typo removeaccessvpn in VirtualRouterElement (#10086)
  UI: remove duplicated Instance Name in Public IP details page (#10087)
  UI: Fixes in the Usage UI (#10000)
  SAML2: add cookie with HttpOnly too #10013 (#10047)
  ui: Allow font-awesome icon usage and optimise icon size inconsistency (#9744)
2024-12-20 14:37:49 +01:00
Wei Zhou
14460f49e1
Merge remote-tracking branch 'apache/4.19' into 4.20 2024-12-12 11:55:39 +01:00
Bernardo De Marco Gonçalves
e57a82a4f5
UI: Fixes in the Usage UI (#10000)
* fixes on usage ui

* dont call API when caller does not have access to it

* handle when lastheartbeat is undefined

* hide last collection datetime when lastheartbeat is undefined

* apply Pearl's suggestions

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* apply Pearl's suggestions

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* apply Pearl's suggestions

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* apply Pearl's suggestions

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* apply Pearl's suggestions

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2024-12-09 14:25:03 -03:00
Rohit Yadav
4876761154
ui: Allow font-awesome icon usage and optimise icon size inconsistency (#9744)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan@onecht.net>
2024-12-09 10:03:37 +01:00
Layon
528fca2f17
Added-cpu-core-and-memory-range-fot-type-offers-CustomConstrained (#9930)
* Added-cpu-core-and-memory-range-fot-type-offers-CustomConstrained

* Update ui/src/components/view/ListView.vue

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>

* Update ui/src/components/view/ListView.vue

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>

---------

Co-authored-by: layon.oliveira <layon.oliveira@scclouds.com.br>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
2024-11-28 14:42:55 -03:00
Abhishek Kumar
82f113b1e1
ui: refactor config update/reset notification (#9639)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-09-06 13:43:01 +02:00
Rohit Yadav
f683d2dd24 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-06 10:59:36 +05:30
Wei Zhou
679ce1a639
feature: Dynamic and Static Routing (#9470)
This PR contains 3 features

- IPv4 Static Routing (Routed mode) #9346
Design document: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=306153967

- AS Numbers Management #9410
Design Document: https://cwiki.apache.org/confluence/display/CLOUDSTACK/BGP+AS+Numbers+Management


- Dynamic routing
Design Document: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=315492858

- Document: https://github.com/apache/cloudstack-documentation/pull/419

Rename nsx mode to routing mode

by
```
git grep -l nsx_mode  |xargs sed -i "s/nsx_mode/routing_mode/g"
git grep -l nsxmode  |xargs sed -i "s/nsxmode/routingmode/g"
git grep -l nsxMode  |xargs sed -i "s/nsxMode/routingMode/g"
git grep -l NsxMode  |xargs sed -i "s/NsxMode/RoutingMode/g"
```
- re-organize sql changes
- fix NPE as rules do not have public ip
- fix missing destination cidr in ingress rules
- disable network usage for routed network
- fix DB exception as network_id is -1 during network creation
- apply ingress/egress routing rules
- VR changes to configure nft rules for isolated network
- VR: setup nft rule for control network
- VR: flush all iptables rules
- fix NPE which is because ingress rules do not have public ip associated
- fix dest cidr is missing in nft tables
- add ip4 routing and ip4 routes to list network and list vpc response
- fix ingress rule is missing when vr is restarted
- fix icmp types in nft rules
- add tab to manage routing firewall rules
- fix ingress rules are not applied when VR is restarted
- add default rules in FORWARD chain
- fix create vpc offerings
- fix public ip is not assigned to vpc
- fix network offering is not listed when create vpc tier
- add is_routing to boot args of vpc vr
- remove table ip4_firewall in vpc vr
- release or remove subnet when remove a network
- implemenent fw_vpcrouter_routing
- fix wrong ip familty when flush ipv4 rules
- fix acl rules are not applied due to wrong version (should be 6 which means ip6 rules are removed)
- add default rules for vpc tiers so that tcp connections (e.g. ssh) work
- append policy rules after default rules
- remove /usr/local/cloud/systemvm/ in routers
- throw an exception when allocate subnet with cidrsize
- fix some TODOs
- add new parameters to update API
- return type Ipv4GuestSubnetNetworkMap when get or create subnet
- fix firewall rules are broken
- add domain_id and account_id to db
- add domain/account/project to ipv4 subnet response
- create ipv4 subnet for domain/account/project
- check conflict when update ipv4 subnet
- ui changes
- add parent subnet to response
- add list for ipv4 subnet
- implement some methods
- fix list subnets for guest networks by zoneid
- UI changes
- fix delete ipv4 subnet for network
- fix ipv4 subnet is set to zone guest network cidr if cidrsize is specified
- add zone info to response if parent subnet is null but network is not
- fix gateway/cidr is not set when create network with cidrsize
- fix order of nft rules in the VRs

* Routed v24

- add classes in marvin base.py

* Routed v25

- add test_01_subnet_zone
- fix dedicate to domain/account failure
- list subnets for network by keyword and subnet

* Routed v26: implement subnet auto-allocation

- add utils for split ip ranges into small subnets
- add utils to get start/end ip of a cidr
- implement subnet auto-generation
- add global settings

* Routed 27: add subnet for VPC

- add db column for vpc_id
- add db record for vpc
- remove db record when delete a vpc
- add checkConflicts methods
- remove duplicated settings
- check ipv4 cidr when create subnet

* Routed v28: update smoke tests

- update test_ipv4_routing.py
- search subnets by networkid

* Routed 29: fix vpc and add more tests

- fix createnetwork in vpc
- add vpc id/name to response
- fix zone id/name are not displayed in some cases
- add smoke test for vpc
- add smoke tests for failed cases
- add smoke test for connectivity checks
- marvin: add "-q" to ssh command

* Routed 31: ui and smoke tests

- UI: add link to network in list view
- add nftables rules check in VRs

* Routed 32: add chain OUTPUT and more rules

- fix the issue 80/443/8080 is not reachable from VR itself

```
2024-06-27 10:21:52,121 INFO     Executing: systemctl start cloud-password-server@172.31.1.1
2024-06-27 10:21:52,128 INFO     Service cloud-password-server@172.31.1.1 start
2024-06-27 10:21:52,129 INFO     Executing: ps aux
2024-06-27 10:24:02,175 ERROR    Failed to update password server due to: <urlopen error [Errno 110] Connection timed out>
```

* Routed: fix dns search from VMs in Isolated networks

* Routed: fix VPC dns issue due to gateway IP is missing in cloud.conf

This is caused by NSX integration, and fixed by
https://github.com/apache/cloudstack/pull/9102/

* Routed: rename routing_mode to network_mode

* Routed: replace centos5.5 template in smoke test as dhclient does not work in the vms

// this does not work
refer to https://dominikrys.com/posts/disable-udp-checksum-validation/#ignoring-udp-checksums-with-nftables
and
https://forum.openwrt.org/t/udp-checksum-with-nftables/161522/11

the vm should have checksum offloading disabled

* Routed: fix smoke test due to wrong cidrlist of egress rules and missing ingress rule from VR

* PR 9346: fix lint error schema-41910to42000.sql

* PR 9346: ui polish v1

* PR 9346: create VPC with cidrsize

* Routed: fix test failures with test_network_ipv6 and test_vpc_ipv6 due to 'ssh -q'

* Routed: fix /usr/local/cloud/systemvm/ are removed after SSVM/CPVM reboot

* Routed: fix IP of additional nics of VPC VR is not gateway

* PR 9346: fix cidrsize check when create VPC with cidrsize

* Routed: fix test/integration/smoke/test_ipv4_routing.py:279:16: E713 test for membership should be 'not in'

* PR9346: fix/Update api

* PR 9346: set response object name

* PR9346: UI refactor and small fixes

* PR9346: change return type of getNetworkMode

* PR9346: move IPv4 subnet to seperated tab

* PR9346: revert IpRangesTabGuest.vue back to original

* PR9346: fix remove ipv4 subnet on UI

* PR9346: fix test_ipv4_routing.py

* AS Number Range Management

* Create AS Number Range for a Zone

* Fix build

* Add ListASNRange and fix create ASN range

* Add List AS numbers

* Add UI for AS Numbers

* Fix UI and filter AS Numbers

* Add AS Number on Isolated network creation and refactor UI and response

* Release AS Number

* Add network offering new columns

* Add UI support to view and add AS number and configure network offering

* Automatically assign AS Number if not specify AS number

* update variable name

* Fix routing mode check

* UI: Only allow selecting AS number when routing mode is Dynamic and specifyAsNumber is true

* UI: Only pass AS number when supported by the network offering

* Release AS number on network deletion

* Add deleteASNRange command (#81)

* API: List ASNumbers by asnumber (#83)

---------

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* AS number management extensions

* Support AS number on VPC tier creation based on the offering

* Fix delete AS Range

* Fix UI values

* UI: Minor fix for releasing AS number

* UI: Move management of AS Range to Zone details view

* Fix specify_as_number column in network_offering table to set the default false

* Add events for AS number operations

* Allow users to list AS Numbers and fix network form for Normal users

* Add AS number details to list networks response

* Fix Allocated time format

* Fix Allocated time format

* support in details view too

* Fix: Do not release AS number if acquired network requires AS number

* Fix: Do not release AS number if acquired network requires AS number

* Fix typo

* Fix allocated release

* Fix event type

* UI: Add Routing mode and Specify AS to the network offering details

* UI: Add Routing mode and Specify AS to the network offering details

* Address comment

* Fix release AS number of network deletion

* Fix release AS number of network deletion

* Fix

* Restore release to its place based on the boolean

* Rename boolean

* API: Add networkId as listASNumber parameter

* Add Network name to the search view filter for AS numbers

* Present allocated time in human readable format - Pubilc IP / AS Numbers

* Add account / domain filter for AS numbers

* Add support for AS numbers on VPC offerings

* Refactor AS number allocation to VPC and non VPC isolated networks

* Checkstyle

* Add support for AS numbers on VPC offerings

* extend vpc offering view and vpcoffering response

* merge https://github.com/shapeblue/cloudstack-playtika/pull/115 and change network_id of as_numbers to include vpc_id

* Display AS number of VPC tiers as the AS number of the VPC

* extend asnumber response and ui support

* improve UI and as number response to view VPC details

* List only dynamic offerings for vpc tiers with specify as numbers

* Fix release AS number

* Fix AS number displayed as 0 when no AS number assigned

* Fix VPC offering creation without specify AS

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>

* Fix release AS number on VPC deletion

* Update server/src/main/java/com/cloud/dc/BGPServiceImpl.java

* Update server/src/main/java/com/cloud/dc/BGPServiceImpl.java

* Fix missing column on asnumber table

* Fix listASNumbers API to support vpcid and obtain AS number from vpc for tiers

* Prevent listing 0 AS number for VPC

* Fix create Isolated Network form

* Update server/src/main/java/com/cloud/network/vpc/VpcManagerImpl.java

* Update server/src/main/java/com/cloud/network/vpc/VpcManagerImpl.java

* Dynamic: move routingmode/specifyasn after networkmode in AddNetworkOffering.vue on UI

* Dynamic: fix ip4routing in network response

* Dynamic/systemvm: add FRR to systemvm template

* Dynamic: BGP peers (DB,VO,Dao)

* Dynamic: BGP peers (VR/server)

* Dynamic: v3

- remove BgpPeer class
- fix vpc vr has bgp peers of only 1 tier
- rename ip4_cidr to guest_ip4_cidr
- rename ip6_cidr to guest_ip6_cidr
- generate /etc/frr/frr.conf
- apply BGP peers on Dynamic-Routed network even if there is no BGP peers

* Dynamic v4: fix vpc vr

- fix duplicated guest cidr in frr.conf in vpc vr

todo
- restart frr / reload frr (reload will cause bgp session to Policy state)
- apis for bgp peers
- assign/release bgp peer from/to network

* Dynamic v5: add apis for bgp peers

* Dynamic v6: fix bugs

- set response object name
- remove required as number when update
- fix checks when update
- allow regular users to list bgp peers

* Dynamic v7: move apis to bgp sub-dir

* Dynamic v8: add tab for manage BGP peers on UI

* Dynamic v9: fix update bgp with same config

* Dynamiv v10: add changeBgpPeersForNetworkCmd

* Dynamic v11: create network with bgppeerids

- create network with bgppeerids
- add marvin classes
- add smoke tests
- remove uuid from bgp_peer_network_map
- fix created/removed in bgp_peer_network_map
- remove bgppeers when remove a network
- UI: fix delete bgp peer

* Dynamic v12: add test for vpc tiers

* Dynamic v13: bug fixes

- fix change BGP peers for network in Allocated state
- fix listing network returns removed record
- fix all vpc tiers have the same settings
- remove BGP peers as part of network removal
- remove FRR settings for vpc tiers without any BGP peers
- UI: fix no error msg when change BGP peers

* Dynamic v14: assign BGP Peers for VPC instead of VPC tiers

- create vpc with bgppeerids
- do not allow create/update vpc tier with bgppeerids
- apply all bgp peers when create/delete a vpc tier
- UI: change bgp peers for vpc
- test: update tests on vpc

* Dynamic: fix build errors after merging as number PR

* Dynamic: fix TODOs

* Dynamic: fix smoke test on VPC

* Allow creation of networks by users with as numbers

* Address review comments

* Move BGPService to bgp package and inject it on BaseCmd

* Revert changes for CKS and address more comments

* Display left side menu option for AS number only for root admin

* Dynamic: create/update BGP peer with details

refer to https://docs.frrouting.org/en/latest/bgp.html

* Dynamic: fix build error and remove access to ListBgpPeers cmd for regular users

* Dynamic: assign all zone BGP peers to user networks

* Dynamic: show BGP peer info of networks only for root admin

* AS number: disable specifyasnumber for non-NSX offerings

* Dynamic: pass bgppeer details to command and fix typo with ip6 addr

* Dynamic: list BGP peers by isdedicated, and fix change bgppeers for network/vpc

* Dynamic: add UI labels

* Dynamic: add bgp peers to vpc response

* Dynamic: list bgp peers by keyword, fix list by asnumber

* Dynamic: fix list bgppeers by keyword and db schema

* Dynamic: fix list bgppeers do not return dedicated peers

* Dynamic: update UI when create network/vpc offering

* Update server/src/main/java/com/cloud/configuration/ConfigurationManagerImpl.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update tools/marvin/setup.py

* Dynamic: network mode must be same when update a network with new offering

* Dynamic: add method networkModel.isAnyServiceSupportedInNetwork

* Dynamic: rename APIs and classes

* Dynamic: fix unit tests due to previous changes

* Dynamic: validateNetworkCidrSize when auto-create subnet

* Dynamic: check AS number overlap

* Dynamic: add ActionEvent

* Dynamic: small code optimization

* Dynamic: fix ui bugs after api rename

* Dynamic: add marvin and test for ASN ranges and AS numbers

* Dynamic: add account setting use.system.bgp.peers

also
- change the default value of routed.ipv4.vpc.max.cidr.size and routed.ipv4.vpc.min.cidr.size
- change the category of settings

* static: fix ui error when delete zone ipv4 subnets

* static: small UI polish

* Dynamic: throw exception when as number is required but not passed

* Dynamic: fix typo when create FRR directory which causes network deletion failures

* Dynamic: connect to ALL (or ALL dedicated) BGP peers if no BGP peer mapping for the network/vpc

* Dynamic: throw exception when as number is required for VPC but not passed

* Dynamic: list bgp peers by useSystemBgpPeers

* Dynamic: fix frr config in VPC VR when change bgp peers

* Dynamic: create frr config even if there is no VPC tiers

* Dynamic: list bgp peers by zoneid (required for account) and account

* Dynamic: only apply FRR config for vpc tiers with dynamic routing

* Dynamic: donot send commands to router if commands size is 0

* Dynamic: fix 'new IPv6 address is not valid' when update bgp peer without IPv6

* Dynamic: throw exception if fail to allocate AS number when create network/vpc with dynamic routing

* Dynamic: enable ipv6 unicast and 'ip nht resolve-via-default'

* Dynamic: delete network/vpc if fail to allocate AS number when create network/vpc with dynamic routing

* test: add unit tests for ASN APIs

* test: add unit tests for core module

* test: add unit tests for API responses

* test: add unit tests for BgpPeerTO

* test: add minor changes

* test: add tests for create/delete/update/list RoutingFirewallRuleCmd

* Static: show ip4 routes for vpc tiers

* test: fix smoke test failure caused by type change of as number

* test: add test for Ipv4SubnetForZoneCmd

* test: add test for Ipv4SubnetForGuestNetworkCmd and BgpPeerCmd

* UI: do not show redundant router when network mode is ROUTED as RVR is not supported

* UI: hide 'Conserve mode' when networkmode is ROUTED

* test: add unit tests for ListASNumbersCmdTest

* Static: remove allocated IPv4 subnet when delete a network or vpc

* test: add unit tests for BgpPeersRules

* Dynamic: set ipv4routing from network offering

* server: list as numbers and ipv4 subnets by keyword

* server: remove dedicated bgp peers and ipv4 subnets when delete an account or domain

* server: fix dedicated ipv4 subnet is allocated to other accounts

* UI: fix allocated time format

* server: ignore project is projectid is -1 so bgppeers/ipv4subnets works in project view

* UI: add project column to bgp peers and ipv4 subnets

* server: fix list AS numbers by domain admin or normal user

* server: fix network creation when ipv4 subnet is dedicated

* UI: polish network.js

* Dynamic: fix frr config for ipv6 routing

* Static routing: support cks cluster

* Static: get/create IPv4 subnet from dedicated subnets at first

* Dynamic: add BGP peers tab

* Static: remove redundant loops

* api: add since to api and response

* server: add unit tests

---------

Co-authored-by: Nicolas Vazquez <nicovazquez90@gmail.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-09-06 08:55:17 +05:30
Rohit Yadav
85765c3125
backup: simple NAS backup plugin for KVM (#9451)
This is a simple NAS backup plugin for KVM which may be later expanded for other hypervisors. This backup plugin aims to use shared NAS storage on KVM hosts such as NFS (or CephFS and others in future), which is used to backup fully cloned VMs for backup & restore operations. This may NOT be as efficient and performant as some of the other B&R providers, but maybe useful for some KVM environments who are okay to only have full-instance backups and limited functionality.

Design & Implementation follows the `networker` B&R plugin, which is simply:

- Implement B&R plugin interfaces
- Use cmd-answer pattern to execute backup and restore operations on KVM host when VM is running (or needs to be restored) - instead of a B&R API client, relies on answers from KVM agent which executes the operations
- Backups are full VM domain snapshots, copied to a VM-specific folders on a NAS target (NFS) along with a domain XML
- Backup uses libvirt feature: https://libvirt.org/kbase/live_full_disk_backup.html orchestrated via virsh/bash script (nasbackup.sh) as the libvirt-java lacks the bindings
- Supported instance volume storage for restore operations: NFS & local storage

Refer the doc PR for feature limitations and usage details:
https://github.com/apache/cloudstack-documentation/pull/429

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-09-05 22:19:13 +05:30
Abhisar Sinha
605534b417
feature: Shared Storage Filesystem as a First Class Feature (#9208)
This PR implements Storage filesystem as a first class feature.
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Filesystem+as+a+First+Class+Feature

Documentation PR: apache/cloudstack-documentation#420

Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-09-05 17:22:32 +05:30
dahn
0fae0c021f
propagate sort order through retrieval sequence (#9624) 2024-09-05 09:37:30 +02:00
Abhishek Kumar
31b0ed0a18
framework/config,server: configkey caching (#9628)
Added caching for ConfigKey value retrievals based on the Caffeine
in-memory caching library.
https://github.com/ben-manes/caffeine
Currently, expire time for a cache is 30s and each update of the
config key invalidates the cache. On any update or reset of the
configuration, cache automatically invalidates for it.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-09-05 09:32:56 +05:30
Vishesh
f9451fce3a
Notify users when upgrades are available or restart is required for network or VPC (#7610)
Co-authored-by: Harikrishna <harikrishna.patnala@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2024-09-04 09:23:57 +02:00
Wei Zhou
6e988b8953
Merge remote-tracking branch 'apache/4.19' 2024-08-30 08:16:31 +02:00
Abhishek Kumar
0204cb75e3
ui: show guest networks for guest vlans list (#9554)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-08-29 15:15:33 +02:00
Vishesh
7fb8d77eb8
UI: Improve router listing page (#9551) 2024-08-20 14:18:49 +02:00
Bernardo De Marco Gonçalves
01c721fcda
Improvements to quota tariffs APIs and UI (#9225)
* reface quotaTariffList process and add listOnlyRemoved parameter

* add unit tests for createQuotaTariffResponse and isUserAllowedToSeeActivationRules methods

* update QuotaTariffListCmdTest

* refactor quota tariffs creation

* refactor quota tariffs update

* fix unit test in JsInterpreter

* remove unused import

* refactor quota listing and add quota deletion

* add functionality to create tariff from UI, not working when specifying dates

* fix date parsing

* add labels

* fix details view of tariffs

* new update tariff view

* fix filter placeholder

* remove debug html

* add labels

* make value field to be required when updating a tariff

* add labels

* add portuguese labels

* remove unused label

* fix updating tariff when there was no enddate specified

* refactor dates

* refactor dates

* clear code

* update disabled dates in date picker

* clear ListView component

* fix unnecessary updates when the new end date was equal to the exising end date

* fix when today was selected to start date

* add keyword to filter

* change usage type response

* add keyword and usagetype filter on UI

* fix disabled end dates in date picker

* modify datepickers to use datetime

* small fixes

* make value an unrequired field on update form

* remove duplicate import

* remove unused css classes

* add UI support for position parameter

* resize input fields to fill all available horizontal space

* remove console.log()

* remove unnecessary fully qualified names

* replace `usagetypeid` property name to `id` on `listUsageTypes` API call

* replace `usagetypeid` property name to `id` on `listUsageTypes` API call
2024-08-15 14:16:44 -03:00
nvazquez
b73f634ea6
Merge branch '4.19' 2024-08-06 12:39:13 -03:00
Vishesh
5baac44139
ui: add UI too to view and download usage records (#8615)
This PR adds a new UI tool for admins for viewing and downloading usage records.

This PR also makes startdate and enddate as non required params for generateUsageRecords. (Fixes: #7133)
2024-07-30 14:38:17 +05:30
Vishesh
0be87ac654
Fixup CKS UI for external managed clusters (#9434) 2024-07-29 10:48:58 +05:30
Rohit Yadav
f358813e1d Merge remote-tracking branch 'origin/4.19' into main
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-26 12:32:40 +00:00
Fabricio Duarte
49cd5ba64a
Fix link to removed volumes being shown in info card and list view (#8833)
* Framework for validating links in the front-end

* Rename valid links map in the list view
2024-07-24 09:09:07 -03:00
Abhishek Kumar
be552fdce9
feature: webhooks (#8674)
* api,server,ui: weebhoks feature

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* registry of message busses

* test bus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix and refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for webhook dispatch history

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes, initial ui

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for account webhook cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix remaining event bus usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for testing webhook dispatch

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* make element

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* buid fix

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for project delete check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add collapse in create

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui fix and refactor for eventditributor publish

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update org.json and add json validation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* schema fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wordings

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: improve progress button

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unrelated change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* search and count

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add payloadurl in info

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* positive progress

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix hmac key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* create webhook form fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor, address feedback

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* indentation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix filters

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove test eventbus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* default scope be Local

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add lifecycle smoke test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add test for webhook deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor - losgs and others

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* build fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* smoke test fix, log refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* get bean from all components

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: missing label

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add some more tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename setting

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0

* fix test delivery layout

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix webhook secret display

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add http to payloadurl when no scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* allow removing secretkey for webhook

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix update sslverification

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* disallow same payload url for same account

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix delivery with url w/o scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* api: listApis should return params based on caller

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/schema/src/main/resources/META-INF/db/schema-41900to42000.sql

* remove unique constraint for now

Constraint is present in Java code validations

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: add option to delete multiple deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add filter for deliveries, delete api start/endtime support

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* do not throw error when no deliveries removed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix deliveries table column sorting, time filter cancel

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove isDebugEnabled wrapping

* merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-06-10 10:40:12 +05:30
Vishesh
1955d8f3db
Add advance settings to fine tune DRS imbalance calculation (#8521)
* Use free/total instead of free metric to calculate imbalance

* Filter out hosts for condensed while checking imbalance

* Make DRS more configurable

* code refactor

* Add unit tests

* fixup

* Fix validation for drs.imbalance.condensed.skip.threshold

* Add logging and other minor changes for drs

* Add some logging for drs

* Change format for drs imbalance to string

* Show drs imbalance as percentage

* Fixup label for memorytotal in en.json
2024-02-13 11:18:53 +05:30
Wei Zhou
b8904f75dd Merge remote-tracking branch 'apache/4.18' into 4.19 2024-02-05 10:08:31 +01:00
Pearl Dsilva
5361b415e6
Image Store: View Access status of the image store and view events (#8467) 2024-02-05 14:17:29 +05:30
Wei Zhou
33bb92acce
Veeam: Support Veeam 11 and 12 (#8241)
This PR fixes several issues in the testing of Veeam 11 and Veeam12
- Import Veeam.Backup.PowerShell and silently ignore the warning messages
- Fix issue when assign vm to backup offerings, which caused by separator (\r\n)
- Fix authorization failure in veeam 12a, which is because v1_4 is not supported in veeam 12a any more
- Fix exception if backup name has space
- Fix backup metrics in veeam12, which is because powershell command does not return the values needed
- Fix Incorrect datetime value, which is because powershell command returns a datetime which is not supported in Java
- Fix issue during backup restoration if VM has both ROOT and DATA disks.

This PR also has the following update
- Add integration test test/integration/smoke/test_backup_recovery_veeam.py
- Make some UI changes
- Add zone setting backup.plugin.veeam.version. If it is not set, CloudStack will get veeam version via powershell commands.
- Add zone setting backup.plugin.veeam.task.poll.interval and backup.plugin.veeam.task.poll.max.retry
2024-01-19 18:42:01 +01:00
sato03
127fd9d2f0
UI: Project column in Default View (#8287)
The Default View has a projects toggle button that allows users to see which resources belong to projects, but there is nothing to indicate which project they belong to.

For this reason, a project column was added if the projects button is enabled, indicating the name of the project to which the resource belongs.

---------

Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2023-12-15 17:42:24 +05:30
kishankavala
5651eab49c
ObjectStore Framework with MinIO and Simulator plugins (#7752)
This PR adds Object Storage feature to CloudStack.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+CloudStack+Object+Store
2023-12-01 17:51:00 +05:30
dahn
0caf18bc1a
ui: indent issue in list view (#8184)
Fixes list view issues in UI after forward merge
2023-11-04 12:33:29 +05:30
Daan Hoogland
a15cb81c85 Merge remote-tracking branch 'apache/4.18' into main 2023-11-03 11:55:26 +01:00
Wei Zhou
e6f048bc2e
CKS: fix wrong format of cluster size on UI (#8182) 2023-11-03 10:01:12 +01:00
Harikrishna
235e4fe190
Oauth2 integration with CloudStack (#7996)
OAuth2, the industry-standard authorization or authentication framework, simplifies the process of
granting access to resources. CloudStack supports OAuth2 authentication wherein users can login into
CloudStack without using a username and password. Support for Google and Github providers has been added.
Other OAuth2 providers can be easily integrated with CloudStack using its plugin framework.

The login page will show provider options when the OAuth2 is enabled and corresponding providers are configured.

"OAuth configuration" sub-section is present under "Configuration" where admins can register the corresponding
OAuth providers.
2023-10-31 13:25:28 +05:30
Wei Zhou
bd52fa8a12
New feature: VNF templates and appliances integration (#8022) 2023-10-27 10:23:00 +02:00
Abhishek Kumar
1bb6130795 Merge remote-tracking branch 'apache/4.18' into main 2023-10-20 11:00:22 +05:30
Abhishek Kumar
f62b634033
ui: correctly show volume physical size (#8119)
Fixes #8073

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-10-19 15:58:58 +05:30
Rohit Yadav
62feb24de6 Merge remote-tracking branch 'origin/4.18'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 16:27:57 +05:30
Rohit Yadav
feb9509547
ui: assorted improvements (#7833)
This PR aims to polish the UI with following tweaks and changes:

- Increase resource and os-logo icons both in list view, user-menu bar and VM deployment form
- Fix css issues in VM deployment form when resource icons are on some of the templates/isos but not all
- Replace edit icon in the resource icon editting button on the infocard, in resource view
- Fix css marging/padding issue for nav bar and left-branding/logo
- Introduce a new Limits option in the user menu, to allow users to see their own limits when they log in
- Rename resource tab to limits tab for accounts, project and domains
- Introduce a new copy-label component, that can be clicked to copy strings; use in info-card and list view for entites such as IP addresses and UUIDs
- Add router-link to /zones/ in case of user-accounts (when /zone isn't routable in the UI)
- Show better list of nics and ssh keys pairs in infocard for VM resource view
- Standardise most resources to show state/status columns right after resource name (wherever applicable)
- Remove displayname column in VM list view, add cpu number and memory by default
- Add k8s version column in k8s list view
- Add size and phy size columns in case of template and ISOs list view, only for root/domain admins
- Add phy network router-link in case of guest VLAN list view; rearrange columns list for consistency
- Add snapshot phy size column in the snapshot list view; and router-link for volume in the snapshot list view; and missing/useful details in the volume snapshot details view
- Add a create and add data disk feature in Instances tab, just like we've add nic feature in the same

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-11 14:00:04 +05:30
GaOrtiga
0bab0db881
Introducing concept of domain VPCs (#7153)
Co-authored-by: Gabriel Ortiga Fernandes <gabriel.fernandes@scclouds.com.br>
Co-authored-by: Lopez <rodrigo@scclouds.com.br>
2023-07-29 20:06:49 +02:00
Vishesh
2fcbe6241f
cks: Add unmanaged kubernetes cluster (#7515)
There are tools like cluster-api which create and manage kubernetes cluster on CloudStack. This PR adds the option to add unmanaged kubernetes cluster which are not managed by CKS plugin. This helps provide a consolidated view of unmanaged clusters on CloudStack. The changes done make sure that operations for managed clusters are not executed for unmanaged clusters.

Two new APIs have also been added:

1. addVirtualMachinesToKubernetesCluster - to add VMs to unmanaged clusters.
2. removeVirtualMachinesFromKubernetesCluster - to remove VMs to unmanaged clusters.

Two APIs have been updated:

1. createKubernetesCluster - made KUBERNETES_VERSION_ID, SERVICE_OFFERING_ID, SIZE as not required for unmanaged clusters. Add an additional parameter, managed, which is true by default.
2. listKubernetesClusters - Add a parameter managed to filter on managed field.

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-07-03 13:07:33 +05:30
dahn
06caf32bc8
Guest os mappings improvements (#6979)
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI

* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements

* Some static analysis fixes

* Removed commented code in listview

* Guest OS list

* UI changes for adding guest os and mappings

* Added guest os mappings in guest os form

* Added new filter to guest os mapping

* Name and description changes

* VMWare Host and cluster MO unit tests

* CheckGuestOsMapping command and answer unit tests

* GetHypervisorGuestOsNames command and answer unit tests

* VmwareResource unitests

* GuestOsMapper unittests

* icon changes

* Addressed review comments

* Renaming fixes

* Removed comments

* marvin tests for guest os operations

* Added marvin tests for OS mappings

* Document links and UI improvements

* Added deduplication for the list guest OS API

* Fixed linter failure

* Few bug fixes and UI changes

* Few improvements

* Addressed code smells

* Fixed UI issues after rebase

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2023-06-27 13:49:20 +05:30