2417 Commits

Author SHA1 Message Date
Rohit Yadav
4baaf736b9 Merge remote-tracking branch 'origin/4.17'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-07-18 19:42:44 +05:30
Abhishek Kumar
4b8af6f0fa
test: add, refactor ipv6 network, vpc tests (#6338)
- Refactor IPv6 related tests
- Adds smoke test for IPv4 network to IPv6 upgrade
- Adds smoke test for IPv6 VPC

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-07-12 12:54:53 +05:30
Harikrishna
65b0bcb069
api: Deprecated changeServiceForVirtualMachine API and replaced all the occurences in the tests with scaleVirtualMachine (#6540) 2022-07-09 21:52:51 +05:30
Rohit Yadav
e57a0f9980 Merge remote-tracking branch 'origin/4.17' 2022-07-06 09:34:02 +05:30
Abhishek Kumar
6ec06aecd7
refactor: new line, lint error fix (#6529)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-07-05 20:27:40 +05:30
Daan Hoogland
a470f3353a Merge branch '4.17' 2022-07-05 09:11:45 +02:00
John Bampton
7d23a0a759
Fix spelling (#6272) 2022-07-05 09:08:53 +02:00
John Bampton
7ba43e447c
Enable flake8 rule W292 No newline at end of file (#6274)
https://www.flake8rules.com/rules/W292.html
2022-06-30 12:08:27 +05:30
Rohit Yadav
25ed664628 Fix main branch by removing unnecessary commits, syncing 4.17 to main
This reverts commit 5b9b117301d5fe0d6e7dac817d9f6d968a69f0de.
This reverts commit 98bc4415da2eeb875dbac66d538257c808060f50.
2022-06-17 20:30:23 +05:30
utchoang
98bc4415da Revert "some component tests fixes (#5442)"
This reverts commit bfe16977211fa441edd2fc9084512149eca4e9c4.
2022-06-16 16:29:28 +07:00
nvazquez
a32f19b34d Merge branch '4.17' 2022-06-16 16:07:04 +07:00
dahn
bfe1697721
some component tests fixes (#5442)
* allow 3 hosts or more

* make numbering consistent

* stray TODO removed

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2022-06-10 08:26:30 -03:00
dahn
90a0ee0b6c
fix pseudo random behaviour in pool selection (#6307)
* refactor and log trace

* tracelogs

* shuffle pools with real randomiser

* sinlge retrieval of async job context

* some review comments addressed

* Apply suggestions from code review

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* log formatting

* integration test for distribution of volumes over storages

* move test to smoke tests

* imports

* sonarcloud issue # AYCOmVntKzsfKlhz0HDh

* spellos

* review comments

* review comments

* sonarcloud issues

* unittest

* import

* Update AbstractStoragePoolAllocatorTest.java

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2022-06-10 08:06:23 -03:00
nvazquez
0bcc609f05
Updating pom.xml version numbers for release 4.18.0.0-SNAPSHOT
Signed-off-by: nvazquez <nicovazquez90@gmail.com>
2022-06-06 12:25:35 -03:00
nvazquez
038a669d6b
Updating pom.xml version numbers for release 4.17.1.0-SNAPSHOT
Signed-off-by: nvazquez <nicovazquez90@gmail.com>
2022-06-06 12:19:44 -03:00
nvazquez
c56220fcf2
Updating pom.xml version numbers for release 4.17.0.0
Signed-off-by: nvazquez <nicovazquez90@gmail.com>
2022-05-31 14:33:47 -03:00
Wei Zhou
e15abfc3d5
cks: upgrade k8s to 1.23.3/1.24.0 in smoke test (#6388) 2022-05-17 11:19:37 -03:00
Abhishek Kumar
b15c202ee5
test: add test for importUnmanagedInstance (#6385)
* test: add test for importUnmanagedInstance

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-05-17 11:18:45 -03:00
Wei Zhou
3c55806324
test_network_ipv6.py : remove wrong icmp type (#6361) 2022-05-06 22:01:34 +02:00
dahn
f572c7ab74
Gateways after Nic update on Shared Network tests (#6355)
* initial code to set up test

* validations in marvin test
2022-05-05 19:53:31 -03:00
Abhishek Kumar
d42c0eeb11
systemvm: setup radvd correctly (#6343)
* systemvm: setup radvd correctly

Fixes radvd setup on VR for isolated network

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* donot start radvd on startup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Revert "donot start radvd on startup"

This reverts commit 2a8f737f3cf548386f025a436da5e66f59af858b.

* ipv6: do not start radvd in common.sh

* ipv6: do not add radvd to enabled_svcs in CsRedundant.py

* systemvm: add radvd to /var/cache/cloud/enabled_svcs when enable radvd

* test: fix test_network_ipv6.py

Co-authored-by: Wei Zhou <weizhou@apache.org>
2022-05-03 17:53:32 -03:00
Abhishek Kumar
23f6db7735
test: fix ipv6 network test for xenserver (#6329)
Default ACS Xenserver template, CentOS 5.6, has IPv6 disabled.
/etc/modeprobe.conf shows "options ipv6 disable=1"
To run IPv6 network test successfully on Xenserver smoketest run get_test_template will be used instead of get_template while deploying guest VM in the IPv6 guest network.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-04-28 23:28:41 -03:00
Nicolas Vazquez
06e2c38679
Fix user shared networks tests (#6319) 2022-04-27 07:30:44 -03:00
Abhishek Kumar
4a914aa88d
network: ipv6 static routes (#5786)
* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* IPv6: configure VR of isolated networks

* IPv6: add default IPv6 route in VR of isolated networks

* Reformat server/src/main/java/com/cloud/network/NetworkServiceImpl.java

* IPv6: update network to offering which support IPv6

* IPv6: update vm nic ipv6 address when update network to new offering

* IPv6: configure VPC VR to support multiple tiers with IPv6

* IPv6: add RDNSS in radvd.conf

* IPv6/UI: support ipv6 protocols in Network ACL

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for diagnostics

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* more import fromo #5594

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* IPv6: fix wrong public ipv6 in VPC VR

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update server/src/main/java/com/cloud/configuration/ConfigurationManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* ui: fix add ipv6 prefix labels, message

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: label fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* logging fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* minor ui refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ip6 events

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ip6 usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unused

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* slaac based public ip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unused

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* diagnostics fix for vr

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* firewall changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* alert and show ipv6 usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* change for network response

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ipv6 network test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix ipaddress listing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix simulator

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test and fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test temp change revert

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* use uuid

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* event syntax fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* assign vlan public IP for dualstack only if both protocols present on same vlan

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* internetprotocol in networkofferingresponse

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add tcp, udp

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* support vpc with ipv6 only on same vlan

- adds new internet protocol param to createVpcOffering API
- When DualStack internet protocol is selected for the VPC offering, tiers with network with or without IPv6 support can be deployed.
- When IPv4 internet protocol is used for the VPC offering, tiers with network with only IPv4 support can be deployed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* change and fix

allow VPC with IPv4 protocol to deploy tiers with IPv6

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix multiple routes, network guest ipv6 gateway

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* stop radvd on backup VR

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix router redundant status with ipv6

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* disable radvd for backup vr

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* correctly set ipv6 in redundant router case

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unused code

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix connection

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: don't show all protocol for egress

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix guest ipv6 for redundant VRs

Redundant VRs will not be assigned an IPv6 by ACS and guest netwrok gateway will be added as IPv6 for guest interface by systemvm scripts during setting redundant state of the VR.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix missing ipv6 on redundant vr

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix syntax

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix vpc tier redirect to show details

When redirecting to VPC tier, details tab should be active by default

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* restart radvd on primary redundant vr

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* check for ipv6 values

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove old ui change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix condition

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove gateway from backup vr

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* network upgrade fail early

when IPv6 network cannot be allocated fail before shutting down the network

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix radvd not running on RVR

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* prepare radvd.conf once

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix job polling

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix RVR for vpc with ipv6

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix ipv6 network acls

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update CsConfig.py

* add check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test: vpc offering test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test: add negative tests for guest prefix, public range

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add default ipv6 route for primary

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix dadfailed on vpc rvr

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix add iprange form, dedicate action visibility

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix adding, deleting ipv6 range

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix failing test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix missing destination cidr in ipv6 firewall

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix ipv6 nftables rules

Allow storing linger IPv6 CIDRs in DB
Specify all port range for TC{, UDP protocol rules withot ports
Fix adding nft rules by creating chains first

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix icmpv6 type, code

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix icmp type, code

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test: add more for ipv6 network

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add warning message for egress policy in ipv6 fw rule

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui,server: update ipv6 vlan range

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* subnet operations inside transaction

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* server: persistent public IPv6 for network

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix action alignment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix vpc acl for tiers

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix removing network placeholder nic

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix acl rules for ip version

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix placeholder nic and nd-neighbor block issue

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test for redundant nw

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix ping

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* systemvm: uprgade to debian 11.3.0

* ipv6: enable ipv6 in sysctl config in bootstrap.sh

* VR: fix KeyError: 'nic_ip6_cidr'

* build fix for latest event changes

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>

Co-authored-by: Wei Zhou <weizhou@apache.org>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2022-04-25 22:51:32 -03:00
Abhishek Kumar
523805c8bc
schema,server,api: events improvement (#5997)
* schema,server,api: events improvement

Add resource ID and resource type to event.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor resourcetype association with API class

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add resource anme to the response

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* more tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* new line

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add resource test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* smoke test for events resource

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* api,ui: add support for listing events for a resource

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* since key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* tests and permission changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* events for domain

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add missing license

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix js console errors

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* sort enumeration

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix event resource for vpc

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* feedback changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix order

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* events with parent resource

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing UI labels

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* donot call cmd resource methods before dispatch

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add restore vm to procedure

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add missing imports

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* resource details for more events

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add test for changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* more test, license fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wrong merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix for more event types

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2022-04-25 09:05:17 -03:00
Pearl Dsilva
48f7f10089
xen: Fix volume snapshot deletion when it has child snapshots (#6296) 2022-04-22 14:36:08 -03:00
dahn
545e89c1cb
Mshost stats (#5588)
* ms stats thread added

* initial data collection for management server

* empty list management server metrics command

* bean copy into MS metrics object

* ms status VO

* further API and DB plumbing

* minimal metrics response in API

* remove commented, refactor data collection plumbing

* javadocs

* surpress stacktrace on expected error

* update status experiment

* ms status publish framework added

* review comment addressed

* static data to DB and API, /proc/ reading

* addressing review comments

* ui for ms details

* small ui adjustment

* beanCopy

* agentcount response and system parameter

* labels

* package-lock

* add version strings to regular list API

* add shutdown time to DB

* add last start and last stop to regular list response

* distro info in regular response/session  count added

* metrics as details

* add heap used and remove details map

* thread-statusses

* move db upgrade to 4.17

* sysmem

* procmem

* ui demo comments applied

* javadoc

* get conf and log file locations

* loginfo

* cpuLoadStats

* no.remote

* extra spaces removed

* clusterlistener

* add unit to kb value

* revert accidental rename

* silly fqcn removed

* get mem info from bean is possible

* refactor long sequence for readability

* registerListener

* listUsageMetrics and isDbLocal

* rats

* local usage and db or not

* minimal listDbMetrics

* db vars and stats

* cleanup and #queries queried

* db stats calculation

* rat

* remove list response wrapper from sinlge details-lists responses

* rudimentary metrics view

* metrics table cleanup

* table makeup, collection dates

* move component to appropriate location

* capitalisation removed

* rebase error resolved

* rename deamon to daemon

* small style comments applied

* another merge issue

* naming comments and boot time

* stop/start prefixed with server

* layout-fix

* listMSMetrics test and test refactor

* usage metrics test

* db metrics test

* extra validations

* Update ui/public/locales/en.json

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* descriptions of loadaverages and replica's

* collection time on top

* cpu load on metrics overview

* DbStatsCollection

* some parameter description texts

* labels adjusted

* new output 'kernelversion' and log info cleanup

* labels

* Update api/src/main/java/com/cloud/server/ManagementServerHostStats.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/response/DbMetricsResponse.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerHostDao.java

Co-authored-by: Rodrigo D. Lopez <19981369+RodrigoDLopez@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java

Co-authored-by: Rodrigo D. Lopez <19981369+RodrigoDLopez@users.noreply.github.com>

* Update api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update api/src/main/java/org/apache/cloudstack/api/response/ManagementServerResponse.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update engine/schema/src/main/java/com/cloud/host/dao/HostDao.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update server/src/main/java/com/cloud/server/StatsCollector.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/ClusterManager.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update framework/cluster/src/main/java/com/cloud/cluster/dao/ManagementServerHostDao.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update server/src/main/java/com/cloud/server/StatsCollector.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update server/src/main/java/com/cloud/server/StatsCollector.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update server/src/main/java/com/cloud/server/StatsCollector.java

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

* Update plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

* some (more) refactorring suggestions applied

* human readable memory sizes

* rat

* actual collection time instead of query time, improved descriptions

* merge errors fixed

* optional metric values

* javadoc and logging

* names of jmx vars have changed

* vue3-compatibility

* new output parameter type

* lower retention default

* vue3 fixes

* polish comments

* polish comments 2, the reckoning

* note on usage servers

* merge conflict errors

* pollish

* conditional assertion to deal with simulator restart

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
Co-authored-by: Rodrigo D. Lopez <19981369+RodrigoDLopez@users.noreply.github.com>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2022-04-22 08:48:19 -03:00
Sina Kashipazha
debfb455ea
Added configuration and Integration test to restrict public template … (#4774)
* Added configuration and Integration test to restrict public template access.

* Move settings to domain.

* Updated integration test.

* Changed Config key's name and description.

* Justified the variable names and removed white spaces.

* Added configuration and Integration test to restrict public template access.

* Move settings to domain.

* Changed Config key's name and description.

* Justified the variable names and removed white spaces.

* Moved configuration to domain scope.

* Added integration test to travis.

* Updated the configuration's name and description.

* Extracted public template check to a separate method.

* Fixed rebase issue.

* Apply tear down changes.

* Update .travis.yml to remove the component test

The test needs to be updated to use the new configuration name

Co-authored-by: Wei Zhou <weizhou@apache.org>
2022-04-21 23:10:21 -03:00
Wei Zhou
a5014a28a6
New feature: give access permission of networks to other accounts in same domain (#5769)
* Enhancement: create Shared networks and VPC private gateways by users

* UI bug fix: pass correct domainid in CreateSharedNetworkForm

* Update #5730: fix test failure with test_guest_vlan_range.py

* Update #5730: fix test failure with test_persistent_network.py

* Update #5730: Add since to new API commands and API parameters

* Update #5730: Get first physical network for VPC private gateway if other ways do not work

* Update #5730: code optimization (return !offering.isSpecifyVlan())

* Update #5730: fix hard-coded network offering id in test_pvlan.py

* Update #5730: skip access check on the network owner if the owner is ROOT/system

* Update #5730: overlap check on cidr/startip/endip

* Update #5730: add methods to get accountid/domainid of shared networks

* Update #5730: improve integration tests

* Update #5730: update as per GutoVeronezi's comments

* Network Sharing: give network access permission to other accounts within a domain

* network: update ip in lb/pf/dnat tables when update vm nic ip

* Update #5757: create 3 separated methods for DNAT/LB/PF update

* travis: install python3-setuptools

* Network Sharing: update integration test

* Update #5769: Remove NetworkPermission.Ops

* Update #5769: Update as per Daan's comments

* Update #5769: Update as per Suresh's comments

* Update #5769: fix UI bug that accounts/projects are not listed

* Update #5769: fix domain admin can deploy vm on L2 network of other users

* Update #5769: Remove method listPermittedNetworkIdsByDomains in NetworkPermissionDao

* Update #5769: Skip network operation permissions check for root admin

* UI: fix create Isolated/L2 network form

* Update #5730: fix create Shared network form

* Update #5769: fix domain admin can deploy vm on L2 network of other users

* test: fix test_storage_policy.py

* Update #5769: fix remove_nic in test_network_permissions.py

* Update #5769: extract some codes to a method

* Update #5769: fix add/remove nic by domain admin

* Update #5769: allow domain admin to enable/disable static nat and create port forwarding rules

* Update #5769: update integration test

* Update #5769: fix unit test AssignLoadBalancerTest.java

* Update #5769: allow normal users to share network permission to other users on UI

* Update #5769: fix small UI bug with label

* Update #5769: Support L2 network as associated network

* test: sleep 30s after restarting mgt server in test_kubernetes_supported_versions.py to fix test failures with test_secondary_storage.py

* Update #5784: revert part of changes in #2420

* Update #5757: invert if condition to reduce code indentation

* Update #5769: fix regular user cannot create L2 network

* Update #5769: Add associated nework id and name in private gateway response

* Update #5769: list networks by networkfilter=Account on UI

* Update #5769: fix ui issue when list private gateways or create shared network if no isolated networks

* Update #5769: fix vue ui warnings

* Update #5679: add BaseResponseWithAssociatedNetwork and extract method setResponseAssociatedNetworkInformation

* Update #5679: extract some methods in VpcManagerImpl.java

* Update #5679: Update smoke tests as per Daan's comments

* Update #5769: fix vpc with private gateways cannot be removed when remove an acount

* Update #5769: fix unit test failures after merging latest main

* Update #5769: fix schema-41610to41700.sql

* Update #5769: fix Request failed due to empty network offering list on UI

* Update #5769: Throw exception when account is not found by name

* Update #5769: display a warning message if network offering list is empty

* Update #5769: fix an UI bug caused by previous commit b286cb76774e301d15cf2b3b6ea316eaa306018f

* Update #5769: fix UI bugs due to vue3 merge

* Update #5769: fix issue due to account type refactoring

* Update #5769: fix ui bugs due to vue3

* Update #5769: fix issue due to vue3 upgrade

* Update #5769: fix issue due to vue3 upgrade part 2

* Update #5769: fix issue due to vue3 upgrade part 3

* Update #5769: highlight default scope when create shared network on UI

* Update #5769: fix domain list is not loaded on UI

* Update #5769: fix restart/delete shared network by normal users

* Update #5769: fix restart domain-scope shared network by domain admin

* Update #5769: fix 3 UI bugs (1) double networks in list; (2) icon of first items in list; (3) account/project autoselect

* Update #5769: fix 2 ui bugs; (1) selected project is not changed when change domain; (2) no network should be selected by default

* Update #5769: fix update shared networks by domain admin/regular user

* Update #5769: fix Flicking warning message about the empty network offerings

* Update #5769: display associated network name in shared network info card

* Update #5769: fix create private gateway form

* Update #5769: fix network lists in project view

* Update #5769: fix duplicated networks in network dropdown

* Update #5769: fix failed to create shared network if associated L2 network is Setup

* Update #5769: check AccessType.OperateEntry on network in its implementation

* Revert "Update #5769: check AccessType.OperateEntry on network in its implementation"

This reverts commit c42c489e5bb26e2c780ac1be79bc8ca299a23f6e.

* Update #5769: fix keyword search in list guest vlans
2022-04-19 11:29:31 -03:00
John Bampton
c265df965e
Fix spelling (#6185)
* Fix spelling

* Update services/secondary-storage/server/src/main/java/org/apache/cloudstack/storage/resource/NfsSecondaryStorageResource.java

Co-authored-by: SadiJr <sadijacinto@gmail.com>

Co-authored-by: Rohit Yadav <rohityadav89@gmail.com>
Co-authored-by: SadiJr <sadijacinto@gmail.com>
2022-04-16 00:55:11 +05:30
John Bampton
cfa4f33940
.github/linters: Enable flake8 W293 blank line contains whitespace (#6268)
https://www.flake8rules.com/rules/W293.html
2022-04-15 20:32:52 +05:30
slavkap
4004dfcfd8
StorPool storage plugin (#6007)
* StorPool storage plugin

Adds volume storage plugin for StorPool SDS

* Added support for alternative endpoint

Added option to switch to alternative endpoint for SP primary storage

* renamed all classes from Storpool to StorPool

* Address review

* removed unnecessary else

* Removed check about the storage provider

We don't need this check, we'll get if the snapshot is on StorPool be
its name from path

* Check that current plugin supports all functionality before upgrade CS

* Smoke tests for StorPool plug-in

* Fixed conflicts

* Fixed conflicts and added missed Apache license header

* Removed whitespaces in smoke tests

* Added StorPool plugin jar for Debian

the StorPool jar will be included into cloudstack-agent package for
Debian/Ubuntu
2022-04-14 11:12:01 -03:00
Pearl Dsilva
431c352a6d
Synchronization of network devices on newly added hosts for Persistent Networks (#5977)
* Persistent Network feature & Marvin component tests

* Cleaned up comments and imports

* fixed small error

* add support to add setup persistent networks' resources when a disabled host is enabled

* small fix

* use wildcard instead of hard-coding the bridge name

* allow clean up of resources when removing a host in maintenance mode

* skip test for simulator hypervisor

Co-authored-by: shatoboar <sang-woo.bae@campus.tu-berlin.de>
2022-04-11 23:12:05 -03:00
Nicolas Vazquez
5435b0abfe
Direct download certificates additions and improvements (#6104)
* Add direct download certificates listing

* Restore class to original project

* Small refactor

* Register API

* Apply suggestions from code review

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Refactor after review

* Fix checkstyle

* Add hosts mapping to API response

* Improvements on revoke certificate

* Refactor revoke certificate API

* Fix condition

* Filter only certificates not revoked for revokeCertificate API

* Improve upload certificate and add provision certificate API

* Improve certificate response output

* Address review comments

* Refactor revoke cert test

* Fix marvin test

* Address review comments

* Fix issues

* Improvements

* Refactor upload template API response

* Fix response

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2022-04-11 22:57:23 -03:00
slavkap
2b075ed39e
Storage-based Snapshots for KVM VMs (#3724)
* VM snapshots of running KVM instance using storage providers plugins for disk snapshots

Added new virtual machine snapshot strategy which is using storage providers plugins to take/revert/delete snapshots.
You can take VM snapshot without VM memory on KVM instance, using storage providers implementations for disk snapshots.
Also revert and delete is added as functionality. Added Thaw/Freeze command for KVM instance.
The snapshots will be consistent, because we freeze the VM during the snapshotting. Backup to secondary storage is executed after
thaw of the VM and if it is enabled in global settings.

* Removed duplicated functionality

Set few methods in DefaultVMSnapshotStrategy to protected to reuse them
without duplicating the code. Remove code that is actualy not needed

* Added requirements in global setting kvm.vmstoragesnapshot.enabled

Added more information in kvm.vmstoragesnapshot.enabled global setting,
that it needs installation of:
- qemu version 1.6+
- qemu-guest-agent installed on guest virtual machine

when the option is enabled

* Added Apache license header

* Removed commented code

* If "kvm.vmstoragesnapshot.enabled" is null should be considered as false

* removed unused imports, replaced default template

Removed unused imports which causing failures and replaced template to
CentOS8

* "kvm.vmstoragesnapshot.enabled" set to dynamic

* Getting status of freeze/thaw commands not the return code

Will chacke the status if freeze/thaw of Guest VM succeded, rather than
looking for return code. Code refactoring

* removed "CreatingKVM" VMsnapshot state and events related to it

* renamed AllocatedKVM to AllocatedVM

the states should not be associated to a hypervisor type

* loggin the result of "drive-backup" command

* Check which VM snapshot strategy could handle the vm snapshots

gets the best match of VM snapshot strategy which could handle the vm
snapshots on KVM.
Other storage plugins could integrate with this functionality to support group snapshots

* Added poolId in canHandle for KVM hypervisors

Added poolId into canHandle method used to check if all volumes are on
the same PowerFlex's storage pool

* skip smoke tests if the hypervisor's OS type is CentOS

This PR works with functionality included in qemu-kvm-ev which
does not come by default on CentOS. The smoke tests will be skipped if
the hypervisor OS is CentOS

* Added missed import in smoke test

* Suggested change to use ` org.apache.commons.lang.StringUtils.isNotBlank`

* Fix getting device on Ubuntu

On Ubuntu the device isn't provided and we have to get it from
node-name parameter. For drive-backup command (for Ubuntu) is needed and job-id which
is the value of node-name (this extra param works on Ubuntu and CentOS as well).

* Removed new snapshot states and functionality for NFS

* throw CloudRuntimeException

provide a properer error message when delete VM snapshot fails

* exclude GROUP snapshots when listing snapshots

* Skip tests if there is pool with NFS/Local

* address comments
2022-04-07 21:42:12 -03:00
Wei Zhou
960513aa97
test: update test_kubernetes_clusters.py to support advanced zone with security groups (#6183) 2022-04-05 12:02:18 +02:00
John Bampton
15937369fe
Fix spelling (#6161)
* Fix spelling

* Fix spelling
2022-03-29 00:21:07 -03:00
John Bampton
182899d0b5
Add Python flake8 linting for W291 trailing whitespace with Super-Linter (#4687)
* refactor: remove trailing whitespace from Python files

* Add the GitHub Super-Linter

Add Python flake8 linting for W291 trailing whitespace

* Add licenses
2022-03-28 11:40:26 -03:00
Pearl Dsilva
f5e77f7cf5
travis: Fix failing travis tests on main (#6152)
* travis: Fix failing travis tests on main

* address comments

* address comments
2022-03-23 09:25:27 -03:00
John Bampton
6401c850b7
Fix spelling (#6064)
* Fix spelling

- `interupted` to `interrupted`
- `paramter` to `parameter`

* Fix more typos
2022-03-08 13:02:35 -03:00
John Bampton
08bb4ad9fe
spelling: Fix spelling Occured to Occurred (#6061)
Fix typos
2022-03-07 15:07:02 +05:30
John Bampton
980c0bff03
Fix spelling (#6041)
Changed `occured` to `occurred`
2022-03-03 13:10:14 -03:00
Suresh Kumar Anaparti
bc70535ee5
Updating pom.xml version numbers for release 4.16.2.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2022-03-03 18:15:33 +05:30
Suresh Kumar Anaparti
cad9332082
Updating pom.xml version numbers for release 4.16.1.0
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2022-02-25 19:01:16 +05:30
Pearl Dsilva
ee947e1310
[marvin, VMware] Fix repeated test failures observed on 'main' branch (#5992)
* [marvin, VMware] Fix test_06_disk_offering_strictness_false observed on 'main' branch

* Fix test

* update config only if false

* reset full clone setting to original values
2022-02-16 13:46:03 -03:00
Suresh Kumar Anaparti
b50542a11c
Merge branch '4.16' into main 2022-02-15 19:26:04 +05:30
Pearl Dsilva
e0a5df50ce
CKS Enhancements and SystemVM template upgrade improvements (#5863)
* This PR/commit comprises of the following:
- Support to fallback on the older systemVM template in case of no change in template across ACS versions
- Update core user to cloud in CKS
- Display details of accessing CKS nodes in the UI - K8s Access tab
- Update systemvm template from debian 11 to debian 11.2
- Update letsencrypt cert
- Remove docker dependency as from ACS 4.16 onward k8s has deprecated support for docker - use containerd as container runtime

* support for private registry - containerd

* Enable updating template type (only) for system owned templates via UI

* edit indents

* Address comments and move cmd from patch file to cloud-init runcmd

* temporary change

* update k8s test to use k8s version 1.21.5 (instead of 1.21.3 - due to https://github.com/kubernetes/kubernetes/pull/104530)

* support for private registry - containerd

* Enable updating template type (only) for system owned templates via UI

* smooth upgrade of cks clusters

* update pom file with temp download.cloudstack.org testing links

* fix pom

* add cgroup config for containerd

* add systemd config for kubelet

* add additional info during image registry config

* update to official links
2022-02-15 18:27:14 +05:30
Suresh Kumar Anaparti
212a4dedfc
Merge branch '4.16' into main 2022-02-10 20:56:52 +05:30
dahn
122b018057
Quota test fixes (#5959)
* fixes and cleanups on quota integration test

* text changes in exception messages

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2022-02-10 18:57:21 +05:30
Suresh Kumar Anaparti
850714c1fc
Merge branch '4.16' into main 2022-02-10 10:57:29 +05:30