cloudstack/test/integration/smoke/test_webhook_lifecycle.py
Abhishek Kumar be552fdce9
feature: webhooks (#8674)
* api,server,ui: weebhoks feature

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* registry of message busses

* test bus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix and refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for webhook dispatch history

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes, initial ui

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for account webhook cleanup

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix remaining event bus usage

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for testing webhook dispatch

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* make element

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* missing

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* buid fix

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes for project delete check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add collapse in create

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui fix and refactor for eventditributor publish

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* update org.json and add json validation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* schema fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wordings

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: improve progress button

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui improvements

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unrelated change

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* search and count

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add payloadurl in info

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* positive progress

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix hmac key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* create webhook form fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor, address feedback

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* indentation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix filters

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove test eventbus

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* default scope be Local

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add lifecycle smoke test

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add test for webhook deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor - losgs and others

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* build fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* smoke test fix, log refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* get bean from all components

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: missing label

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comments

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add some more tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* lint

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename setting

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* upgrade: move 4.19.0->4.20.0 to 4.19.1->4.20.0

* fix test delivery layout

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix webhook secret display

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add http to payloadurl when no scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* allow removing secretkey for webhook

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix update sslverification

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* disallow same payload url for same account

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix delivery with url w/o scheme

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* api: listApis should return params based on caller

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* wip changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/schema/src/main/resources/META-INF/db/schema-41900to42000.sql

* remove unique constraint for now

Constraint is present in Java code validations

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fixes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: add option to delete multiple deliveries

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* add filter for deliveries, delete api start/endtime support

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* do not throw error when no deliveries removed

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: fix deliveries table column sorting, time filter cancel

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove isDebugEnabled wrapping

* merge fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Daan Hoogland <daan@onecht.net>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-06-10 10:40:12 +05:30

393 lines
14 KiB
Python

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
""" BVT tests for webhooks lifecycle functionalities
"""
# Import Local Modules
from marvin.cloudstackTestCase import cloudstackTestCase
from marvin.cloudstackAPI import (listEvents)
from marvin.lib.base import (Account,
Domain,
Webhook,
SSHKeyPair)
from marvin.lib.common import (get_domain,
get_zone)
from marvin.lib.utils import (random_gen)
from marvin.cloudstackException import CloudstackAPIException
from nose.plugins.attrib import attr
import logging
# Import System modules
import time
from datetime import datetime
_multiprocess_shared_ = True
HTTP_PAYLOAD_URL = "http://smee.io/C9LPa7Ei3iB6Qj2"
HTTPS_PAYLOAD_URL = "https://smee.io/C9LPa7Ei3iB6Qj2"
class TestWebhooks(cloudstackTestCase):
@classmethod
def setUpClass(cls):
testClient = super(TestWebhooks, cls).getClsTestClient()
cls.apiclient = testClient.getApiClient()
cls.services = testClient.getParsedTestDataConfig()
# Get Zone, Domain and templates
cls.domain = get_domain(cls.apiclient)
cls.zone = get_zone(cls.apiclient, testClient.getZoneForTests())
cls._cleanup = []
cls.logger = logging.getLogger('TestWebhooks')
cls.logger.setLevel(logging.DEBUG)
@classmethod
def tearDownClass(cls):
super(TestWebhooks, cls).tearDownClass()
def setUp(self):
self.cleanup = []
self.domain1 = Domain.create(
self.apiclient,
self.services["domain"])
self.cleanup.append(self.domain1)
def tearDown(self):
super(TestWebhooks, self).tearDown()
def popItemFromCleanup(self, item_id):
for idx, x in enumerate(self.cleanup):
if x.id == item_id:
self.cleanup.pop(idx)
break
def createDomainAccount(self, isDomainAdmin=False):
self.account = Account.create(
self.apiclient,
self.services["account"],
admin=isDomainAdmin,
domainid=self.domain1.id)
self.cleanup.append(self.account)
self.userapiclient = self.testClient.getUserApiClient(
UserName=self.account.name,
DomainName=self.account.domain
)
def runWebhookLifecycleTest(self, apiclient, scope=None, domainid=None, account=None, normaluser=None, payloadurl=None, description=None, sslverification=None, secretkey=None, state=None, isdelete=True):
name = "Test-" + random_gen()
if payloadurl is None:
payloadurl = HTTP_PAYLOAD_URL
self.webhook = Webhook.create(
apiclient,
name=name,
payloadurl=payloadurl,
description=description,
scope=scope,
sslverification=sslverification,
secretkey=secretkey,
state=state,
domainid=domainid,
account=account
)
self.cleanup.append(self.webhook)
self.assertNotEqual(
self.webhook,
None,
"Check webhook created"
)
webhook_id = self.webhook.id
self.logger.debug("Created webhook: %s" % str(self.webhook.__dict__))
self.assertEqual(
name,
self.webhook.name,
"Check webhook name"
)
self.assertEqual(
payloadurl,
self.webhook.payloadurl,
"Check webhook payloadurl"
)
if state is None:
state = 'Enabled'
self.assertEqual(
state,
self.webhook.state,
"Check webhook state"
)
if scope is None or normaluser is not None:
scope = 'Local'
self.assertEqual(
scope,
self.webhook.scope,
"Check webhook scope"
)
if sslverification is None:
sslverification = False
self.assertEqual(
sslverification,
self.webhook.sslverification,
"Check webhook sslverification"
)
if domainid is not None:
if normaluser is not None:
domainid = normaluser.domainid
self.assertEqual(
domainid,
self.webhook.domainid,
"Check webhook domainid"
)
if account is not None:
self.assertEqual(
account,
self.webhook.account,
"Check webhook account"
)
if description is not None:
self.assertEqual(
description,
self.webhook.description,
"Check webhook description"
)
if secretkey is not None:
self.assertEqual(
secretkey,
self.webhook.secretkey,
"Check webhook secretkey"
)
list_webhook = Webhook.list(
apiclient,
id=webhook_id
)
self.assertNotEqual(
list_webhook,
None,
"Check webhook list"
)
self.assertEqual(
len(list_webhook),
1,
"Check webhook list length"
)
self.assertEqual(
list_webhook[0].id,
webhook_id,
"Check webhook list item"
)
if isdelete == False:
return
self.webhook.delete(apiclient)
self.popItemFromCleanup(webhook_id)
list_webhook = Webhook.list(
apiclient,
id=webhook_id
)
self.assertTrue(
list_webhook is None or len(list_webhook) == 0,
"Check webhook list after delete"
)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_01_create_webhook_admin_local(self):
self.runWebhookLifecycleTest(self.apiclient)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_02_create_webhook_admin_domain(self):
self.runWebhookLifecycleTest(self.apiclient, 'Domain', self.domain1.id)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_03_create_webhook_admin_global(self):
self.runWebhookLifecycleTest(self.apiclient, 'Global')
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_04_create_webhook_domainadmin_local(self):
self.createDomainAccount(True)
self.runWebhookLifecycleTest(self.userapiclient)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_05_create_webhook_domainadmin_subdomain(self):
self.createDomainAccount(True)
self.domain11 = Domain.create(
self.apiclient,
self.services["domain"],
parentdomainid=self.domain1.id)
self.cleanup.append(self.domain11)
self.runWebhookLifecycleTest(self.userapiclient, 'Domain', self.domain11.id)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_06_create_webhook_domainadmin_global_negative(self):
self.createDomainAccount(True)
try:
self.runWebhookLifecycleTest(self.userapiclient, 'Global')
except CloudstackAPIException as e:
self.assertTrue(
"errorText:Scope Global can not be specified for owner" in str(e),
"Check Global scope error check"
)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_07_create_webhook_user_local(self):
self.createDomainAccount()
self.runWebhookLifecycleTest(self.userapiclient)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_08_create_webhook_user_domain(self):
"""For normal user scope will always be Local irrespective of the passed value
"""
self.createDomainAccount()
self.runWebhookLifecycleTest(self.userapiclient, 'Domain', self.domain1.id, normaluser=self.account)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_09_create_webhook_user_gloabl(self):
"""For normal user scope will always be Local irrespective of the passed value
"""
self.createDomainAccount()
self.runWebhookLifecycleTest(self.userapiclient, 'Global', normaluser=self.account)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_10_create_webhook_admin_advanced(self):
self.createDomainAccount()
self.runWebhookLifecycleTest(
self.apiclient,
payloadurl=HTTPS_PAYLOAD_URL,
scope="Local",
description="Webhook",
sslverification=True,
secretkey="webhook",
state="Disabled",
domainid=self.domain1.id,
account=self.account.name
)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_11_update_webhook(self):
self.createDomainAccount()
self.runWebhookLifecycleTest(self.userapiclient, isdelete=False)
description = "Desc-" + random_gen()
secretkey = random_gen()
state = 'Disabled'
updated_webhook = self.webhook.update(
self.userapiclient,
description=description,
secretkey=secretkey,
state=state
)['webhook']
self.assertNotEqual(
updated_webhook,
None,
"Check updated webhook"
)
self.assertEqual(
description,
updated_webhook.description,
"Check webhook description"
)
self.assertEqual(
secretkey,
updated_webhook.secretkey,
"Check webhook secretkey"
)
self.assertEqual(
state,
updated_webhook.state,
"Check webhook state"
)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_12_list_user_webhook_deliveries(self):
self.createDomainAccount()
self.runWebhookLifecycleTest(self.userapiclient, isdelete=False)
now = datetime.now() # current date and time
start_time = now.strftime("%Y-%m-%d %H:%M:%S")
self.keypair = SSHKeyPair.register(
self.userapiclient,
name="Test-" + random_gen(),
publickey="ssh-rsa: e6:9a:1e:b5:98:75:88:5d:56:bc:92:7b:43:48:05:b2"
)
self.logger.debug("Registered sshkeypair: %s" % str(self.keypair.__dict__))
cmd = listEvents.listEventsCmd()
cmd.startdate = start_time
cmd.listall = True
events = self.apiclient.listEvents(cmd)
register_sshkeypair_event_count = 0
if events is not None:
for event in events:
if event.type == "REGISTER.SSH.KEYPAIR":
register_sshkeypair_event_count = register_sshkeypair_event_count + 1
time.sleep(5)
list_deliveries = self.webhook.list_deliveries(
self.userapiclient,
page=1,
pagesize=20
)
self.assertNotEqual(
list_deliveries,
None,
"Check webhook deliveries list"
)
self.assertTrue(
len(list_deliveries) > 0,
"Check webhook deliveries list length"
)
register_sshkeypair_delivery_count = 0
for delivery in list_deliveries:
if delivery.eventtype == "REGISTER.SSH.KEYPAIR":
register_sshkeypair_delivery_count = register_sshkeypair_delivery_count + 1
self.assertEqual(
register_sshkeypair_event_count,
register_sshkeypair_delivery_count,
"Check sshkeypair webhook deliveries count"
)
self.webhook.delete_deliveries(
self.userapiclient
)
list_deliveries = self.webhook.list_deliveries(
self.userapiclient
)
self.assertTrue(
list_deliveries is None or len(list_deliveries) == 0,
"Check webhook deliveries list after delete"
)
@attr(tags=["devcloud", "advanced", "advancedns", "smoke", "basic", "sg"], required_hardware="false")
def test_13_webhook_execute_delivery(self):
self.createDomainAccount()
self.runWebhookLifecycleTest(self.userapiclient, isdelete=False)
payload = "{ \"CloudStack\": \"Integration Test\" }"
delivery = self.webhook.execute_delivery(
self.userapiclient,
payload=payload
)
self.assertNotEqual(
delivery,
None,
"Check test webhook delivery"
)
self.assertEqual(
self.webhook.id,
delivery.webhookid,
"Check test webhook delivery webhook"
)
self.assertEqual(
payload,
delivery.payload,
"Check test webhook delivery payload"
)
self.assertEqual(
self.webhook.id,
delivery.webhookid,
"Check test webhook delivery webhook"
)