CLOUDSTACK-9552: Allow egress TCP/53 implicitly in Basic NetworkingAllow DNS queries over TCP when egress filtering is configured.
When using DNSSEC more and more queries are done over TCP and this
requires 53/TCP to be allowed.
Signed-off-by: Wido den Hollander wido@widodh.nl
* pr/1713:
CLOUDSTACK-9552: Allow egress TCP/53 implicitly in Basic Networking
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
SSVM downloader now handles redirects properly.New version of #1607, opened against 4.9 so it can be forward merged.
**Original Description**
Previously it was using the HttpClient to make an initial request to an ISO. This would follow redirects. Then it would make another request using built-in Java URL and InputStream, which doesn't follow redirects. This results in the ISO getting stuck at 0% forever and also causing DOS effects.
* pr/1746:
SSVM downloader now handles redirects properly.
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario
* pr/1714:
CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9554 Juniper Contrail plug-in is publishing events to wronJuniper Contrail plug-in is publishing events to message bus instead of event bus
* pr/1715:
CLOUDSTACK-9554 Juniper Contrail plug-in is publishing events to wrong message bus
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
A constructor signature has changed between 4.8 and 4.9+ branches which caused
failure in a unit test introduced by PR #1694. This fixes the unit test by
passing null as the additional parameter (the test does not need instantiated
object).
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9551: Move java tmp dir to cloudstack-agent's path to avoidMove java tmp dir to cloudstack-agent's path to avoid noexec on /tmp
* pr/1728:
CLOUDSTACK-9551: Move java tmp dir to cloudstack-agent's path to avoid noexec on /tmp
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9509: Host Connects Without StorageKVM hosts on shared storage failure was accepted by mgmt server with the
host state as Up, even though there was no primary/shared storage available on
it. This patch offers a quick fix by throwing an exception in the storage monitor
which connects storage pool on host. The failure is trapped by agent manager
that disconnects the agent without any investigation.
Based on Lab tests, KVM agent may take upto 2 minutes to attempt NFS mount when
the storage is inaccessible (firewalled, or shutdown) before returning back with
an error. It is safe to assume that this won't add pressure on mgmt server due to
several reconnection attempts, and KVM agent would retry reconnection every 2
minutes.
For such KVM hosts, where failure happens due to storage issues; they will be
briefly put in Alert state but will be mostly be in Connecting state during which
the KVM host attempts to mount/reconfigure NFS storage pool.
/cc @jburwell @karuturi
@blueorangutan package
* pr/1694:
CLOUDSTACK-9509: Host Connects Without Storage
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Previously it was using the HttpClient to make an initial request
to an ISO. This would follow redirects. Then it would make
another request using built-in Java URL and InputStream, which
doesn't follow redirects. This results in the ISO getting stuck
at 0% forever and also causing DOS effects.
CLOUDSTACK-9534: Allow VR to be destroy in UI when in running stateThis allows us to destroy a VR that is in running state from the UI. This action is possible via cloudmonkey/APIs, and would be useful for anyone who wants to simply destroy a VR from the UI without having to stop it first.
As this is a pure UI fix, manual testing would be needed but no regression integration testing is necessary.
/cc @jburwell @karuturi
@blueorangutan package
Screenshot:
* pr/1701:
CLOUDSTACK-9534: Allow VR to be destroy in UI when in running state
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9550: Use context to filter items in a metrics viewUse available context to filter a metrics view based on zone, cluster, host
in the context object. This fixes metrics view filtering when metrics view is
viewed via Zone->Compute and Storage-> for a resource.
/cc @jburwell @karuturi -- this is a pure UI fix, one manual test LGTM would be required.
* pr/1712:
CLOUDSTACK-9550: Use context to filter items in a metrics view
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Allow DNS queries over TCP when egress filtering is configured.
When using DNSSEC more and more queries are done over TCP and this
requires 53/TCP to be allowed.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
In VmwareResource, findRouterEthDeviceIndex() method find ethernet interface index given
the mac address. This method is used, once a nic is plugged to determine ethernet interface.
"/proc/sys/net/ipv4/conf" from the VR and looped through the devices to find the right
ethernet interface. However current logic read it once, and loops through the device list.
Its observerd device may not show up '/proc/sys/net/ipv4/conf' immediatly once NIC is plugged
in the VM from vCenter.Fix ensured, while waiting for 15 sec in the loop, read the latest
content from /proc/sys/net/ipv4/conf, so that right device list is processed.
're' meta chars, and causing VPN user add/deelte to fail
-there is no real use of python 're' in CsFile.py utility methods searchString, deleteLine
Replacing with regular string search instead.
-modifying the smoke test for VPN user add/delete to have all permissable chars
CLOUDSTACK-9565: Fix intermittent failure in test_oobm_zchange_passwordFixes intermittent integration smoke test failures caused in
test_oobm_zchange_password test.
The scope is limited to the integration test only, and full integration test suite is not necessary. We can only consider code reviews and merge on basis of Travis results.
* pr/1731:
CLOUDSTACK-9565: Fix intermittent failure in test_oobm_zchange_password
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical network
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
* pr/1724:
CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical network
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
Signed-off-by: Murali Reddy <muralimmreddy@gmail.com>
This closes#1724
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
Signed-off-by: Murali Reddy <muralimmreddy@gmail.com>
This closes#1724
Fixes regarding VOLUME_DELETE events resulting from account deletionNew version of #1491.
**Original Description**
New version of #1373, but updated for the 4.7 branch with another fix that allows it to properly find expunged root volumes. This is a bug fix, which is why we target the 4.7 branch.
Original pull request:
Fixes regarding usage event emission.
UsageEventUtils was previously not checking deleted accounts, which meant that if an account was deleted that had some resources running on it, those resources would get destroyed without emitting any events.
Furthermore, the VOLUME_DELETE event of ROOT volumes is the responsibility of the UserVmManager, which gets circumvented when expunging resources following the account deletion. Added a check to the AccountManager which catches the ROOT volumes that need to be deleted and emits events for them.
To test this: Create a new user. As that user, create and destroy an instance. This should cause the VM_CREATE, VM_START, VM_STOP, VM_DESTROY, VOLUME_CREATE, and VOLUME_DELETE events to be emitted.
Create a new instance as the same user. Log in as admin, and delete the user. The same set of events should be emitted, and there should be no duplicate DELETE events for the ROOT volume of the previous instance.
* pr/1624:
Emit a VOLUME_DELETE usage event when account deletion destroys an instance.
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
Currently the logic about volume deletion seems to be that an event
should be emitted when the volume delete is requested, not when the
deletion completes.
The VolumeStateListener specifically ignores destroy events for ROOT
volumes, assuming that the ROOT volume only gets deleted when the
instance is destroyed and the UserVmManager should take care of it.
When deleting an account, all of its resources get destroyed, but the
instance expunging circumvents the UserVmManager, and thus we miss the
VOLUME_DESTROY usage event. The account manager now attempts to
propperly destroy the vm before expunging it. This way the destroy
logic is respected, including the event emission.
KVM hosts on shared storage failure was accepted by mgmt server with the
host state as Up, even though there was no primary/shared storage available on
it. This patch offers a quick fix by throwing an exception in the storage monitor
which connects storage pool on host. The failure is trapped by agent manager
that disconnects the agent without any investigation.
Based on Lab tests, KVM agent may take upto 2 minutes to attempt NFS mount when
the storage is inaccessible (firewalled, or shutdown) before returning back with
an error. It is safe to assume that this won't add pressure on mgmt server due to
several reconnection attempts, and KVM agent would retry reconnection every 2
minutes.
For such KVM hosts, where failure happens due to storage issues; they will be
briefly put in Alert state but will be mostly be in Connecting state during which
the KVM host attempts to mount/reconfigure NFS storage pool.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fix Smoke Test Failuresfix ping tests to properly recognise successful ping test (on 4.8 branch)
* pr/1692:
CLOUDSTACK-9529: Cleanup resources after marvin test completes
CLOUDSTACK-9533: gateway of public IP is not handled correctly when parsing the cmd_line.json to create ips.json databag
CLOUDSTACK-9532: Use macchinina as a template for failing tests
CLOUDSTACK-9527: test_01_test_vm_volume_snapshot making test negative again
CLOUDSTACK-9531: Try template teardown without failure
CLOUDSTACK-9527: Skip tests not supported for hypervisor
CLOUDSTACK-9524: Check router hypervisor before ssh to VR
CLOUDSTACK-9522: Check for available attribute in marvin response
CLOUDSTACK-9526: Marvin test_deploy_vgpu_enabled_vm.py - Fix a hardcoded username and password
CLOUDSTACK-9515: internal LB vm is not handled when parsing cmd_line.json, resulting in internal LB vm not come up
CLOUDSTACK-9161: move quota test to plugins
Marvin Tests: Fix VPC network offering selection
fix macchinina template specificied for vmware in Marvin tests
fix ping tests to properly recognise successful ping test
CLOUDSTACK-9514: Making the credentials of the host to be picked up from the
CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical networks
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Use available context to filter a metrics view based on zone, cluster, host
in the context object. This fixes metrics view filtering when metrics view is
viewed via Zone->Compute and Storage-> for a resource.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9535: [API] listVMSnapshots improvement### Improvements
- Include missing fields in response: `account`, `domain`, `domainid`, `zoneid`
- Display total count of snapshots, not depending on page size
### Example
After creating 2 vm snapshots for a given vm, and making this API call: `command=listVMSnapshot&listAll=true&virtualmachineid=c8531ef8-8502-4b42-b1c5-c52ace0e7801&_=1475516598524&pagesize=1&page=1` we get this response:
```
<listvmsnapshotresponse cloud-stack-version="4.9.1.0-SNAPSHOT">
<count>2</count>
<vmSnapshot>
<id>88f7416a-8799-4245-99c6-c707cfbe6f47</id>
<name>i-2-10482-VM_VS_20161003174340</name>
<state>Ready</state>
<description>2</description>
<displayname>testsnap2</displayname>
<zoneid>0d074f25-ed31-482f-8bc5-44c9314fc417</zoneid>
<virtualmachineid>c8531ef8-8502-4b42-b1c5-c52ace0e7801</virtualmachineid>
<parent>24e44fe5-5f2e-4d35-a8f8-109b644a04e0</parent>
<parentName>testsnap</parentName>
<current>true</current>
<type>Disk</type>
<created>2016-10-03T10:43:40-0700</created>
<account>admin</account>
<domainid>5a7ffa07-3fca-11e5-9c45-005056ad45b7</domainid>
<domain>ROOT</domain>
</vmSnapshot>
</listvmsnapshotresponse>
```
**NOTES:** As in `listVirtualMachines`, despite `pagesize=1`, `count` field shows total snapshots count for given vm. Also, `account`, `domain`, `domainid`, `zoneid` fields are listed
* pr/1702:
CLOUDSTACK-9535: [API] listVMSnapshots improvement
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
This allows admins/users to destroy a VR that is in running state from the UI.
This action is possible via cloudmonkey/APIs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Switches to macchinina as template for VM in the tests
- Modifies the ostype of the macchinina template to 'Other Linux (64-bit)'
- Check template download status, fixes Nonetype iterable issue
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
