CLOUDSTACK-9438: Fix for CLOUDSTACK-9252 - Make NFS version changeable in UIJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9438
### Introduction
From #1361 it was possible to configure NFS version for secondary storage mount.
However, changing NFS version requires inserting an new detail on `image_store_details` table, with `name = 'nfs.version'` and `value = X` where X is desired NFS version, and then restarting management server for changes to take effect.
Our improvement aims to make NFS version changeable from UI, instead of previously described workflow.
### Proposed solution
Basically, NFS version is defined as an image store ConfigKey, this implied:
* Adding a new Config scope: **ImageStore**
* Make `ImageStoreDetailsDao` class to extend `ResourceDetailsDaoBase` and `ImageStoreDetailVO` implement `ResourceDetail`
* Insert `'display'` column on `image_store_details` table
* Extending `ListCfgsCmd` and `UpdateCfgCmd` to support **ImageStore** scope, which implied:
** Injecting `ImageStoreDetailsDao` and `ImageStoreDao` on `ConfigurationManagerImpl` class, on `cloud-server` module.
### Important
It is important to mention that `ImageStoreDaoImpl` and `ImageStoreDetailsDaoImpl` classes were moved from `cloud-engine-storage` to `cloud-engine-schema` module in order to Spring find those beans to inject on `ConfigurationManagerImpl` in `cloud-server` module.
We had this maven dependencies between modules:
* `cloud-server --> cloud-engine-schema`
* `cloud-engine-storage --> cloud-secondary-storage --> cloud-server`
As `ImageStoreDaoImpl` and `ImageStoreDetailsDao` were defined in `cloud-engine-storage`, and they needed in `cloud-server` module, to be injected on `ConfigurationManagerImpl`, if we added dependency from `cloud-server` to `cloud-engine-storage` we would introduce a dependency cycle. To avoid this cycle, we moved those classes to `cloud-engine-schema` module
* pr/1615:
CLOUDSTACK-9438: Fix for CLOUDSTACK-9252 - Make NFS version changeable in UI
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9565: Fix intermittent failure in test_oobm_zchange_passwordFixes intermittent integration smoke test failures caused in
test_oobm_zchange_password test.
The scope is limited to the integration test only, and full integration test suite is not necessary. We can only consider code reviews and merge on basis of Travis results.
* pr/1731:
CLOUDSTACK-9565: Fix intermittent failure in test_oobm_zchange_password
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9504: System VMs on Managed StorageThis PR makes it easier to spin up system VMs on managed storage.
Managed storage is when you have a dedicated volume on a SAN for a particular virtual disk (making it easier to deliver QoS).
For example, with this PR, you'd likely have a single virtual disk for a system VM. On XenServer, that virtual disk resides by itself in a storage repository (no other virtual disks share this storage repository).
It was possible in the past to spin up system VMs that used managed storage, but this PR facilitates the use case by making changes to the System Service Offering dialog (and by putting in some parameter checks in the management server).
JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-9504
* pr/1642:
Added support for system VMs to make use of managed storage
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9319: Use timeout when applying config to virtual routerFrom the [JIRA issue](https://issues.apache.org/jira/browse/CLOUDSTACK-9319):
> The timeout parameter is not passed down to `applyConfigToVR` inside `VirtualRoutingResource` in all cases.
>
> This timeout is worked out as 3 seconds per command or 120 seconds (whichever is larger), but because it's not passed to the first invocation, the default (120 seconds, DEFAULT_EXECUTEINVR_TIMEOUT) is used.
>
> In a recent upgrade of our Virtual Routers, the timeout was being hit and increasing `router.aggregation.command.each.timeout` had no effect. I built a custom 4.8 agent with the timeout increased to allow the upgrade to continue.
* pr/1451:
Remove dangerous prototype of applyConfigToVR
Use timeout when applying config to virtual router
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical network
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
* pr/1724:
CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical network
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
Signed-off-by: Murali Reddy <muralimmreddy@gmail.com>
This closes#1724
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
Signed-off-by: Murali Reddy <muralimmreddy@gmail.com>
This closes#1724
fix to ensure only physical network with guest traffic is picked up for
creating a private network for vpc private gateway
Signed-off-by: Murali Reddy <muralimmreddy@gmail.com>
This closes#1724
Fixes regarding VOLUME_DELETE events resulting from account deletionNew version of #1491.
**Original Description**
New version of #1373, but updated for the 4.7 branch with another fix that allows it to properly find expunged root volumes. This is a bug fix, which is why we target the 4.7 branch.
Original pull request:
Fixes regarding usage event emission.
UsageEventUtils was previously not checking deleted accounts, which meant that if an account was deleted that had some resources running on it, those resources would get destroyed without emitting any events.
Furthermore, the VOLUME_DELETE event of ROOT volumes is the responsibility of the UserVmManager, which gets circumvented when expunging resources following the account deletion. Added a check to the AccountManager which catches the ROOT volumes that need to be deleted and emits events for them.
To test this: Create a new user. As that user, create and destroy an instance. This should cause the VM_CREATE, VM_START, VM_STOP, VM_DESTROY, VOLUME_CREATE, and VOLUME_DELETE events to be emitted.
Create a new instance as the same user. Log in as admin, and delete the user. The same set of events should be emitted, and there should be no duplicate DELETE events for the ROOT volume of the previous instance.
* pr/1624:
Emit a VOLUME_DELETE usage event when account deletion destroys an instance.
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
Currently the logic about volume deletion seems to be that an event
should be emitted when the volume delete is requested, not when the
deletion completes.
The VolumeStateListener specifically ignores destroy events for ROOT
volumes, assuming that the ROOT volume only gets deleted when the
instance is destroyed and the UserVmManager should take care of it.
When deleting an account, all of its resources get destroyed, but the
instance expunging circumvents the UserVmManager, and thus we miss the
VOLUME_DESTROY usage event. The account manager now attempts to
propperly destroy the vm before expunging it. This way the destroy
logic is respected, including the event emission.
Fix Smoke Test Failuresfix ping tests to properly recognise successful ping test (on 4.8 branch)
* pr/1692:
CLOUDSTACK-9529: Cleanup resources after marvin test completes
CLOUDSTACK-9533: gateway of public IP is not handled correctly when parsing the cmd_line.json to create ips.json databag
CLOUDSTACK-9532: Use macchinina as a template for failing tests
CLOUDSTACK-9527: test_01_test_vm_volume_snapshot making test negative again
CLOUDSTACK-9531: Try template teardown without failure
CLOUDSTACK-9527: Skip tests not supported for hypervisor
CLOUDSTACK-9524: Check router hypervisor before ssh to VR
CLOUDSTACK-9522: Check for available attribute in marvin response
CLOUDSTACK-9526: Marvin test_deploy_vgpu_enabled_vm.py - Fix a hardcoded username and password
CLOUDSTACK-9515: internal LB vm is not handled when parsing cmd_line.json, resulting in internal LB vm not come up
CLOUDSTACK-9161: move quota test to plugins
Marvin Tests: Fix VPC network offering selection
fix macchinina template specificied for vmware in Marvin tests
fix ping tests to properly recognise successful ping test
CLOUDSTACK-9514: Making the credentials of the host to be picked up from the
CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical networks
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9395: Add Virtio RNG device to Instances when configuredBy adding a Random Number Generator device to Instances we can prevent
entropy starvation inside guest.
The default source is /dev/random on the host, but this can be configured
to another source when present, for example a hardware RNG.
When enabled it will add the following to the Instance's XML definition:
<rng model='virtio'>
<backend model='random'>/dev/random</backend>
</rng>
If the Instance has the proper support, which most modern distributions have,
it will have a /dev/hwrng device which it can use for gathering entropy.
* pr/1572:
CLOUDSTACK-9395: Add Virtio RNG device to Instances when configured
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
[CLOUDSTACK-9337]Enhance vcenter.py to create data center in vcenter server automaticallyThese changes have been made to support vmware deployments in CI.
For CI to create cloudstack setup with vmware, it is required to create datacenter, cluster and hosts in vcenter server before adding in cloudstack. Added few methods in vcenter.py to perform these tasks.
Please refer to CLOUDSTACK-9337 for more details.
* pr/1464:
[CLOUDSTACK-9337]Enhance vcenter.py to created data center in vcenter server automatically (Programmatically)
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9535: [API] listVMSnapshots improvement### Improvements
- Include missing fields in response: `account`, `domain`, `domainid`, `zoneid`
- Display total count of snapshots, not depending on page size
### Example
After creating 2 vm snapshots for a given vm, and making this API call: `command=listVMSnapshot&listAll=true&virtualmachineid=c8531ef8-8502-4b42-b1c5-c52ace0e7801&_=1475516598524&pagesize=1&page=1` we get this response:
```
<listvmsnapshotresponse cloud-stack-version="4.9.1.0-SNAPSHOT">
<count>2</count>
<vmSnapshot>
<id>88f7416a-8799-4245-99c6-c707cfbe6f47</id>
<name>i-2-10482-VM_VS_20161003174340</name>
<state>Ready</state>
<description>2</description>
<displayname>testsnap2</displayname>
<zoneid>0d074f25-ed31-482f-8bc5-44c9314fc417</zoneid>
<virtualmachineid>c8531ef8-8502-4b42-b1c5-c52ace0e7801</virtualmachineid>
<parent>24e44fe5-5f2e-4d35-a8f8-109b644a04e0</parent>
<parentName>testsnap</parentName>
<current>true</current>
<type>Disk</type>
<created>2016-10-03T10:43:40-0700</created>
<account>admin</account>
<domainid>5a7ffa07-3fca-11e5-9c45-005056ad45b7</domainid>
<domain>ROOT</domain>
</vmSnapshot>
</listvmsnapshotresponse>
```
**NOTES:** As in `listVirtualMachines`, despite `pagesize=1`, `count` field shows total snapshots count for given vm. Also, `account`, `domain`, `domainid`, `zoneid` fields are listed
* pr/1702:
CLOUDSTACK-9535: [API] listVMSnapshots improvement
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9525 add support for windows 10 guest os in xenserver 6.5added guest_os and guest_os_hypervisor mappings for windows10 and xenserver 6.5.0
* pr/1698:
CLOUDSTACK-9525 add support for windows 10 guest os in KVM
CLOUDSTACK-9525 add support for windows 10 guest os in vmware ESX 6.0
CLOUDSTACK-9525 add support for windows 10 guest os in xenserver 6.5.0
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
- Switches to macchinina as template for VM in the tests
- Modifies the ostype of the macchinina template to 'Other Linux (64-bit)'
- Check template download status, fixes Nonetype iterable issue
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
As per previous discussions and ticket, a template deletion may result in failure
(exception thrown) for templates that are not properly downloaded. The tearDown
method, a template may be tried for deletion but on failure we may ignore it
as account deletion/tearDown would retry to cleanup resource owned by the account.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- test_01_test_vm_volume_snapshot not supported for Xen, tests keep failing
- Skip snapshot tests for centos6/kvm as snapshot is not supported by older
qemu-img versions
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
SSH to VR for vmware, goes via the mgmt server and uses ssh keys at
/var/cloudstack path. Add suitable checks to tests failing on vmware.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Handle case where physical network instance does not have vlan attribute
- Handle case where listIso response may not have status attribute
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
resulting in internal LB vm not come up
parsing cmd_line to create 'ips' data bag, never handled internal lb vm, but still
worked due to another bug. support for internal lb vm is added with this fix
The quota integration test requires special setup and is moved to plugins
directory as in 4.9 and master branch.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9513 Migrate transifex workflow and format to jsonMove the localization resource files from Java Properties format to JSON Key-Value format
Change the Transifex sync script to handle JSON resource files instead of Properties files
Update the README
Remove old version from the Transifex configuration file
Remove unused gen-l10n.py script and update the ui/pom.xml to remove the execution of this script
cc @rhtyd @jburwell
* pr/1699:
CLOUDSTACK-9513 Migrate transifex workflow and format to json
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
By adding a Random Number Generator device to Instances we can prevent
entropy starvation inside guest.
The default source is /dev/random on the host, but this can be configured
to another source when present, for example a hardware RNG.
When enabled it will add the following to the Instance's XML definition:
<rng model='virtio'>
<rate period='1000' bytes='2048' />
<backend model='random'>/dev/random</backend>
</rng>
If the Instance has the proper support, which most modern distributions have,
it will have a /dev/hwrng device which it can use for gathering entropy.
More information: https://libvirt.org/formatdomain.html#elementsRng
Move the localization resource files from Java Properties format to JSON Key-Value format
Change the Transifex sync script to handle JSON resource files instead of Properties files
Update the README
Remove old version from the Transifex configuration file
Remove unused gen-l10n.py script and update the ui/pom.xml to remove the execution of this script
