CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing
Uses listViewDataProvider to implement pagination on listTemplates API
requests in the UI, when a template's zone tab is viewed.
(cherry picked from commit 95c76efc3944aab9a620559a026fe37e53d8f53d)
This from the other PR, for master branch.
* pr/740:
CLOUDSTACK-8766: Fix infinite scrolling pagination for zonal template listing
Signed-off-by: Remi Bergsma <github@remi.nl>
Removing unit test that was failing to cover the resolution of the nano time - The difference between 2 consecutive calls to System.nanoTime() returns a resolution that depends on the OS implementation. Due to that, the test was failing in an intermittent manner.
- Today we decided to remove the test since it was not testing a direct feature of ACS
I did not create an issue on Jira because it's only about removing an unit test. There is no bug fix involved or feature developed.
@miguelaferreira @remibergsma @bhaisaab @DaanHoogland
Could you have a look at this change, please?
Cheers,
Wilder
* pr/747:
Removing unit test that was covering the resolution of the nano time
Signed-off-by: Remi Bergsma <github@remi.nl>
- The difference between 2 consecultive calls to System.nanoTime() returns a resolution that depends on the OS implementation. Due to that, the test was failing in an inttermmitent manner
- Today we decided to remove the test since it was not testing a direct feature of ACS
CLOUDSTACK-8721: Fixed Setting details of VM through API results in removal of all other details except the one passed in API
* pr/673:
CLOUDSTACK-8721: Fixed Setting details of VM through API results in removal of all other details except the one passed in API
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Default to notify only script to handle non-CLVM/QCOW cases.This relates to [CLOUDSTACK-8732](https://issues.apache.org/jira/browse/CLOUDSTACK-8732)
Before this commit the call to `getResizeScriptType` would throw an exception (earlier versions returned `null`, which was fine) - this caused the RBD case to fail. By changing the default to notify only we fix the case for any non-CLVM and non-QCOW cases, too.
This is RBD for now, but this should extend to new storage types supported by Libvirt natively in future.
This is my first attempted contribution: I can see a case for adding RBD logic to the actual getResizeScriptType call, too, but I felt that putting it `LibvirtResizeVolumeCommandWrapper.java` kept the special-casing of RBD (and comments about that) in one place.
### Caveat:
With Libvirt 1.2.2 this actually doesn't do the right thing - but it does do what the documentation *says* should be the right thing, so I'm going to test if this is a Libvirt bug which is fixed in a later version.
(To make it work I need to execute something like:
virsh blockresize --path vda --size 100G i-7-44-VM
where vda is the path as far as the *guest* is concerned, and not an `rbd/` path - which *should* work, but doesn't.)
* pr/696:
Default to notify only script to handle non-CLVM/QCOW cases.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8773 : NPE in CheckRouterTask, when a DomainRouter happens to be expunged at the same time
* pr/745:
CLOUDSTACK-8773 : NPE in CheckRouterTask, when a DomainRouter happens to be expunged at the same time
Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
CLOUDSTACK-8716: Verify creation of snapshot from volume when the task is performed repeatedly in zone wide primary StorageOn VMWare with a Zone wide primary storage and more than two clusters verify successful creation of snapshot multiple times.
* pr/665:
CLOUDSTACK-8716: Verify creation of snapshot from volume when the task is performed repeatedly in zone wide primary Storage
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8723: Verify API call "listUsageRecords" returns usage of new volume created after restore VMAfter restoring a running VM current ROOT disk gets destroyed and new ROOT disk gets created.
This testcase is to check if volume usage of this newly created volume is listed in listUsageRecords API.
* pr/675:
CLOUDSTACK-8723: Verify API call "listUsageRecords" returns usage of new volume created after restore VM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8765: fix vm migration failure due to different dev name on KVM
* pr/736:
CLOUDSTACK-8765: fix vm migration failure due to different dev name on KVM
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8757:FTP modules are not loaded in VRcheck bug 8757 for more details .
* pr/729:
CLOUDSTACK-8757:FTP modules are not loaded in VR
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8754: VM migration triggered by dynamic scaling is failingThis is caused by serialization failure for VmWorkMigrateForScale object. Replaced DeployDestination member present in VmWorkMigrateForScale with serializable types.
Haven't added any unit test as couldn't find a clean way to do it. Simply adding a test to do (de)serialization won't help catch any new member addition to the type which breaks serializability.
* pr/725:
CLOUDSTACK-8754: VM migration triggered by dynamic scaling is failing This is caused by serialization failure for VmWorkMigrateForScale object. Replaced DeployDestination member present in VmWorkMigrateForScale with serializable types.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Uses listViewDataProvider to implement pagination for listing templates and ISOs
in the zones tab. Dedupes isos and templates in the list views.
This closes#740
(cherry picked from commit 26700fbe766e06c3b953ee9ebae3f51ff1a08968)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-8758: Handle redirects in communication with NXS controller (a.k.a. Nicira NVP)When an NSX controller node is part of a cluster it will redirect API calls to the master controller. Because the ACS management server does not follow such redirects, if there is a change of master within a NSX cluster, the NSX device (a.k.a. NiciraNvp) needs to be reconfigured (via the management server DB).
The goal of this PR is to enable ACS management server to follow HTTP redirects sent by NSX controllers. However, other changes were made to the cloud-utils module that provides the REST client that the NSX plugin uses.
Cosmetic changes:
* Upgrade maven module structure for cloud-utils and cloud-plugin-netowkr-nvp to comply with maven default
* Several refactorings on both modules to consistently format the code, remove unused code, declare final when possible, remove auto generated comments, etc
Other changes:
* Upgrade HTTP library used in REST client to version 4.5 of Apache HTTP Components
* Implement generic REST client that supports HTTP redirects
* Implement NSX specific REST client
* Simplify NSX api implementation
* Previously existing unit tests for both the REST client and NSX api were either maintained in the same test classes, moved to new test classes (because code under tests also moved), or removed (because code under tests was also removed)
* New Marvin tests for NSX controllers
Testing:
* Ran all unit tests present in the project
* Ran Java Integration tests for NSX api targeting both a master and a slave controller
* Ran new Marvin test for NSX controller
* Manual inspection of logs to confirm redirection is taking place
* pr/737:
Use NSX specific RestClient in API implementation (NiciraNvpApi)
Refactor NSX api implementation (NiciraNvpApi)
Add NSX specific RestClient implementation
Delegate HTTP protocol activity in RESTServiceConnector to RestClient
Add basic RestClient implentation based on HTTP Components 4.5
Code clean up in cloud-plugin-network-nvp project
Code clean up in cloud-utils project
Refactor cloud-plugin-network-nvp project into Maven default structure
Refactor cloud-utils project into Maven default structure
Fix unit-test library dependencies
Add Marvin test for Nicira NVP plugin
Create Nicira NVP devices and enable plugin at deploy
Signed-off-by: Remi Bergsma <github@remi.nl>
- Add -noverify JVM arg to surefire plugin, to allow Powermockito to
de-encapsulate private methods
- Add dependency on cloud-utils test-jar to use custom HttpRequest
matchers
- All HTTP protocol activities are now handled by RestClient
- This service is now only responsible for creating requests, and
dispatching them to the client
- Provides a Simple API for creating, updating, retrieving and deleting
objects
- Upgrade version of HTTP Components to 4.5
- Add helper class to create Http clients
- Add helper class to build http requests
- Add enum with the different Http Methods
- Add constants class for HTTP related values
- XML formatting
- Fix license header
- Reorder hamcrest, junit, mockito and powermock dependencies
* Since several libraries include a subset of hamcrest classes,
hamcrest should be declared first in the pom, inorder for its
classes to be loaded first by the JVM
- Depdend on hamcrest-all and exclude hamcrest classes from other
libraries
- Support for ScaleVmCommand/NetworkRulesVmSecondaryIpCommand in resource layer
- Added support for scaling up a running VM in simulator
- Fixed some method names not following convention
CLOUDSTACK-8685: Fix for missing defaultroute on VPC router https://issues.apache.org/jira/browse/CLOUDSTACK-8685
Refactored CsRoute and CsAddress to:
- add methods to add and verify the defaultroute
- renamed methods flush, add, and routeTable to flush_table, add_route, add_table respectively.
Added some suboptimal unit tests to TestCsRoutes
* pr/738:
Fixed defaultroute on VPC routers in CsRoute and CsAddress Refactored CsRoute
pep8 cleanup
Signed-off-by: Remi Bergsma <github@remi.nl>
CLOUDSTACK-8443: Support CentOS 7 as KVM hypervisorThis adds support for CentOS 7 as a KVM hypervisor. As discussed in the Jira issue, the root cause of the issue was the `cloudstack-setup-agent` script that added this line to `/etc/libvirt/qemu.conf`:
`cgroup_controllers=["cpu"]`
On CentOS 6 this 'works', but since by default no cgroups are mounted, this setting is ignored by `libvirtd`. On CentOS 7 is does not work, because the `cpu` and `cpuacct` are 'co-mounted'. This simpy means you cannot use one without the order. The config line above instructs `libvirtd` to only use the `cpu` cgroup, which it can't.
There was code added to make sure the `cpu` cgroup was not co-mounted. It did this with a shell script, but after a reboot it was gone. It wasn't reliable and not even a solution at all. I went ahead and removed the code, because the co-mounted cgroups work fine. The real issue was the config line we put in `/etc/libvirt/qemu.conf` so much better to fix that instead.
The default for `libvirtd` is to use only cgroups that are mounted. When you remove the 'cgroup_controllers' line, the default kicks in. According to the `qemu.conf` file, the default is:
`cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]`
Then again, if it's not mounted, it's not used. So, that's a nice way to control it.
This works just fine for both RHEL/CentOS 6 and RHEL/CentOS 7.
Along the debugging, I came across some small issues here and there which I also fixed in this PR.
If you test this, be sure to build RPMs from this branch and install those on the KVM hypervisor.
At Schuberg Philis we run KVM hypervisors on CentOS 7 for months with these fixes. Now I took the time to fix the issues in CloudStack, rather than work around them.
@bhaisaab @karuturi could you please review this? Thanks!
Please note:
The next step is to fix CLOUDSTACK-8625 (Systemd profile for KVM Agent) as currently a sysvinit script is still used. Although it works, we do not have any systemd benefits. As it is a separate issue, I'll try to address it in a separate PR.
* pr/731:
CLOUDSTACK-8443: detect CentOS 7.x as RHEL 7 alike
CLOUDSTACK-8443: mention the correct logfile
CLOUDSTACK-8443: display the right hypervisor type
CLOUDSTACK-8443: don't try to fix co-mounted cgroups
Signed-off-by: Remi Bergsma <github@remi.nl>
Git PR merge script improvementsProcessed feedback from @bhaisaab and (a colleague of) @karuturi
- make the help message better (I also updated Confluence)
- make sure we can use it on other repo's as well (Documentation and such)
Tested it and works fine for me.
* pr/734:
fixed hardcoded merge message location
fix bug where the countdown would be on multiple lines
make it work with any Apache repository mirrored on GitHub
Improve error/help message in case of missing git remote
Signed-off-by: Remi Bergsma <github@remi.nl>
Example of problem:
ATTENTION: Merging pull request #731 from remibergsma/centos7-kvm into 'master' branch in 5 seconds. CTRL+c to abort..
-n 5
-n 4
-n 3
-n 2
-n 1
-n 0
Should be compatible with more environments if printf is used instead.
This setting works on CentOS 6 / RHEL 6 but does nothing, as
"cpu" cgroup is not mounted. On CentOS 7 / RHEL 7 systemd does
mount cgroups and "cpu" is co-mounted with "cpuacc". Hence, if
we specify "cpu" then this results in an error because it can
only use them both, or none.
By removing the setting, we rely on the default of qemu, which
is:
cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]
Only if they are really mounted, they will be used. So, this will
work on both version 6 and 7.
The 'fix script' didn't work well, as after a reboot you'd still have qemu
throwing errors. Now we can handle the co-mountedcgroups.
