29504 Commits

Author SHA1 Message Date
Rohit Yadav
65eb3202c2 CLOUDSTACK-9534: Allow VR to be destroy in UI when in running state
This allows admins/users to destroy a VR that is in running state from the UI.
This action is possible via cloudmonkey/APIs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-07 15:15:37 +05:30
Rohit Yadav
409e504f6f CLOUDSTACK-9529: Cleanup resources after marvin test completes
- Cleans up volumes, snapshots and VM after running test_snapshots

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-06 16:41:51 +05:30
Murali Reddy
93ac134f1e CLOUDSTACK-9533: gateway of public IP is not handled correctly when parsing the
cmd_line.json to create ips.json databag

using 'gateway' in cmd_line for the public ip gateway while creating ips.json databag
2016-10-06 16:41:29 +05:30
Rohit Yadav
0c146e14c6 CLOUDSTACK-9532: Use macchinina as a template for failing tests
- Switches to macchinina as template for VM in the tests
- Modifies the ostype of the macchinina template to 'Other Linux (64-bit)'
- Check template download status, fixes Nonetype iterable issue

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-06 16:41:22 +05:30
Boris
b0d527688f CLOUDSTACK-9527: test_01_test_vm_volume_snapshot making test negative again
- test_01_test_vm_volume_snapshot not supported for Xen, tests keep failing

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-06 15:38:08 +05:30
Rohit Yadav
5545bcaa9d CLOUDSTACK-9531: Try template teardown without failure
As per previous discussions and ticket, a template deletion may result in failure
(exception thrown) for templates that are not properly downloaded. The tearDown
method, a template may be tried for deletion but on failure we may ignore it
as account deletion/tearDown would retry to cleanup resource owned by the account.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-06 15:38:01 +05:30
Boris
394f2e359f CLOUDSTACK-9527: Skip tests not supported for hypervisor
- test_01_test_vm_volume_snapshot not supported for Xen, tests keep failing
- Skip snapshot tests for centos6/kvm as snapshot is not supported by older
  qemu-img versions

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Rohit Yadav
dc93268788 CLOUDSTACK-9524: Check router hypervisor before ssh to VR
SSH to VR for vmware, goes via the mgmt server and uses ssh keys at
/var/cloudstack path. Add suitable checks to tests failing on vmware.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Murali Reddy
1f27874eaf CLOUDSTACK-9522: Check for available attribute in marvin response
- Handle case where physical network instance does not have vlan attribute
- Handle case where listIso response may not have status attribute

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Boris
d4f0bf1ca9 CLOUDSTACK-9526: Marvin test_deploy_vgpu_enabled_vm.py - Fix a hardcoded username and password
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Murali Reddy
052a9af023 CLOUDSTACK-9515: internal LB vm is not handled when parsing cmd_line.json,
resulting in internal LB vm not come up

parsing cmd_line to create 'ips' data bag, never handled internal lb vm, but still
worked due to another bug. support for internal lb vm is added with this fix
2016-10-05 15:11:00 +05:30
Rohit Yadav
0eb4fae8b7 CLOUDSTACK-9161: move quota test to plugins
The quota integration test requires special setup and is moved to plugins
directory as in 4.9 and master branch.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Paul Angus
79f0adf397 Marvin Tests: Fix VPC network offering selection
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Paul Angus
99da5338ff fix macchinina template specificied for vmware in Marvin tests
- Fix macchnina template url
- Fix a variable scope issue

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Paul Angus
47104cf2b3 fix ping tests to properly recognise successful ping test 2016-10-05 15:11:00 +05:30
Boris
7a1efd687e CLOUDSTACK-9514: Making the credentials of the host to be picked up from the
Use config file, instead of hardcoded crendentials

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Murali Reddy
fcdc2cc529 CLOUDSTACK-9511: fix test_privategw_acl.py to handle multiple physical networks
modified script to handle case where there can be multple physical networks
in the zone

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-05 15:11:00 +05:30
Wido den Hollander
c1997a1705
CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector
Both host and path could have been NULL which causes the StatsCollector
no to start properly.

By checking if the Strings are not Empty or Null we make sure the StatsCollector
always runs and does not prevent the Management Server from starting.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

Conflicts:
	server/src/com/cloud/server/StatsCollector.java
2016-09-29 16:00:38 +02:00
Abhinandan Prateek
a813baed49 CLOUDSTACK-9460: For long running transactions, if the connection is
timed out by the mysql server then refresh it
2016-09-19 14:08:26 +05:30
Nathan Johnson
53fd4a7997 CLOUDSTACK-9451
Honor the forced parameter to stop virtual machine api call.
2016-09-09 13:00:17 -05:00
Rohit Yadav
9eb8b2e90b Merge pull request #1636 from milamberspace/L10N-update-4.8-20160812-fix-spanish-quote
Fix a quote issue with Spanish L10N (from transifex translation)This fix is for the 4.8 branch.

* pr/1636:
  Fix a quote issue with Spanish L10N (from transifex translation)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-09-02 00:33:10 +00:00
Rohit Yadav
f4ae87cb05 Merge pull request #1654 from shapeblue/jsb/4.8.2.0-version
Updating pom.xml version numbers for release 4.8.2.0-SNAPSHOTOften, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: John Burwell <meaux@cockamamy.net>

/cc @rhtyd @karuturi

* pr/1654:
  Adds support for four position versions and optional db upgrades

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-09-01 15:11:20 +05:30
John Burwell
8d11511b1f Adds support for four position versions and optional db upgrades
Often, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 13:32:32 +05:30
Rohit Yadav
19f159a96b Merge pull request #1657 from shapeblue/4.8-usageserverfix
[lts/blocker] CLOUDSTACK-9467: Add symlink to key file for usage serverOn fresh installation, the usage server fails to start if the `key` file does
not exist in its classpath. The issue is reproducible in environments (such as Trillian)
where the usage server is installed before cloudstack-setup-databases has been called.
Before the cloudstack db has been setup, the key file does not exist at its
default location and installation of usage-server fails to add a symlink to the
key file.

This fix adds a default symlink to `/etc/cloudstack/management/key` if a
symlink/file does not already exist in the /etc/cloudstack/usage directory.

On new installation, in the post-installation steps it checks if the symlink
or file exists, and adds a symlink if it does not exist. On existing
installations, if symlink or file exists then it will skip adding symlink.

/cc @jburwell @PaulAngus @karuturi
@blueorangutan package

* pr/1657:
  CLOUDSTACK-9467: Add symlink to key file for usage server

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-24 13:04:02 +05:30
Rohit Yadav
3fa87626a2 Merge pull request #1656 from shapeblue/4.9-fk410to420fix
CLOUDSTACK-9466: Fix fk constraint failure in upgrade pathIn the 4.1.0-4.2.0 db upgrade path, it creates new tables to store secondary
(nfs) storage in image_store table and volumes in volume_store_ref table. In
the upgrade path, it first tries to migrate NFS storage pool where it excludes
storage pools which have been removed, but it migrates all the volumes without
checking if their storage pools have been removed. This causes fk constraint
failure as the volume/row being inserted refers to a storage pool which does
not exist in the image_store table.

The fix migrates all the nfs storage pools to image_store including removed
storage pools and in doing so migrates with the 'removed' field. This fixes
db upgrade for old pre-4.0 and 4.0/4.1 CloudStack clouds.

/cc @jburwell @PaulAngus @karuturi @abhinandanprateek @murali-reddy

* pr/1656:
  CLOUDSTACK-9466: Fix fk constraint failure in upgrade path

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-24 12:39:46 +05:30
Rohit Yadav
7530f4b6dd CLOUDSTACK-9466: Fix fk constraint failure in upgrade path
In the 4.1.0-4.2.0 db upgrade path, it creates new tables to store secondary
(nfs) storage in image_store table and volumes in volume_store_ref table. In
the upgrade path, it first tries to migrate NFS storage pool where it excludes
storage pools which have been removed, but it migrates all the volumes without
checking if their storage pools have been removed. This causes fk constraint
failure as the volume/row being inserted refers to a storage pool which does
not exist in the image_store table.

The fix migrates all the nfs storage pools to image_store including removed
storage pools and in doing so migrates with the 'removed' field. This fixes
db upgrade for old pre-4.0 and 4.0/4.1 CloudStack clouds.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-24 12:03:46 +05:30
Rohit Yadav
8cd44b1684 CLOUDSTACK-9467: Add symlink to key file for usage server
On fresh installation, the usage server fails to start if the `key` file does
not exist in its classpath. The issue is reproducible in environments where
the usage server is installed before cloudstack-setup-databases has been called.
Before the cloudstack db has been setup, the key file does not exist at its
default location and installation of usage-server fails to add a symlink to the
key file.

This fix adds a default symlink to `/etc/cloudstack/management/key` if a
symlink/file does not already exist in the /etc/cloudstack/usage directory.

On new installation, in the post-installation steps it checks if the symlink
or file exists, and adds a symlink if it does not exist. On existing
installations, if symlink or file exists then it will skip adding symlink.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-23 17:20:41 +05:30
Milamber
6df5df5694 Fix a quote issue with Spanish L10N (from transifex translation) 2016-08-23 06:36:42 +01:00
Will Stevens
740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens
007c9f0d3a Merge branch '4.8.1-RC20160808T1006' into 4.8 2016-08-15 10:28:33 -04:00
Will Stevens
a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
4.8.1
2016-08-08 10:06:21 -04:00
Rajani Karuturi
a54a3b5cd5 Merge release branch 4.7 to 4.8
* 4.7:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:29:16 +05:30
Rajani Karuturi
9127af61e4 Merge pull request #1612 from shapeblue/package-marvin-tests
packaging: Marvin and integration-tests packagesThis introduces two new cloudstack packages: marvin and integration-tests.
The two packages will make it easier for CI systems to install Marvin for a
specific cloudstack release/build and run integration tests that are specific
for that version/build.

Since Marvin may have its own dependencies, we're bundling the Marvin source tarball and installing it with pip in the post-installation script/phase.

This also fixed a build issue with using juniper-api maven repo.

/cc @swill @PaulAngus @jburwell @wido @bvbharat

* pr/1612:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-05 11:55:48 +05:30
Rohit Yadav
e5750b3331 server: give more memory to tests
Increases allowed max and permgen memory flags to maven-surefire plugins.
This fixes unit test failures in cloud-server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit fd7273b446738c0ebfae84189502dbdcd18bfd42)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-04 15:52:29 +05:30
Rohit Yadav
de041df74d packaging: Marvin and integration-tests packages
This introduces two new cloudstack packages: marvin and integration-tests.
The two packages will make it easier for CI systems to install Marvin for a
specific cloudstack release/build and run integration tests that are specific
for that version/build.

- maven: add explicit juniper-contrail-api maven repository
- marvin: build source distribution for both install and package mvn phases

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-07-20 16:31:23 +05:30
Will Stevens
01758ee867 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9342: Site to Site VPN PFS not being set correctly
2016-07-18 14:06:08 -04:00
Will Stevens
adaf410109 Merge pull request #1480 from Slair1/S2S-VPN-PFS-Setting
CLOUDSTACK-9342: Site to Site VPN PFS not being set correctlyBug in code set PFS to the same value (yes/no) as DPD.

file.addeq(" pfs=%s" % CsHelper.bool_to_yn(obj['dpd']))

* pr/1480:
  CLOUDSTACK-9342: Site to Site VPN PFS not being set correctly

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:05:18 -04:00
Will Stevens
142f07d77f Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage
  Added ASF license to unit test file
  Added unit test to verify ordering
  Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted
2016-06-28 11:20:16 -04:00
Will Stevens
8eedeade75 Merge pull request #1596 from anshul1886/CLOUDSTACK-9353
CLOUDSTACK-9353: [XenServer] Fixed VM migration with storageIn turn this also fixes VM migration with local storage

This PR is created against 4.7 and can be forward merged to future branches also.

* pr/1596:
  CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-06-28 11:18:58 -04:00
Will Stevens
3952e3e83e Merge pull request #1581 from pdube/network-acl-rules-order
CLOUDSTACK-9404 Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted.

Issue: https://issues.apache.org/jira/browse/CLOUDSTACK-9404

In this example, I created rules with the port numbers the same as the rule numbers.

Chain ACL_INBOUND_eth2 (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             225.0.0.50
ACCEPT     all  --  anywhere             vrrp.mcast.net
DROP       tcp  --  anywhere             anywhere             tcp dpt:netstat
DROP       tcp  --  anywhere             anywhere             tcp dpt:10
DROP       tcp  --  anywhere             anywhere             tcp dpt:5
DROP       tcp  --  anywhere             anywhere             tcp dpt:3
DROP       tcp  --  anywhere             anywhere             tcp dpt:2
DROP       all  --  anywhere             anywhere

We can see above that the rules are inverted.

After the fix:

Chain ACL_INBOUND_eth2 (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             225.0.0.50
ACCEPT     all  --  anywhere             vrrp.mcast.net
DROP       tcp  --  anywhere             anywhere             tcp dpt:2
DROP       tcp  --  anywhere             anywhere             tcp dpt:3
DROP       tcp  --  anywhere             anywhere             tcp dpt:5
DROP       tcp  --  anywhere             anywhere             tcp dpt:10
DROP       tcp  --  anywhere             anywhere             tcp dpt:netstat
DROP       all  --  anywhere             anywhere

* pr/1581:
  Added ASF license to unit test file
  Added unit test to verify ordering
  Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-06-28 11:17:45 -04:00
Anshul Gangwar
458264aac8 CLOUDSTACK-9353: [XenServer] Fixed VM migration with storage 2016-06-24 09:31:20 +05:30
Patrick Dube
9cdd23fdc7 Added ASF license to unit test file 2016-06-03 08:48:47 -04:00
Patrick Dube
4c97a3981d Added unit test to verify ordering 2016-06-02 13:44:39 -04:00
Patrick Dube
caf4a48075 Fixed ordering of network ACL rules being sent to the VR. The comparator was inverted 2016-06-02 13:15:38 -04:00
Will Stevens
744f9d5615 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
  CLOUDSTACK-9369: Restrict default login to ldap/native users
  Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.
  Emit template UUID and class type over event bus when deleting templates.
2016-05-27 15:03:42 -04:00
Will Stevens
9275ba27f3 Merge pull request #1565 from greenqloud/pr-lsb-release-4.7
Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.New version of #1412, based on the 4.7 branch.

* pr/1565:
  Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:02:46 -04:00
Will Stevens
cd5f3320c8 Merge pull request #1564 from greenqloud/pr-emit-template-uuid-on-delete-4.7
Emit template UUID and class type over event bus when deleting templatesNew version of #1378 for the 4.7b branch instead of 4.6.

* pr/1564:
  Emit template UUID and class type over event bus when deleting templates.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:02:17 -04:00
Rohit Yadav
0cb60a72fe CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
Restricts use of listemplates API with templatefilter=all for root admin only.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:01:15 -04:00
Rohit Yadav
566e7d9fac CLOUDSTACK-9369: Restrict default login to ldap/native users
- Restricts default login auth handler to ldap and native-cloudstack users
- Refactors and create re-usable method to find domain by id/path
- Adds unit test for refactored method in DomainManagerImpl
- Adds smoke test for login handler

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:00:05 -04:00
Will Stevens
7974bfd313 Merge release branch 4.7 to 4.8
* 4.7:
  lower the time we wait for interfaces to appear
2016-05-26 15:50:29 -04:00