29953 Commits

Author SHA1 Message Date
Rohit Yadav
b87eda230a cloudstack: upgrade path from 4.9.0 to 4.9.1
- Adds db upgrade path from 4.9.0 to 4.9.1
- CLOUDSTACK-9449: Fix typo in default user role description

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:53:15 +05:30
Rohit Yadav
7a53feee22 marvin: fix codegeneration against API discovery endpoint
This makes the commands.xml based codegeneration equivalent to the
API discovery end point based discovery.

This fixes the fields that the (api discovery based) codegenerator should
produce in the generated python classes (cmd and response classes per
api/module). The issue was that the autogenerated cloudstackAPI differed between
api-based and apidocs-based code generation. With this fix the generated classes
match exactly thereby allowing us to go with either methods to generate
cloudstackAPI.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:43:48 +05:30
Abhinandan Prateek
a2fcc65d90 CLOUDSTACK-9459: the try's catch block was shortening the preparedstatement life
resulting in bad resultset when used outside of try catch.
2016-08-16 14:19:07 +05:30
Rajani Karuturi
d3e45515ee Merge pull request #1628 from karuturi/patch-1
updated contributing.mdAs this is the doc that is showed while creating a pull request, updated it with information from release principles wiki.

* pr/1628:
  updated contributin.md

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-16 13:54:24 +05:30
Will Stevens
740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens
007c9f0d3a Merge branch '4.8.1-RC20160808T1006' into 4.8 2016-08-15 10:28:33 -04:00
Rohit Yadav
7b8ba24c64 CLOUDSTACK-9452: add python-argparse dependency on el6,7 rpms
The patchviasocket script was rewritten in Python from PR #1533 and made
assumptions that Python 2.7 would be available. In case of CentOS, python 2.7
may not be available or installed. This change ensures that python-argparse
is installed which is used by this script.

Expose cmd error in the logs when patch command fails.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-10 16:25:17 +05:30
Rajani Karuturi
56a3526502 Merge release branch 4.9 to master
* 4.9:
  CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso
2016-08-09 15:51:15 +05:30
Rajani Karuturi
bdc409c7a2 Merge pull request #1626 from shapeblue/systemvmtemplate-4dot9
[blocker] Fix systemvm template buildPrevious PR: https://github.com/apache/cloudstack/pull/1531

Fixes failing systemvmtemplate build.

* pr/1626:
  CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-09 15:49:04 +05:30
Pierre-Luc Dion
3cc3aea449 Add to project detail page: cpu,memory,template,storage and VMs count 2016-08-08 20:04:55 -04:00
Will Stevens
a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
4.8.1
2016-08-08 10:06:21 -04:00
Rajani Karuturi
3afedbb24f Merge pull request #1631 from karuturi/merge-49-to-master-deb-build
Fix debian build error due to commit 3315eb5debian build is broken due to the changelog changes in commit 3315eb5. Fixed the same.
To test, do a debian following the instructions at http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.8/building_from_source.html#building-deb-packages

cc: @pdion891 @rhtyd Please review.

* pr/1631:
  Fix debian build error due to commit 3315eb5420e1b2cedf906aeb709740efc2f08cfd

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-08 16:47:09 +05:30
Rajani Karuturi
d87828af24 Fix debian build error due to commit 3315eb5420e1b2cedf906aeb709740efc2f08cfd 2016-08-08 15:02:03 +05:30
Pierre-Luc Dion
98228c5fb1 add projectid to project details page 2016-08-06 14:38:20 -04:00
Rajani Karuturi
4c387feebb updated contributin.md
As this is the doc that is showed while creating a pull request, updated it with information from release principles wiki.
2016-08-05 15:06:52 +05:30
Rajani Karuturi
2875af7112 Merge release branch 4.9 to master
* 4.9:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:41:31 +05:30
Rajani Karuturi
3315eb5420 Merge release branch 4.9 to master
* 4.9:
  Updating pom.xml version numbers for release 4.9.0
2016-08-05 14:38:09 +05:30
Rajani Karuturi
c03d035c8b Merge release branch 4.8 to 4.9
* 4.8:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:30:45 +05:30
Rajani Karuturi
a54a3b5cd5 Merge release branch 4.7 to 4.8
* 4.7:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:29:16 +05:30
Rohit Yadav
2cddaf3d36 CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso
- Update base debian iso to version 7.11
- Upgrade ruby version to 2.3.0 (latest/stable)
- Fix Gemfile
- Update README
- Fix openswan pkg name with the same version
- Remove cloud-cleanup it's not available

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-05 12:40:07 +05:30
Rajani Karuturi
9127af61e4 Merge pull request #1612 from shapeblue/package-marvin-tests
packaging: Marvin and integration-tests packagesThis introduces two new cloudstack packages: marvin and integration-tests.
The two packages will make it easier for CI systems to install Marvin for a
specific cloudstack release/build and run integration tests that are specific
for that version/build.

Since Marvin may have its own dependencies, we're bundling the Marvin source tarball and installing it with pip in the post-installation script/phase.

This also fixed a build issue with using juniper-api maven repo.

/cc @swill @PaulAngus @jburwell @wido @bvbharat

* pr/1612:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-05 11:55:48 +05:30
Rajani Karuturi
b607a3900d Merge pull request #1625 from shapeblue/dbupgrade-fix-4.10
[blocker] cloudstack: fix upgrade paths to 4.10.0This adds an upgrade to master since the recent 4.9 cut, adds an upgrade path to the new code version 4.10. Without this PR all PRs to master are failing due to missing upgrade path. Please merge this on priority.

Code review and Travis results should be enough to validate the changes.

/cc @karuturi @wido @ProjectMoon @abhinandanprateek

* pr/1625:
  server: give more memory to tests
  cloudstack: fix upgrade paths to 4.10.0

Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
2016-08-04 16:40:05 +05:30
Rohit Yadav
e5750b3331 server: give more memory to tests
Increases allowed max and permgen memory flags to maven-surefire plugins.
This fixes unit test failures in cloud-server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit fd7273b446738c0ebfae84189502dbdcd18bfd42)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-04 15:52:29 +05:30
Rohit Yadav
fd7273b446 server: give more memory to tests
Increases allowed max and permgen memory flags to maven-surefire plugins.
This fixes unit test failures in cloud-server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-04 15:31:51 +05:30
Rohit Yadav
546a3f8884 cloudstack: fix upgrade paths to 4.10.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-04 13:03:46 +05:30
Rohit Yadav
8ac3c883cd oobm: simply change password transactional logic
- Simplifies change password transactional logic without using pessmistic locks
- Adds a re-enter password field in the UI to valid ipmi/oobm password

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-02 12:47:15 +05:30
Will Stevens
62aa3b2bfa Updating pom.xml version numbers for release 4.10.0-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-29 10:11:34 -04:00
Will Stevens
227ff3884d Updating pom.xml version numbers for release 4.9.0
Signed-off-by: Will Stevens <williamstevens@gmail.com>
4.9.0
2016-07-25 16:56:04 -04:00
Will Stevens
d302269fe5 CLOUDSTACK-9437: Create egress chain on upgrade and cleanup for allow all traffic
- Ensure that FW_EGRESS_RULE chain exists after upgrading the router
- Flush allow all egress rule on 0.0.0.0/0, if such a rule exists in the config
  it will be added later (CLOUDSTACK-9437)
2016-07-25 16:44:38 -04:00
Will Stevens
818063c8ee Merge pull request #1616 from pdube/4.9-vr-iptables-fixes
Added missing rules on router config, fixed ordering of multiple rules, removed duplicate rules, added fix for network stats, added a check for b64 decoding (to pad incorrect b64). Also added a catch exception to be logged on the configure main.

https://issues.apache.org/jira/browse/CLOUDSTACK-9430
https://issues.apache.org/jira/browse/CLOUDSTACK-9431
https://issues.apache.org/jira/browse/CLOUDSTACK-9435
https://issues.apache.org/jira/browse/CLOUDSTACK-9440

* pr/1616:
  Added missing rules on router config, fixed ordering of multiple rules, removed duplicate rules, added fix for network stats, added a check for b64 decoding (to pad incorrect b64). Also added a catch exception to be logged on the configure main.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-25 16:37:38 -04:00
Will Stevens
0a0839ea2d Merge pull request #1613 from nvazquez/vmnetworkmapissue
CLOUDSTACK-9436: vm_network_map table cleanup, release network resources on expunge commandJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9436

This PR replaces #1594

Due to error on `test/integration/smoke/test_vpc_redundant.py` it was found out that `vm_network_map` table should be less aggresive on vm stop

* pr/1613:
  CLOUDSTACK-9436: Release network resources on expunge command

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-25 16:37:18 -04:00
Patrick Dube
9ab676206a Added missing rules on router config, fixed ordering of multiple rules, removed duplicate rules, added fix for network stats, added a check for b64 decoding (to pad incorrect b64). Also added a catch exception to be logged on the configure main. 2016-07-22 15:32:20 -04:00
nvazquez
148e974482 CLOUDSTACK-9436: Release network resources on expunge command 2016-07-21 12:25:22 -03:00
Will Stevens
46a6530e70 Revert "Merge pull request #1594 from nvazquez/vmnetworkmapissue"
This reverts commit 9be93c6e9022f35a2cd3dab6a099d69a77dd72ff, reversing
changes made to 8d45d711bff631efb45954631c123dcb2beffb3c.
2016-07-21 11:04:10 -04:00
Rohit Yadav
de041df74d packaging: Marvin and integration-tests packages
This introduces two new cloudstack packages: marvin and integration-tests.
The two packages will make it easier for CI systems to install Marvin for a
specific cloudstack release/build and run integration tests that are specific
for that version/build.

- maven: add explicit juniper-contrail-api maven repository
- marvin: build source distribution for both install and package mvn phases

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-07-20 16:31:23 +05:30
Will Stevens
9be93c6e90 Merge pull request #1594 from nvazquez/vmnetworkmapissue
CLOUDSTACK-9407: vm_network_map table doesnt get cleaned up properlyJIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9407

### Introduction
It was found out that in production environments `vm_network_map` table entries were slowly growing. It was investigated how this entries were cleaned up.

### Behaviour
On vm creation, vm mappings are inserted on `vm_network_map`.
On vm stop, mappings are deleted from `vm_network_map` for vm, as a result of the release of its nics.

### Problem
If created vm is stopped from hypervisor side (at least on vSphere in which we tested it), when CloudStack realizes vm is stopped it doesn't clean up `vm_network_table,` and, as cleanup is made during vm stop, when vm is eventually destroyed and expunged it won't clean up their entries in that table.

### Proposed solution
We propose to move `vm_network_map` table cleanup to expunge command instead of stop command.

* pr/1594:
  CLOUDSTACK-9407: Refactor
  CLOUDSTACK-9407: Release network resources on expunge command

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-19 08:18:18 -04:00
Will Stevens
8d45d711bf Merge pull request #1583 from milamberspace/L10N-update-Master-20160607
Update L10N resource files with 4.9 strings from Transifex (20160607)cc @swill before the 4.9 release. Just only the latest FR translation.

* pr/1583:
  Update L10N resource files with 4.9 strings from Transifex (20160709)

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-19 08:17:40 -04:00
Will Stevens
54039f9497 Merge pull request #1610 from wido/db-properties-mysql-driver
packaging: Add db.X.driver=jdbc:mysql to db.properties on upgradeThis is required afther the upgrade to 4.9.0 and for convience we
add this to the configuration so our users do not have to.

* pr/1610:
  packaging: Add db.X.driver=jdbc:mysql to db.properties on upgrade

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:12:03 -04:00
Will Stevens
a566cde145 Merge pull request #1609 from pdube/network-acl-add-order
[CLOUDSTACK-9430] Added fix for adding/editing Network ACL rule orderingBUG: https://issues.apache.org/jira/browse/CLOUDSTACK-9430

The issue occurred because all of the ACL rules get inserted before the old ones. Then, the cleanup deletes the duplicate rows, and leaves any new rule in front of the old ones.

Here is an example with a simplified iptables view for ACL
Ex: adding a rule 4
before add:
1,2,3

during add:
1',2',3',4',1,2,3

after add:
4',1,2,3

After fix:
before add:
1,2,3

during add:
1,2,3,1',2',3',4'

after add:
1',2',3',4'

* pr/1609:
  Added fix for adding/editing Network ACL rule ordering

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:11:13 -04:00
Will Stevens
bb9d94bc64 Merge pull request #1608 from myENA/upstream/context-cleanup
Cleanup RBD contexts after exceptions to prevent potential agent crashWe noticed that when an exception occurs within the cleanup loop inside
the deletePhysicalDisk routine that the previously allocated contexts
are not cleaned up.  This seemed to cause an eventual crash of the host
agent after multiple exceptions within the loop.

In addition to ensuring the contexts are always freed we also improved
the logging when exceptions do occur to include the actual return code
from the underlying library in deletePhysicalDisk and deleteSnapshot.

* pr/1608:
  improve logging readability
  Cleanup rbd contexts and improve exception logging

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:10:37 -04:00
Will Stevens
ea48e95bdd Merge pull request #1601 from shapeblue/nio-aggressive-selector
CLOUDSTACK-9348: Reduce Nio selector wait timeThis reduced the Nio loop selector wait time, this way the selector will
check frequently (as much as 100ms per iteration) and handle any pending
connection/tasks. This would make reconnections very quick at the expense of
some CPU usage.

/cc @swill @kiwiflyer guys can you please apply this fix in your env and test if you're still able to produce any Nio related error b/w mgmt server(s) and kvm agent(s) not being able to connect quickly. Please also watch out for any increased CPU usage (there should not be any significant change), in which case we may increase the timeout from 100ms to 200-400ms.

* pr/1601:
  CLOUDSTACK-9348: Reduce Nio selector wait time

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:10:06 -04:00
Will Stevens
a38279de6a Merge pull request #1595 from shapeblue/ui-resize-show-allusers
ui: show resize volume button to all users![screenshot from 2016-06-23 12-29-56](https://cloud.githubusercontent.com/assets/95203/16294438/436f6dbc-393e-11e6-91b5-cb2e49a01cc6.png)

The resize volume is support on all major hypervisors (Xen, VMware, KVM).
The hypervisor key is returned by the list volumes response only for admins
but not for users or domain admin users. This removes the check, as the operation
is supported on all major hypervisors that CloudStack supports.

With this bug fix all users would see resize volume button in the UI.

/cc @swill

* pr/1595:
  ui: show resize volume button to all users

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:09:18 -04:00
Will Stevens
c806f3d8b2 Merge pull request #1455 from sanju1010/vlan
[CLOUDSTACK-9328]: Fix vlan issues from test suite test_privategw_acl.py in BVTPlease refer to CLOUDSTACK-9328 for the details.

Test Results:
==========
test_01_vpc_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_01_vpc_privategw_acl | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 108.273s

OK

* pr/1455:
  [CLOUDSTACK-9328]: Fix vlan issues from test  suite test_privategw_acl.py in BVT Bug-Id:## CLOUDSTACK-9328

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:08:39 -04:00
Will Stevens
49df7f283c Merge release branch 4.8 to master
* 4.8:
  CLOUDSTACK-9342: Site to Site VPN PFS not being set correctly
2016-07-18 14:07:09 -04:00
Will Stevens
01758ee867 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9342: Site to Site VPN PFS not being set correctly
2016-07-18 14:06:08 -04:00
Will Stevens
adaf410109 Merge pull request #1480 from Slair1/S2S-VPN-PFS-Setting
CLOUDSTACK-9342: Site to Site VPN PFS not being set correctlyBug in code set PFS to the same value (yes/no) as DPD.

file.addeq(" pfs=%s" % CsHelper.bool_to_yn(obj['dpd']))

* pr/1480:
  CLOUDSTACK-9342: Site to Site VPN PFS not being set correctly

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-18 14:05:18 -04:00
Wido den Hollander
343ea6098e
packaging: Add db.X.driver=jdbc:mysql to db.properties on upgrade
This is required afther the upgrade to 4.9.0 and for convience we
add this to the configuration so our users do not have to.
2016-07-12 10:13:23 +02:00
Patrick Dube
6dd6ef0c9a Added fix for adding/editing Network ACL rule ordering 2016-07-11 15:12:41 -04:00
Aaron Hurt
c8fce3ff31 improve logging readability 2016-07-11 12:05:06 -05:00
Milamber
fed26bd5c0 Update L10N resource files with 4.9 strings from Transifex (20160709) 2016-07-09 10:46:42 +00:00