9698 Commits

Author SHA1 Message Date
Rohit Yadav
abfcd5b95f CLOUDSTACK-9632: Upgrade bouncy castle to version 1.55
- Upgrades Maven dependency version to v1.55
- Fixes bountycastle usages and issues
- Adds timeout to jetty/annotation scanning
- Fixes servlet issue, uses servlet 3.1.0
- Downgrade javassist used by reflections to fix annotation process errors
- Make console-proxy-rdp bc dependency same as rest of the codebase
- Picks up PR #1510 by Daan

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-06 17:24:24 +05:30
Rohit Yadav
50f80cc2a0
Merge branch '4.8' into 4.9 2016-11-25 13:03:04 +05:30
Rohit Yadav
5811d33658 Merge pull request #1635 from myENA/feature/honor_force_stop_vm
CLOUDSTACK-9451https://issues.apache.org/jira/browse/CLOUDSTACK-9451

Re-doing against 4.8 since this is a bug, not a feature.

* pr/1635:
  CLOUDSTACK-9451

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-25 12:56:23 +05:30
John Burwell
dc1a7b4338 Merge release branch 4.8 to 4.9
* 4.8:
  CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector
2016-11-17 00:46:12 -05:00
John Burwell
293ec4f3fc Merge pull request #1673 from wido/CLOUDSTACK-9071
CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollectorBoth host and path could have been NULL which causes the StatsCollector
no to start properly.

By checking if the Strings are not Empty or Null we make sure the StatsCollector
always runs and does not prevent the Management Server from starting.

* pr/1673:
  CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector

Signed-off-by: John Burwell <meaux@cockamamy.net>
2016-11-17 00:45:21 -05:00
Rohit Yadav
240f44392c
Merge branch '4.8' into 4.9 2016-11-03 16:55:47 +05:30
Rohit Yadav
c0a6fc2bfb
Merge pull request #1714 from yvsubhash/CLOUDSTACK-9553
CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario

* pr/1714:
  CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-03 16:55:21 +05:30
Rohit Yadav
25b6990e22
server: Fix build regression caused by 32a39
A constructor signature has changed between 4.8 and 4.9+ branches which caused
failure in a unit test introduced by PR #1694. This fixes the unit test by
passing null as the additional parameter (the test does not need instantiated
object).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-03 09:14:35 +05:30
Rohit Yadav
19951346ad
Merge branch '4.8' into 4.9 2016-11-02 23:03:31 +05:30
Rohit Yadav
68f22e2a43
Merge pull request #1694 from shapeblue/kvm-no-storage-failfast
CLOUDSTACK-9509: Host Connects Without StorageKVM hosts on shared storage failure was accepted by mgmt server with the
host state as Up, even though there was no primary/shared storage available on
it. This patch offers a quick fix by throwing an exception in the storage monitor
which connects storage pool on host. The failure is trapped by agent manager
that disconnects the agent without any investigation.

Based on Lab tests, KVM agent may take upto 2 minutes to attempt NFS mount when
the storage is inaccessible (firewalled, or shutdown) before returning back with
an error. It is safe to assume that this won't add pressure on mgmt server due to
several reconnection attempts, and KVM agent would retry reconnection every 2
minutes.

For such KVM hosts, where failure happens due to storage issues; they will be
briefly put in Alert state but will be mostly be in Connecting state during which
the KVM host attempts to mount/reconfigure NFS storage pool.

/cc @jburwell @karuturi
@blueorangutan package

* pr/1694:
  CLOUDSTACK-9509: Host Connects Without Storage

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-11-02 22:57:51 +05:30
Rohit Yadav
fcee71f35b Merge branch '4.8' into 4.9 2016-10-28 11:43:16 +05:30
Marc-Aurèle Brothier
158497d68a CLOUDSTACK-9544: Check access on account trying to generate user API keys
This fixes CVE-2016-6813

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-27 22:15:49 +05:30
subhash_y
3987b8c6b4 CLOUDSTACK-9553 Usage event is not getting recorded for snapshots in a specific scenario 2016-10-24 11:47:53 +05:30
Rajani Karuturi
72612f79cd Merge release branch 4.8 to 4.9
* 4.8:
  Emit a VOLUME_DELETE usage event when account deletion destroys an instance.
2016-10-24 11:13:06 +05:30
nnesic
d989c5d8be Emit a VOLUME_DELETE usage event when account deletion destroys an instance.
Currently the logic about volume deletion seems to be that an event
should be emitted when the volume delete is requested, not when the
deletion completes.

The VolumeStateListener specifically ignores destroy events for ROOT
volumes, assuming that the ROOT volume only gets deleted when the
instance is destroyed and the UserVmManager should take care of it.

When deleting an account, all of its resources get destroyed, but the
instance expunging circumvents the UserVmManager, and thus we miss the
VOLUME_DESTROY usage event. The account manager now attempts to
propperly destroy the vm before expunging it. This way the destroy
logic is respected, including the event emission.
2016-10-21 09:26:59 +00:00
Rohit Yadav
32a397aa93 CLOUDSTACK-9509: Host Connects Without Storage
KVM hosts on shared storage failure was accepted by mgmt server with the
host state as Up, even though there was no primary/shared storage available on
it. This patch offers a quick fix by throwing an exception in the storage monitor
which connects storage pool on host. The failure is trapped by agent manager
that disconnects the agent without any investigation.

Based on Lab tests, KVM agent may take upto 2 minutes to attempt NFS mount when
the storage is inaccessible (firewalled, or shutdown) before returning back with
an error. It is safe to assume that this won't add pressure on mgmt server due to
several reconnection attempts, and KVM agent would retry reconnection every 2
minutes.

For such KVM hosts, where failure happens due to storage issues; they will be
briefly put in Alert state but will be mostly be in Connecting state during which
the KVM host attempts to mount/reconfigure NFS storage pool.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-10-21 10:22:32 +05:30
nvazquez
f4a42ab0d7 CLOUDSTACK-9535: [API] listVMSnapshots improvement 2016-10-05 17:42:14 -03:00
Wido den Hollander
c1997a1705
CLOUDSTACK-9071: Properly parse stats.output.uri in StatsCollector
Both host and path could have been NULL which causes the StatsCollector
no to start properly.

By checking if the Strings are not Empty or Null we make sure the StatsCollector
always runs and does not prevent the Management Server from starting.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

Conflicts:
	server/src/com/cloud/server/StatsCollector.java
2016-09-29 16:00:38 +02:00
Nathan Johnson
53fd4a7997 CLOUDSTACK-9451
Honor the forced parameter to stop virtual machine api call.
2016-09-09 13:00:17 -05:00
John Burwell
8d11511b1f Adds support for four position versions and optional db upgrades
Often, patch and security releases do not require schema migrations or
data migrations.  However, if an empty upgrade class and associated
scripts are not defined, the upgrade process will break.  With this
change, if a release does not have an upgrade, a noop DbUpgrade is added
to the upgrade path.  This approach allows the upgrade to proceed and
for the database to properly reflect the installed version.  This change
should make the release process simpler as RMs no longer need to
rememeber to create this boilerplate code when starting a new release.

Beginning with the 4.8.2.0 and 4.9.1.0 releases, the project will
formally adopt a four (4) position release number to properly accomodate
rekeases that contain only CVE fixes.  The DatabaseUpgradeChecker and
Version classes made assumptions that they would always parse and
compare three (3) position version numbers.  This change adds the
CloudStackVersion value object that supports both three (3) and four (4)
version numbers.   It encapsulates version comparsion logic, as well as,
the rules to allow three (3) and four (4) to interoperate.

  * Modifies DatabaseUpgradeChecker to handle derive an upgrade path for
  a version that was not explicitly specified.  It determines the
  releases the first release before it with database migrations and uses
  that list as the basis for the list for version being calculated.  A
  noop upgrade is then added to the list which causes no schema changes
  or data migrations, but will update the database to the version.
  * Adds unit tests for the upgrade path calculation logic in
  DatabaseUpgradeChecker
  * Removes dummy upgrade logic for the 4.8.2.0 introduced in previous
  versions of this patch
  * Introduces the CloudStackVersion value object which parses and
  compares three (3) and four (4) position version numbers.  This class
  is intended to replace com.cloud.maint.Version.
  * Adds the junit-dataprovider dependency -- allowing test data to be
  concisely generated separately from the execution of a test case.
  Used extensively in the CloudStackVersionTest.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-30 13:32:32 +05:30
Rohit Yadav
e49ca1ecd2 Merge pull request #1646 from shapeblue/4.9-491upgradepath
[4.9/LTS] Add upgrade path from 4.9.0 to 4.9.1, change version to 4.9.1.0-SNAPSHOTThis adds db upgrade path from 4.9.0 to 4.9.1 and fixes a typo in default user role description (CLOUDSTACK-9449)

/cc @karuturi @jburwell  -- this will cause issues when fwd-merged to master, I can do the fwd-merging if you would like to avoid fixing the conflicts yourself

@blueorangutan package

* pr/1646:
  Updating pom.xml version numbers for release 4.9.1.0-SNAPSHOT
  cloudstack: upgrade path from 4.9.0 to 4.9.1

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-23 13:48:30 +05:30
Rohit Yadav
1060dc47e5 Merge pull request #1620 from shapeblue/4.9-oobm-password-fix
oobm: simply change password transactional logic- Simplifies change password transactional logic without using pessmistic locks
- Adds a re-enter password field in the UI to valid ipmi/oobm password

* pr/1620:
  oobm: simply change password transactional logic

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-22 15:52:14 +05:30
Rohit Yadav
f13c224da1 Updating pom.xml version numbers for release 4.9.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-19 13:53:39 +05:30
Will Stevens
740bd45be6 Updating pom.xml version numbers for release 4.8.2-SNAPSHOT
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-15 10:31:09 -04:00
Will Stevens
007c9f0d3a Merge branch '4.8.1-RC20160808T1006' into 4.8 2016-08-15 10:28:33 -04:00
Will Stevens
a63db21d16 Updating pom.xml version numbers for release 4.8.1
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-08-08 10:06:21 -04:00
Rajani Karuturi
c03d035c8b Merge release branch 4.8 to 4.9
* 4.8:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:30:45 +05:30
Rajani Karuturi
a54a3b5cd5 Merge release branch 4.7 to 4.8
* 4.7:
  server: give more memory to tests
  packaging: Marvin and integration-tests packages
2016-08-05 14:29:16 +05:30
Rohit Yadav
e5750b3331 server: give more memory to tests
Increases allowed max and permgen memory flags to maven-surefire plugins.
This fixes unit test failures in cloud-server.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit fd7273b446738c0ebfae84189502dbdcd18bfd42)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-04 15:52:29 +05:30
Rohit Yadav
8ac3c883cd oobm: simply change password transactional logic
- Simplifies change password transactional logic without using pessmistic locks
- Adds a re-enter password field in the UI to valid ipmi/oobm password

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-02 12:47:15 +05:30
Will Stevens
227ff3884d Updating pom.xml version numbers for release 4.9.0
Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-07-25 16:56:04 -04:00
nvazquez
148e974482 CLOUDSTACK-9436: Release network resources on expunge command 2016-07-21 12:25:22 -03:00
Will Stevens
46a6530e70 Revert "Merge pull request #1594 from nvazquez/vmnetworkmapissue"
This reverts commit 9be93c6e9022f35a2cd3dab6a099d69a77dd72ff, reversing
changes made to 8d45d711bff631efb45954631c123dcb2beffb3c.
2016-07-21 11:04:10 -04:00
nvazquez
c754a0cf30 CLOUDSTACK-9407: Refactor 2016-06-21 13:49:55 -03:00
nvazquez
d3f3fb0590 CLOUDSTACK-9407: Release network resources on expunge command 2016-06-16 12:35:42 -03:00
Nick Livens
ffe72ca227 CLOUDSTACK-9399 : NPE during deletion of host when clusterId is null 2016-06-08 08:18:14 +02:00
Will Stevens
123e9f81ab fixed a variable name broken by merge conflicts 2016-05-27 16:32:20 -04:00
Will Stevens
75c8a55014 Merge pull request #1251 from koushik-das/CLOUDSTACK-9180
CLOUDSTACK-9180: Optimize concurrent VM deployment operation on same network

Check if VR needs to be allocated for a given network and only acquire lock if required

Refer to the bug for details.

* pr/1251:
  CLOUDSTACK-9180: Optimize concurrent VM deployment operation on same network Check if VR needs to be allocated for a given network and only acquire lock if required

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:20:49 -04:00
Will Stevens
121b3d6403 Merge pull request #1567 from exoscale/CLOUDSTACK-9238
CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VOI will update the PR to add max field length in the API commands too

* pr/1567:
  API: update url field max length
  not needed on host table
  Fix URL length to 2048 for all url fields in VO

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:20:22 -04:00
Will Stevens
20f779298d merging 4.8 -> master 2016-05-27 15:19:33 -04:00
Will Stevens
744f9d5615 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
  CLOUDSTACK-9369: Restrict default login to ldap/native users
  Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.
  Emit template UUID and class type over event bus when deleting templates.
2016-05-27 15:03:42 -04:00
Will Stevens
cd5f3320c8 Merge pull request #1564 from greenqloud/pr-emit-template-uuid-on-delete-4.7
Emit template UUID and class type over event bus when deleting templatesNew version of #1378 for the 4.7b branch instead of 4.6.

* pr/1564:
  Emit template UUID and class type over event bus when deleting templates.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:02:17 -04:00
Rohit Yadav
0cb60a72fe CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
Restricts use of listemplates API with templatefilter=all for root admin only.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:01:15 -04:00
Rohit Yadav
566e7d9fac CLOUDSTACK-9369: Restrict default login to ldap/native users
- Restricts default login auth handler to ldap and native-cloudstack users
- Refactors and create re-usable method to find domain by id/path
- Adds unit test for refactored method in DomainManagerImpl
- Adds smoke test for login handler

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:00:05 -04:00
Marc-Aurèle Brothier
a59ee03fd7 Fix URL length to 2048 for all url fields in VO 2016-05-27 08:16:05 +02:00
jeff
7aec943ef8 Emit template UUID and class type over event bus when deleting templates.
The behavior is now consistent with template creation. This commit
also adds a unit test for this functionality to make sure that it will
always happen.
2016-05-26 11:20:39 +00:00
Will Stevens
3c800b4bfe Merge pull request #1518 from nvazquez/testnfs
CLOUDSTACK-9368: Fix for Support configurable NFS version for Secondary Storage mounts## Description
JIRA TICKET: https://issues.apache.org/jira/browse/CLOUDSTACK-9368
This pull request address a problem introduced in #1361 in which NFS version couldn't be changed after hosts resources were configured on startup (for hosts using `VmwareResource`), and as host parameters didn't include `nfs.version` key, it was set `null`.

## Proposed solution
In this proposed solution `nfsVersion` would be passed in `NfsTO` through `CopyCommand` to `VmwareResource`, who will check if NFS version is still configured or not. If not, it will use the one sent in the command and will set it to its storage processor and storage handler. After those setups, it will proceed executing command.

* pr/1518:
  CLOUDSTACK-9368: Fix for Support configurable NFS version for Secondary Storage mounts

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:05:07 -04:00
Will Stevens
978184bccb Merge pull request #1424 from syed/create-template-api-bug
[CLOUDSTACK-8973] Fix create template from snapshot returning null in case of region storeThis PR fixes a case where when we create a template from a snpashot in a region wide store, we don't set the cross zone flag which causes a null response to be returned
Tests:

Before fix
```
(local)  > create template snapshotid=33aa3f3b-5a47-4d2a-8d27-12952c01ebed displaytext=t2 ostypeid=20c8ead6-d750-11e5-9f8c-06524200007c name=t9

accountid = 1b13d7c2-d750-11e5-9f8c-06524200007c
cmd = org.apache.cloudstack.api.command.admin.template.CreateTemplateCmdByAdmin
created = 2016-02-23T16:09:24+0000
jobid = 4f9f5ff9-e7f0-4af6-999c-799431fd47de
jobinstanceid = a08a9711-bd31-43bb-80a2-49cf9d722a19
jobinstancetype = Template
jobprocstatus = 0
jobresult:
null:
crossZones = False
isfeatured = False
ispublic = False
isready = False
tags:
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 1b140f08-d750-11e5-9f8c-06524200007c
```

See the *null* in response

After fix:

```
(local)  > create template snapshotid=33aa3f3b-5a47-4d2a-8d27-12952c01ebed displaytext=t2 ostypeid=20c8ead6-d750-11e5-9f8c-06524200007c name=t11

accountid = 1b13d7c2-d750-11e5-9f8c-06524200007c
cmd = org.apache.cloudstack.api.command.admin.template.CreateTemplateCmdByAdmin
created = 2016-02-25T21:47:03+0000
jobid = 1b74209b-b3c1-4168-a243-f559aa0c081b
jobinstanceid = 06ecee5a-b1f2-4e67-80fb-f0f44b0aa198
jobinstancetype = Template
jobprocstatus = 0
jobresult:
template:
id = 06ecee5a-b1f2-4e67-80fb-f0f44b0aa198
name = t11
account = admin
created = 2016-02-25T21:47:03+0000
crossZones = True
displaytext = t2
domain = ROOT
domainid = 1b13ab80-d750-11e5-9f8c-06524200007c
format = VHD
hypervisor = XenServer
isdynamicallyscalable = False
isextractable = True
isfeatured = False
ispublic = False
isready = True
ostypeid = 20c8ead6-d750-11e5-9f8c-06524200007c
ostypename = CentOS 5 (64-bit)
passwordenabled = False
size = 21474836480
sourcetemplateid = 1af0f0cc-d750-11e5-9f8c-06524200007c
sshkeyenabled = False
status = Download Complete
tags:
templatetype = USER
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 1b140f08-d750-11e5-9f8c-06524200007c
```
Works correctly

* pr/1424:
  Fix create template from snapshot returning null in case of region store

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:04:31 -04:00
Will Stevens
6a6e164448 Merge pull request #1360 from greenqloud/pr-refactor-system-vm-network-creation
Refactor system VM default network creationTwo small commits which moves the retrieval of the default network for the console proxy and the SSVM into a separate protected method. It's a small change that makes the code more readable/maintainable and also makes the class more suitable for overriding should one want to do this. It's forward-ported from our 4.2 branch.

No new tests since this should not change any functionality, and thus should be covered by the existing unit tests.

Now on the master branch (#1359 was on the wrong branch).

* pr/1360:
  Refactor ssvm default network retrieval.
  Refactor console proxy default network retrieval.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:03:32 -04:00
Will Stevens
b5606dcce8 Merge pull request #1553 from nlivens/mysql_driver_issue
Dynamically load drivers before creating our DB connectionsSolution to the mailing thread titled "MySQL : No suitable driver found for jdbc:mysql".
It doesn't harm that we explicitely load the MySQL driver, and for those which would use a commons-dbcp version < 1.4 this would fix it as well. Since JDBC 4.0, the JDBC driver can auto-register itself, but for some weird cases (like ours), it's not working. Therefore we need to explicitly load the JDBC driver.

* pr/1553:
  Dynamic loading of DB driver + support for other DB providers

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:00:42 -04:00