9721 Commits

Author SHA1 Message Date
Will Stevens
7e41747c33 Merge pull request #816 from mike-tutkowski/addremovehosts2
Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster

This PR addresses the following JIRA ticket:

https://issues.apache.org/jira/browse/CLOUDSTACK-8813

The problem is that there needs to be notifications sent when a host is added to, about to be removed from, and removed from a cluster.

Such notifications can be used for many purposes. For example, it can allow storage plug-ins to update ACLs on their storage systems. Also, it can allow us to clean up IQNs from ESXi hosts that are no longer needed.

* pr/816:
  CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:07:36 -04:00
Will Stevens
143265c49f Merge pull request #1502 from shapeblue/outofband-master
CLOUDSTACK-9299: Out-of-band Management for CloudStackSupport access to a hosts out-of-band management interface (e.g. IPMI, iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and iDRA,
the service implementation allows for development of separate drivers as plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature, please install `ipmitool` (using yum/apt/brew) and `ipmisim`:
https://pypi.python.org/pypi/ipmisim

The default ipmitool location is assumed in /usr/bin, if this is different in your env please fix the global setting, see FS for details on various global settings.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

/cc @jburwell @swill @abhinandanprateek @murali-reddy @borisstoyanov

* pr/1502:
  maven: ignore utils/testsmallfileinactive for rat checking
  CLOUDSTACK-9378: Fix for #1497
  HypervisorUtilsTest: increate timeout to 8seconds
  travis: Use patched version of ipmitool for tests
  CLOUDSTACK-9299: Out-of-band Management for CloudStack

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:06:13 -04:00
Will Stevens
103d62ee02 Merge release branch 4.8 to master
* 4.8:
  CLOUDSTACK-9287 - Improve test by checking if pvt gw is removed and fix typos
  Handle private gateways more reliably
  CLOUDSTACK-9287 - Fix RVR public interface
  CLOUDSTACK-9287 - Add integration test to cover the private gateway related changes
  CLOUDSTACK-9287 - Refactor the interface state configuration
  CLOUDSTACK-9287 - Check if the nic profile has already been removed from a certain router
  CLOUDSTACK-9287 - Bring up the private gw interface on state change to master
  CLOUDSTACK-9287 - Make sure private gw interface is not used for default gw
  CLOUDSTACK-9287 - Add integration test to cover the private gw interface/mac address issues
  CLOUDSTACK-9287 - Put private gateway interface down on backup router
  CLOUDSTACK-9287 - Generate new mac address if router is redundant and nic profile exists
  Add private gateway IP to router initialization config
  apply static routes on change to master state
2016-05-12 11:04:53 -04:00
Will Stevens
dbbc37ecc7 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9287 - Improve test by checking if pvt gw is removed and fix typos
  Handle private gateways more reliably
  CLOUDSTACK-9287 - Fix RVR public interface
  CLOUDSTACK-9287 - Add integration test to cover the private gateway related changes
  CLOUDSTACK-9287 - Refactor the interface state configuration
  CLOUDSTACK-9287 - Check if the nic profile has already been removed from a certain router
  CLOUDSTACK-9287 - Bring up the private gw interface on state change to master
  CLOUDSTACK-9287 - Make sure private gw interface is not used for default gw
  CLOUDSTACK-9287 - Add integration test to cover the private gw interface/mac address issues
  CLOUDSTACK-9287 - Put private gateway interface down on backup router
  CLOUDSTACK-9287 - Generate new mac address if router is redundant and nic profile exists
  Add private gateway IP to router initialization config
  apply static routes on change to master state
2016-05-12 11:03:57 -04:00
Will Stevens
3fab75772f Merge pull request #1474 from remibergsma/47_private_gw_initial_config
Handle private gateways more reliablyWhen initialising a VPC router we need to know which IP/device corresponds to a private gateway. This is to solve a problem when stop/starting a VPC router (which gets the private gateway config as a guest network and as a result breaks the functionality). You read it right, the private gateway is sent as type=guest after reboot and type=public initially.

Before this change, you could add a private gw to a running router but you couldn't restart it (it would mix up the tiers). Now the private gateway is detected properly and it works just fine.

Booting without private gateway:
```
root@r-167-VM:~# cat /etc/cloudstack/cmdline.json
{
    "config": {
        "baremetalnotificationapikey": "V2l1u3wKJVan01h8kq63-5Y5Ia3VLEW1v_Z6i-31QIRJXlt5vkqaqf6DVcdK0jP3u79SW6X9pqJSLSwQP2c2Rw",
        "baremetalnotificationsecuritykey": "OXI16srCrxFBi-xOtEwcYqwLlMfSFTlTg66YHtXBBqR7HNN1us3HP5zWOKxfVmz4a3C1kUNLPrUH13gNmZlu4w",
        "disable_rp_filter": "true",
        "dns1": "8.8.8.8",
        "domain": "cs2cloud",
        "eth0ip": "169.254.0.42",
        "eth0mask": "255.255.0.0",
        "host": "192.168.22.61",
        "name": "r-167-VM",
        "port": "8080",
        "privategateway": "None",
        "redundant_router": "false",
        "template": "domP",
        "type": "vpcrouter",
        "vpccidr": "10.0.0.0/24"
    },
    "id": "cmdline"
```

Booting with private gateway:
```
root@r-167-VM:~# cat /etc/cloudstack/cmdline.json
{
    "config": {
        "baremetalnotificationapikey": "V2l1u3wKJVan01h8kq63-5Y5Ia3VLEW1v_Z6i-31QIRJXlt5vkqaqf6DVcdK0jP3u79SW6X9pqJSLSwQP2c2Rw",
        "baremetalnotificationsecuritykey": "OXI16srCrxFBi-xOtEwcYqwLlMfSFTlTg66YHtXBBqR7HNN1us3HP5zWOKxfVmz4a3C1kUNLPrUH13gNmZlu4w",
        "disable_rp_filter": "true",
        "dns1": "8.8.8.8",
        "domain": "cs2cloud",
        "eth0ip": "169.254.2.227",
        "eth0mask": "255.255.0.0",
        "host": "192.168.22.61",
        "name": "r-167-VM",
        "port": "8080",
        "privategateway": "10.201.10.1",
        "redundant_router": "false",
        "template": "domP",
        "type": "vpcrouter",
        "vpccidr": "10.0.0.0/24"
    },
    "id": "cmdline"
```

And:
```
cat cmdline
vpccidr=10.0.0.0/24 domain=cs2cloud dns1=8.8.8.8 privategateway=10.201.10.1 template=domP name=r-167-VM eth0ip=169.254.2.227 eth0mask=255.255.0.0 type=vpcrouter disable_rp_filter=true baremetalnotificationsecuritykey=OXI16srCrxFBi-xOtEwcYqwLlMfSFTlTg66YHtXBBqR7HNN1us3HP5zWOKxfVmz4a3C1kUNLPrUH13gNmZlu4w baremetalnotificationapikey=V2l1u3wKJVan01h8kq63-5Y5Ia3VLEW1v_Z6i-31QIRJXlt5vkqaqf6DVcdK0jP3u79SW6X9pqJSLSwQP2c2Rw host=192.168.22.61 port=8080
```

Logs:
```
2016-02-24 20:08:45,723 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl] (Work-Job-Executor-4:ctx-458d4c52 job-1402/job-1403 ctx-d5355fca) (logid:5772906c) Set privategateway field in cmd_line.json to 10.201.10.1
```

* pr/1474:
  Handle private gateways more reliably
  Add private gateway IP to router initialization config

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:02:55 -04:00
Will Stevens
309a60ea71 Merge pull request #1483 from remibergsma/pr1413-wilder-47
CLOUDSTACK-9287 - Fix unique mac address per rVPC routerThis is work by @wilderrodrigues, see PR #1413 It contains important fixes and I think it needs to be included so I send the PR again.

* pr/1483:
  CLOUDSTACK-9287 - Improve test by checking if pvt gw is removed and fix typos
  CLOUDSTACK-9287 - Fix RVR public interface
  CLOUDSTACK-9287 - Add integration test to cover the private gateway related changes
  CLOUDSTACK-9287 - Refactor the interface state configuration
  CLOUDSTACK-9287 - Check if the nic profile has already been removed from a certain router
  CLOUDSTACK-9287 - Bring up the private gw interface on state change to master
  CLOUDSTACK-9287 - Make sure private gw interface is not used for default gw
  CLOUDSTACK-9287 - Add integration test to cover the private gw interface/mac address issues
  CLOUDSTACK-9287 - Put private gateway interface down on backup router
  CLOUDSTACK-9287 - Generate new mac address if router is redundant and nic profile exists

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:00:40 -04:00
Paul Angus
11e3582fdf Remove extraneous log directory and add catalina.out log rotation 2016-05-12 08:41:39 +01:00
Rohit Yadav
07564469e9 CLOUDSTACK-9299: Out-of-band Management for CloudStack
Support access to a host’s out-of-band management interface (e.g. IPMI, iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and iDRA,
the service implementation allows for development of separate drivers as plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature `ipmisim` can be used:
https://pypi.python.org/pypi/ipmisim

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-12 11:05:40 +05:30
Mike Tutkowski
dad9e5d868 CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster 2016-05-11 08:02:46 -06:00
Nick Livens
71c9c90e0c CLOUDSTACK-9365 : updateVirtualMachine with userdata should not error when a VM is attached to multiple networks from which one or more doesn't support userdata 2016-05-11 07:59:38 +02:00
Rohit Yadav
003b97bcff maven: Fix jstl version usage
This would fix regression from recent mvn version changes. Without this
patch users get redirected to error.jsp as jstl-1.2 jar is not installed

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:58:27 +05:30
Rohit Yadav
afcbd270b2 CLOUDSTACK-8562: Deprecate commands.properties
- Removes commands.properties file
- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Rohit Yadav
4347776ac6 CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.

A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack

DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
  current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
  Admin, Domain Admin and User) which maintains this association by requiring
  all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
  removal of roles and/or modifications of permissions, without the need
  of restarting management server(s)

Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
  deployments will continue to use the older static role based api access checker
  with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
  roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
  to add existing set of permissions to the default roles. cloud.account
  will have a new role_id column which will be populated based on default roles
  as well

Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Rohit Yadav
95abb6efc3 CLOUDSTACK-9361: Centrally handle API validations
Validate API arguments based on annotations. Introduces:
- NotNullOrEmpty: for doing null and empty string checks
- PositiveNumber: number > 0 (natural number)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:40:22 +05:30
Will Stevens
456680dbb2 Merge pull request #880 from SudharmaJain/cs-8901
CLOUDSTACK-8901: PrepareTemplate job thread hard-coded to max 8 threads The thread pool was hardcoded to use 8 threads,
com.cloud.template.TemplateManagerImpl.configure(String, Map<String, Object>):
_preloadExecutor = Executors.newFixedThreadPool(8, new NamedThreadFactory("Template-Preloader"));

Added the change to pick threadpool size from configuration.

* pr/880:
  CLOUDSTACK-8901: PrepareTemplate job thread hard-coded to max 8 threads

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-11 00:08:45 -04:00
Will Stevens
de582a41c0 Merge pull request #1497 from nvazquez/addidsparam
CLOUDSTACK-9351: Add ids parameter to resource listing API calls## General behaviour
A new parameter is added in each method, its type a list of IDs of the entity, it will be mutually exclusive with id. (Similar to <code>id</code> and <code>ids</code> parameters in <code>listVirtualMachines</code> method)

### API Methods affected
* <code>listTemplates</code>: new parameter **<code>ids</code>**, mutually exclusive with <code>id</code>
* <code>listVolumes</code>: new parameter **<code>ids</code>**, mutually exclusive with <code>id</code>
* <code>listSnapshots</code>: new parameter **<code>ids</code>**, mutually exclusive with <code>id</code>
* <code>listVMSnapshots</code>: new parameter **<code>vmsnapshotids</code>**, mutually exclusive with <code>vmsnapshotid</code>

* pr/1497:
  CLOUDSTACK-9351: Add marvin test and add it to travis file
  CLOUDSTACK-9351: Add ids parameter to resource listing API calls

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-10 23:49:55 -04:00
Will Stevens
fa3bce5a83 Merge pull request #1496 from shapeblue/kvm-ha
CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage- KVM-HA- Fix CheckOnHost for Local storage
 - Also skip HA on VMs that are using local storage

* pr/1496:
  CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage     - Also skip HA on VMs that are using local storage

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-10 23:49:04 -04:00
Will Stevens
8a3fd10615 Merge pull request #1124 from rafaelweingartner/CID-1338387
CID-1338387: Deletion of method endPointSelector.selectHypervisorHostFollowing the discussions and analysis presented on PR #1056 create by @DaanHoogland
This PR is intended to push those changes that were discussed there regarding the of endPointSelector.selectHypervisorHost method.

* pr/1124:
  Deletion of method endPointSelector.selectHypervisorHost

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-10 23:47:50 -04:00
Syed
7b5d5648d8 Fix create template from snapshot returning null in case of region store 2016-05-09 15:59:23 -04:00
weingartner
417d9a5d80 Removed unnecessary check when creating the “userVmResponse” object. 2016-05-06 15:16:37 -03:00
nvazquez
baa8610061 CLOUDSTACK-9351: Add ids parameter to resource listing API calls 2016-05-06 14:42:42 -03:00
Olivier Lemasle
61d2692597 CLOUDSTACK-9358: StringIndexOutOfBoundsException on events
Fixes JSON deserialization of cmdInfo (current process fails with
StringIndexOutOfBoundsException when cmdEventType is the last parameter
in the JSON string.
2016-05-04 01:15:03 +02:00
Rohit Yadav
4d57ec04ac Merge branch '4.8'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-03 08:22:37 +05:30
Rohit Yadav
126a039eb8 Merge branch '4.7' into 4.8 2016-05-03 08:20:27 +05:30
Will Stevens
62d9f444e3 Merge pull request #1321 from nitin-maharana/CloudStack-Nitin5_4.7
CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM idWhen calling listServiceOfferings with VM id as parameter. It is returning incompatible tagged offerings. It should only list all compatible tagged offerings. Compatible means the new service offering should contain all the tags of the existing service offering(Existing offering SUBSET of new offering). If that is the case It should list in the result and can be upgraded to that offering.

* pr/1321:
  CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM id

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-02 16:59:57 -04:00
Will Stevens
9654c990c8 Merge pull request #1365 from shapeblue/4.7-vmware-diskchain
[4.7] vmware: improve support for disks- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info

* pr/1365:
  vmware: improve support for disks

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-02 16:58:41 -04:00
Maneesha.P
7ad3c5e834 CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware. 2016-04-29 14:06:49 -03:00
Abhinandan Prateek
3b89cbe733 CLOUDSTACK-9350: KVM-HA- Fix CheckOnHost for Local storage
- Also skip HA on VMs that are using local storage
2016-04-29 12:38:33 +05:30
Will Stevens
12d8e53dd5 Merge pull request #1454 from shapeblue/host-maint
CLOUDSTACK-9323: Fix cancel host maintenance canFix cancel host maintenance so that if maintenance is cancelled the host come back to normal state gracefully.

Added marvin tests for host maintennace.

* pr/1454:
  CLOUDSTACK-9323: Fix Cancel maintenance so that if maintenance is cancelled the host come back to normal state gracefully. Added marvin tests for host maintennace.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-28 16:05:10 -04:00
Will Stevens
a5ee4432e7 Merge release branch 4.8 to master
* 4.8:
  Removed sleeps and used validateList as requested.
  Added required_hardware="false" attr above test_02_root_volume_attach_detach
  Modified test_volumes.py to include a hypervisor test for root attach/detach testing
  Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
2016-04-28 16:04:20 -04:00
Will Stevens
d11194abec Merge release branch 4.7 to 4.8
* 4.7:
  Removed sleeps and used validateList as requested.
  Added required_hardware="false" attr above test_02_root_volume_attach_detach
  Modified test_volumes.py to include a hypervisor test for root attach/detach testing
  Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
2016-04-28 16:03:46 -04:00
Will Stevens
1979f42b2d Merge pull request #1500 from myENA/KVM_root_detach
CLOUDSTACK-9349: Enable root disk detach for KVM with new Marvin testsThis PR addresses the KVM detach/attach ROOT disks from VMs (CLOUDSTACK-9349).  In short, this allows the KVM Hypervisor, and I added the Simulator as a valid hypervisor for ease of development and testing of marvin, to detach a root volume and the reattach a root volume using the deviceid=0 flag to the attachVolume API.  I have also written a marvin integration test that verifies this feature works for both KVM and the Simulator.

Below is the marvin results files of the full marvin test_volumes.py.  All tests pass, including the new root detach/attach, on our KVM lab running with the patches in this PR.

[test_volumes_KIR4G3.zip](https://github.com/apache/cloudstack/files/223799/test_volumes_KIR4G3.zip)

* pr/1500:
  Removed sleeps and used validateList as requested.
  Added required_hardware="false" attr above test_02_root_volume_attach_detach
  Modified test_volumes.py to include a hypervisor test for root attach/detach testing
  Let hypervisor type KVM and Simulator detach root volumes. Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-28 16:02:54 -04:00
Will Stevens
91ce3b9ffb Merge pull request #1522 from myENA/logjobid_master
Log asynchronous responses in the api logCurrently API responses for synchronous calls are logged, but asynchronous call responses are not.  This pull request makes a minor modification that logs the response including the JobId of all asynchronous requests.

As an example, here is what a stopVirtualMachine request looked like in the logs:

    2016-04-27 10:43:11,084 INFO  [a.c.c.a.ApiServer] (catalina-exec-3:ctx-37d9f693 ctx-d2368de3) (logid:3a0fad97) (userId=2 accountId=2 sessionId=AF8B1F726ACB5C3A637B8B300AA218A7) 10.103.0.207 -- GET command=stopVirtualMachine&id=f63b6fcc-e0b0-480f-8f7a-cba329634ba1&forced=false&response=json&_=1461771791036 200

After this modification, here is what the logs look like:

    2016-04-27 13:37:11,338 INFO  [a.c.c.a.ApiServer] (catalina-exec-6:ctx-915b5c84 ctx-a03152fa) (logid:66249df0) (userId=2 accountId=2 sessionId=9EF127EED5CA6E74797DFE487D980FAF) 10.103.0.207 -- GET command=stopVirtualMachine&id=f63b6fcc-e0b0-480f-8f7a-cba329634ba1&forced=false&response=json&_=1461782231194 200 {"stopvirtualmachineresponse":{"jobid":"5b9f4a9b-eabe-4fa4-849d-3d004bb65634"}}

* pr/1522:
  Log responses from asynchronous api commands

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-28 15:59:57 -04:00
Will Stevens
4db5e48f80 Merge pull request #1510 from shapeblue/4.9-mvn-version-safeupgradeonly
4.9 mvn version safeupgradeonlyUpgrades maven dependencies versions that can be safely upgraded without breaking console-proxy/crypto usage.

Bisected changes from: https://github.com/apache/cloudstack/pull/1397

cc @swill @DaanHoogland

* pr/1510:
  maven: fix dependency version support by JDK7
  further maven dependency updates from Daan
  framework/quota: fix checkstyle issue
  maven: Upgrade dependency versions

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-28 15:59:17 -04:00
Nathan Johnson
7778e24bf7 Log responses from asynchronous api commands
This is mainly useful so that we can see the job ids in the logs
whenever the job is created via the API.
2016-04-27 19:43:57 -05:00
Will Stevens
5c1ad900a2 Merge pull request #826 from kansal/CLOUDSTACK-8853
Fixed: Error given when creating VPN user in one network if VR for another network is stopped.

Problem: Adding a VPN user to an IP on an isolated network fails with an exception if the end-user has two isolated networks and one of them is not in use (i.e. VR is stopped).

Fix: Presently in the code, all the VR which are not in Running state are forced to throw error. Added a check for the Stopped and Stopping state routers. Configurations will be applied to them when they get restarted.

* pr/826:
  Fixed Coverity Issue: Unintentional Integer FLow
  Fixed: Error given when creating VPN user in one network if VR for another network is stopped

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-26 13:10:30 -04:00
Will Stevens
9b525f51a2 Merge pull request #1452 from prashanthvarma/master
CLOUDSTACK-9322: Support for Internal LB fuctionality with Nuage VSP SDN Plugin including Marvin test coverageTask: https://issues.apache.org/jira/browse/CLOUDSTACK-9322

PR contents:
1) UI changes to support LB provider InternalLbVm during VPC offering creation.
2) Bug fix for CLOUDSTACK-9320.
3) Nuage VSP SDN Plugin related enhancements for VPC network functionality.
4) Marvin test coverage for Internal LB support on master with Nuage VSP SDN Plugin.
5) Enhancements on our exiting Marvin test code (nuagevsp plugins directory).
6) PEP8 & PyFlakes compliance with our Marvin test code.

Test run:
CloudStack$ nosetests --with-marvin --marvin-config=nuage_ant.cfg test/integration/plugins/nuagevsp/ -a tags=nuagevsp

Test results:
Test user data and password reset functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_UserDataPasswordReset | Status : SUCCESS ===
ok
Test Nuage VSP VPC Offering with different combinations of LB service providers ... === TestName: test_01_nuage_internallb_vpc_Offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Network Offering with and without Internal LB service ... === TestName: test_02_nuage_internallb_vpc_network_offering | Status : SUCCESS ===
ok
Test Nuage VSP VPC Networks with and without Internal LB service ... === TestName: test_03_nuage_internallb_vpc_networks | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different combinations of Internal LB rules ... === TestName: test_04_nuage_internallb_rules | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality by performing (wget) traffic tests within a VPC ... === TestName: test_05_nuage_internallb_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with different LB algorithms by performing (wget) traffic tests ... === TestName: test_06_nuage_internallb_algorithms_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with restarts of VPC network components by performing (wget) ... === TestName: test_07_nuage_internallb_vpc_network_restarts_traffic | Status : SUCCESS ===
ok
Test Nuage VSP VPC Internal LB functionality with InternalLbVm appliance operations by performing (wget) ... === TestName: test_08_nuage_internallb_appliance_operations_traffic | Status : SUCCESS ===
ok
Test Basic VPC Network Functionality with Nuage VSP SDN plugin ... === TestName: test_nuage_vpc_network | Status : SUCCESS ===
ok
Test Nuage VSP SDN plugin with basic Isolated Network functionality ... === TestName: test_nuage_vsp | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 11 tests in 12094.705s

OK

Test run logs:
[results.txt](https://github.com/apache/cloudstack/files/187587/results.txt)
[runinfo.txt](https://github.com/apache/cloudstack/files/187588/runinfo.txt)

Test config file:
[nuage_ant.txt](https://github.com/apache/cloudstack/files/222711/nuage_ant.txt)

Note: Attached the Marvin config file as .txt instead of .cfg.

PEP8 & PyFlakes Compliance:
CloudStack$ pep8 --max-line-length=150 test/integration/plugins/nuagevsp/*.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/nuageTestCase.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_password_reset.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_internal_lb.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vpc_network.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/test_nuage_vsp.py
CloudStack$ pyflakes test/integration/plugins/nuagevsp/*.py

#CLOUDSTACK-9322

* pr/1452:
  CLOUDSTACK-9322 : Marvin tests for Internal Lb with Nuage VSP
  CLOUDSTACK-9320 : InternalLBVM is not getting destroyed when the last Internal Load Balancer rule is removed for the corresponding source IP address
  CLOUDSTACK-9322 : Changes to support InternalLbVm with Nuage VSP plugin

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-25 15:53:58 -04:00
Abhinandan Prateek
182ab648b5 CLOUDSTACK-9323: Fix Cancel maintenance so that if maintenance is cancelled the host come back to normal state gracefully.
Added marvin tests for host maintennace.
2016-04-25 15:28:32 +05:30
Daan Hoogland
101668994d further maven dependency updates from Daan
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-22 16:36:37 +05:30
Will Stevens
f530a4c63a Merge pull request #1200 from koushik-das/CLOUDSTACK-9130
CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag

RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.

* pr/1200:
  CLOUDSTACK-9130: Make RebootCommand similar to start/stop/migrate agent commands w.r.t. "execute in sequence" flag RebootCommand now behaves in the same way as start/stop/migrate agent commands w.r.t. to sequential/parallel execution.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-21 16:35:27 -04:00
Kshitij Kansal
fae20570c2 Fixed Coverity Issue: Unintentional Integer FLow 2016-04-21 12:06:36 +05:30
Kshitij Kansal
e5e226e549 Fixed: Error given when creating VPN user in one network if VR for another network is stopped 2016-04-21 12:06:36 +05:30
SudharmaJain
93ce108e0c CLOUDSTACK-8901: PrepareTemplate job thread hard-coded to max 8 threads 2016-04-21 09:13:06 +05:30
David Mabry
0b7cc087f8 Let hypervisor type KVM and Simulator detach root volumes.
Updated test_volumes.py to include a test for detaching and reattaching a root volume from a vm. I also had to update base.py to allow attach_volume to have the parameter deviceid to be passed as needed.
2016-04-18 15:40:36 -05:00
Rohit Yadav
8e4644e413 vmware: improve support for disks
- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info
- Refactor and move VirtualMachineDiskInfo to cloud-utils
- Allows mixing of scsi controller types
- Fixes a NPE case with map passed as null, for example in case of detach volume
  command
- Use a osdefault translator that allow use of recent os types added (enums of
  which) are not available in the sdk

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-18 22:44:18 +05:30
Will Stevens
08f2ec5044 Merge pull request #1263 from rafaelweingartner/lrg-cs-hackday-018
Removed unused code from com.cloud.api.ApiServer**Removed \_ from variables names**: private variables with \_ at the beginning is common in C++ but not in Java.

**Removed unused code from ApiServer:**
- com.cloud.api.ApiServer.getPluggableServices(): unused method;
- com.cloud.api.ApiServer.getApiAccessCheckers(): unused method;

**Methods and variables access level reviewed:**
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String ,Object): this method was private but the annotation @MessageHandler requests public methods, as can be seen in org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class\<?\>), which searches methods with the @MessageHandler annotation and changes
it to be accessible (setAccessible(true)). Thus, there is no reason for handleAsyncJobPublishEvent be a private method and lead some other dev to wrong conclusions about the use of the method;
- Global variables and methods called just by this class (ApiServer) were changed to private.

**Changed variables and methods from static to non-static (if possible):** as some variables/methods are used just by one object of this class, instantiated by Spring, they were changed to non-static.

With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used static methods from ApiServer, were changed from ApiServer.\<staticMethodName\> to \_apiServer.\<methodName\> that refers to the org.apache.cloudstack.api.ApiServerService interface. Thus, methods com.cloud.api.ApiServer.getJSONContentType() and com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be added in the interface (org.apache.cloudstack.api.ApiServerService, interface implemented by class ApiServer).

* pr/1263:
  The goal of this PR is to review com.cloud.api.ApiServer class, with the following actions:

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-18 08:34:53 -04:00
gabrascher
48888e7405 The goal of this PR is to review com.cloud.api.ApiServer class, with the
following actions:

Removed “_” in beginning of global variables names:
Variables was changed from “_<variablename>” to “<variablename>”, as
this convension (private veriables with “_”) is common in C++ but not in
Java.

Removed unused code from ApiServer:
- com.cloud.api.ApiServer.getPluggableServices():
	Unused method.
- com.cloud.api.ApiServer.getApiAccessCheckers():
	Unused method.

Methods and variables access level reviewed:
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String,
Object):
	This method was private but the annotation @MessageHandler requests
public methods, as can be seen in
org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class<?>),
which searches methods with the @MessageHandler annotation and changes
it to accessible (“setAccessible(true)”). Thus, there is no reason for
handleAsyncJobPublishEvent be a private method.

- Global variables and methods called just by this class (ApiServer)
were changed to private.

Changed variables and methods from static to non static (if possible):
As some variables/methods are used just by one object of this class
(instantiated by springer), they were changed to non static.

With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used
static methods from ApiServer, was changed from
ApiServer.<staticMethodName> to _apiServer.<methodName> that refers to
the org.apache.cloudstack.api.ApiServerService interface. Thus, methods
com.cloud.api.ApiServer.getJSONContentType() and
com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be
included in the interface (org.apache.cloudstack.api.ApiServerService,
interface implemented by class ApiServer).

However, com.cloud.api.ApiServer.isEncodeApiResponse() was keept static,
as its call hierarchy would have to be changed (more than planed for
this PR).
2016-04-12 15:46:07 -03:00
Koushik Das
d1def0a730 Merge pull request #1287 from DaanHoogland/securityrules-cleanup
SecurityGroupRulesCmd code cleanupWrote a test and cleaned some duplicate code with the objective to evaluate the jenkins pull request process at builds.a.o
worthwhile to keep, IMHO.

* pr/1287:
  SecurityGroupRulesCmd code cleanup review comments handled
  deal with PMD warnings
  code cleanup
  security rules test
  remove autogenerated pydev files

Signed-off-by: Koushik Das <koushik@apache.org>
2016-04-11 21:49:57 +05:30
Wilder Rodrigues
850fb1a557 CLOUDSTACK-9287 - Check if the nic profile has already been removed from a certain router
- In case of redundant VPCs, the ACL items are revoked in the first iteration. Since the econd iteration
     is needed in order to remove the private network, we have to check if the nic profile is gone before trying
     to revoke the ACL items again, which would throw a NPE.
   - Some variable extraction in order to ease debugging.
2016-04-09 21:14:17 +02:00
Wilder Rodrigues
f921ec686b CLOUDSTACK-9287 - Generate new mac address if router is redundant and nic profile exists 2016-04-09 21:13:28 +02:00