2121 Commits

Author SHA1 Message Date
Rohit Yadav
abfcd5b95f CLOUDSTACK-9632: Upgrade bouncy castle to version 1.55
- Upgrades Maven dependency version to v1.55
- Fixes bountycastle usages and issues
- Adds timeout to jetty/annotation scanning
- Fixes servlet issue, uses servlet 3.1.0
- Downgrade javassist used by reflections to fix annotation process errors
- Make console-proxy-rdp bc dependency same as rest of the codebase
- Picks up PR #1510 by Daan

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-12-06 17:24:24 +05:30
nvazquez
f4a42ab0d7 CLOUDSTACK-9535: [API] listVMSnapshots improvement 2016-10-05 17:42:14 -03:00
Will Stevens
123e9f81ab fixed a variable name broken by merge conflicts 2016-05-27 16:32:20 -04:00
Will Stevens
121b3d6403 Merge pull request #1567 from exoscale/CLOUDSTACK-9238
CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VOI will update the PR to add max field length in the API commands too

* pr/1567:
  API: update url field max length
  not needed on host table
  Fix URL length to 2048 for all url fields in VO

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:20:22 -04:00
Will Stevens
20f779298d merging 4.8 -> master 2016-05-27 15:19:33 -04:00
Will Stevens
744f9d5615 Merge release branch 4.7 to 4.8
* 4.7:
  CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
  CLOUDSTACK-9369: Restrict default login to ldap/native users
  Add lsb-release dependency to mgmt server and agent on Debian/Ubuntu.
  Emit template UUID and class type over event bus when deleting templates.
2016-05-27 15:03:42 -04:00
Rohit Yadav
0cb60a72fe CLOUDSTACK-9376: Restrict listTemplates API with filter=all for root admin
Restricts use of listemplates API with templatefilter=all for root admin only.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:01:15 -04:00
Rohit Yadav
566e7d9fac CLOUDSTACK-9369: Restrict default login to ldap/native users
- Restricts default login auth handler to ldap and native-cloudstack users
- Refactors and create re-usable method to find domain by id/path
- Adds unit test for refactored method in DomainManagerImpl
- Adds smoke test for login handler

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-27 15:00:05 -04:00
Marc-Aurèle Brothier
a59ee03fd7 Fix URL length to 2048 for all url fields in VO 2016-05-27 08:16:05 +02:00
Will Stevens
978184bccb Merge pull request #1424 from syed/create-template-api-bug
[CLOUDSTACK-8973] Fix create template from snapshot returning null in case of region storeThis PR fixes a case where when we create a template from a snpashot in a region wide store, we don't set the cross zone flag which causes a null response to be returned
Tests:

Before fix
```
(local)  > create template snapshotid=33aa3f3b-5a47-4d2a-8d27-12952c01ebed displaytext=t2 ostypeid=20c8ead6-d750-11e5-9f8c-06524200007c name=t9

accountid = 1b13d7c2-d750-11e5-9f8c-06524200007c
cmd = org.apache.cloudstack.api.command.admin.template.CreateTemplateCmdByAdmin
created = 2016-02-23T16:09:24+0000
jobid = 4f9f5ff9-e7f0-4af6-999c-799431fd47de
jobinstanceid = a08a9711-bd31-43bb-80a2-49cf9d722a19
jobinstancetype = Template
jobprocstatus = 0
jobresult:
null:
crossZones = False
isfeatured = False
ispublic = False
isready = False
tags:
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 1b140f08-d750-11e5-9f8c-06524200007c
```

See the *null* in response

After fix:

```
(local)  > create template snapshotid=33aa3f3b-5a47-4d2a-8d27-12952c01ebed displaytext=t2 ostypeid=20c8ead6-d750-11e5-9f8c-06524200007c name=t11

accountid = 1b13d7c2-d750-11e5-9f8c-06524200007c
cmd = org.apache.cloudstack.api.command.admin.template.CreateTemplateCmdByAdmin
created = 2016-02-25T21:47:03+0000
jobid = 1b74209b-b3c1-4168-a243-f559aa0c081b
jobinstanceid = 06ecee5a-b1f2-4e67-80fb-f0f44b0aa198
jobinstancetype = Template
jobprocstatus = 0
jobresult:
template:
id = 06ecee5a-b1f2-4e67-80fb-f0f44b0aa198
name = t11
account = admin
created = 2016-02-25T21:47:03+0000
crossZones = True
displaytext = t2
domain = ROOT
domainid = 1b13ab80-d750-11e5-9f8c-06524200007c
format = VHD
hypervisor = XenServer
isdynamicallyscalable = False
isextractable = True
isfeatured = False
ispublic = False
isready = True
ostypeid = 20c8ead6-d750-11e5-9f8c-06524200007c
ostypename = CentOS 5 (64-bit)
passwordenabled = False
size = 21474836480
sourcetemplateid = 1af0f0cc-d750-11e5-9f8c-06524200007c
sshkeyenabled = False
status = Download Complete
tags:
templatetype = USER
jobresultcode = 0
jobresulttype = object
jobstatus = 1
userid = 1b140f08-d750-11e5-9f8c-06524200007c
```
Works correctly

* pr/1424:
  Fix create template from snapshot returning null in case of region store

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 23:04:31 -04:00
Will Stevens
550aabb308 Merge pull request #1550 from ustcweizhou/CLOUDSTACK-9380-listDomains-NPE
CLOUDSTACK-9380: fix NPE in listDomains API for a mistakeThe issue happens if volumeTotal is NULL in database.
This is caused by commit 0407fb334f3a79f570217f35636b47076b06d500 for CLOUDSTACK-7847.

* pr/1550:
  CLOUDSTACK-9380: fix NPE in listDomains API for a mistake

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:58:46 -04:00
Will Stevens
9456aa0e38 Merge pull request #1503 from Amysta/fix-CLOUDSTACK-9358
CLOUDSTACK-9358: StringIndexOutOfBoundsException on eventsFixes JSON deserialization of `cmdInfo` (current process fails with `StringIndexOutOfBoundsException` when `cmdEventType` is the last parameter in the JSON string).

A `StringIndexOutOfBoundsException` is thrown in some cases during event publication.

Example: a stopVirtualMachine API request is executed, and fails with:

```
2016-04-15 09:24:43,080 ERROR [o.a.c.f.m.MessageDispatcher] (catalina-exec-1:ctx-840cbaa7 ctx-8daf0e9c ctx-f63af073) Unexpected exception when calling com.cloud.api.ApiServer.handleAsyncJobPublishEvent
java.lang.reflect.InvocationTargetException
        at sun.reflect.GeneratedMethodAccessor307.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.cloudstack.framework.messagebus.MessageDispatcher.dispatch(MessageDispatcher.java:75)
        at org.apache.cloudstack.framework.messagebus.MessageDispatcher.onPublishMessage(MessageDispatcher.java:45)
        at org.apache.cloudstack.framework.messagebus.MessageBusBase$SubscriptionNode.notifySubscribers(MessageBusBase.java:441)
        at org.apache.cloudstack.framework.messagebus.MessageBusBase.publish(MessageBusBase.java:178)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl.publishOnEventBus(AsyncJobManagerImpl.java:1052)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl.submitAsyncJob(AsyncJobManagerImpl.java:180)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl.submitAsyncJob(AsyncJobManagerImpl.java:168)
        at com.cloud.api.ApiServer.queueCommand(ApiServer.java:687)
        at com.cloud.api.ApiServer.handleRequest(ApiServer.java:528)
        at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:296)
        at com.cloud.api.ApiServlet$1.run(ApiServlet.java:127)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:124)
        at com.cloud.api.ApiServlet.doGet(ApiServlet.java:86)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1720)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1679)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -468
        at java.lang.String.substring(String.java:1967)
        at com.cloud.api.ApiServer.handleAsyncJobPublishEvent(ApiServer.java:270)
        at sun.reflect.GeneratedMethodAccessor307.invoke(Unknown Source)
        ... 41 more
```

The VM is not stopped, and the API returns the following error:

      HTTP/1.1 530 : InvocationTargetException when invoking event handler for subject: job.eventpublish

The error comes from JSON parsing in 339355594c/server/src/com/cloud/api/ApiServer.java (L270)

The `substring()` method throws a StringIndexOutOfBoundsException when `cmdEventType` is the last field in the JSON object.

The JSON object being the JSON serialization of a `HashMap` object, the order of the fields is not guaranteed. In my tests, I observed that there is no error with Java 7, but it fails with Java 8, which has a different Hashmap implementation.

For example, with request http://127.0.0.1:8096/client/api?command=stopVirtualMachine&id=734ef752-fe99-4d46-8f31-18130496473c, cmdInfo is:

```
{"ctxUserId":"1","httpmethod":"GET","ctxStartEventId":"105","id":"734ef752-fe99-4d46-8f31-18130496473c","ctxDetails":"{\"interface com.cloud.vm.VirtualMachine\":\"734ef752-fe99-4d46-8f31-18130496473c\"}","ctxAccountId":"1","uuid":"734ef752-fe99-4d46-8f31-18130496473c","cmdEventType":"VM.STOP"}
```

and it fails.

Please note that the error is not present for all API requests, depending on the parameters fields.

* pr/1503:
  CLOUDSTACK-9358: StringIndexOutOfBoundsException on events

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:56:18 -04:00
Wei Zhou
59edbb5337 CLOUDSTACK-9380: fix NPE in listDomains API for a mistake 2016-05-17 14:45:50 +02:00
Will Stevens
8c3722d953 Merge pull request #1444 from rafaelweingartner/workAroundPR780
CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VMThis PR introduces the changes proposed in PR #780 with some work to make the code null safe.

During this PR, I have also removed some unused code.

* pr/1444:
  Removed unnecessary check when creating the “userVmResponse” object.
  Fixed issues from CLOUDSTACK-8800 that were introduced in PR 780
  CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 16:49:14 -04:00
Rohit Yadav
07564469e9 CLOUDSTACK-9299: Out-of-band Management for CloudStack
Support access to a host’s out-of-band management interface (e.g. IPMI, iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and iDRA,
the service implementation allows for development of separate drivers as plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature `ipmisim` can be used:
https://pypi.python.org/pypi/ipmisim

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-12 11:05:40 +05:30
Rohit Yadav
afcbd270b2 CLOUDSTACK-8562: Deprecate commands.properties
- Removes commands.properties file
- Fixes apidocs and marvin to be independent of commands.properties usage
- Removes bundling of commands.properties in deb/rpm packaging
- Removes file references across codebase

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Rohit Yadav
4347776ac6 CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.

A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack

DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
  current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
  Admin, Domain Admin and User) which maintains this association by requiring
  all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
  removal of roles and/or modifications of permissions, without the need
  of restarting management server(s)

Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
  deployments will continue to use the older static role based api access checker
  with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
  roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
  to add existing set of permissions to the default roles. cloud.account
  will have a new role_id column which will be populated based on default roles
  as well

Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Rohit Yadav
95abb6efc3 CLOUDSTACK-9361: Centrally handle API validations
Validate API arguments based on annotations. Introduces:
- NotNullOrEmpty: for doing null and empty string checks
- PositiveNumber: number > 0 (natural number)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:40:22 +05:30
Syed
7b5d5648d8 Fix create template from snapshot returning null in case of region store 2016-05-09 15:59:23 -04:00
weingartner
417d9a5d80 Removed unnecessary check when creating the “userVmResponse” object. 2016-05-06 15:16:37 -03:00
nvazquez
baa8610061 CLOUDSTACK-9351: Add ids parameter to resource listing API calls 2016-05-06 14:42:42 -03:00
Olivier Lemasle
61d2692597 CLOUDSTACK-9358: StringIndexOutOfBoundsException on events
Fixes JSON deserialization of cmdInfo (current process fails with
StringIndexOutOfBoundsException when cmdEventType is the last parameter
in the JSON string.
2016-05-04 01:15:03 +02:00
Rohit Yadav
4d57ec04ac Merge branch '4.8'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-03 08:22:37 +05:30
Rohit Yadav
126a039eb8 Merge branch '4.7' into 4.8 2016-05-03 08:20:27 +05:30
Will Stevens
62d9f444e3 Merge pull request #1321 from nitin-maharana/CloudStack-Nitin5_4.7
CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM idWhen calling listServiceOfferings with VM id as parameter. It is returning incompatible tagged offerings. It should only list all compatible tagged offerings. Compatible means the new service offering should contain all the tags of the existing service offering(Existing offering SUBSET of new offering). If that is the case It should list in the result and can be upgraded to that offering.

* pr/1321:
  CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM id

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-02 16:59:57 -04:00
Maneesha.P
7ad3c5e834 CLOUDSTACK-8800 : Improved the listVirtualMachines API call to include memory utilization information for a VM for xenserver,kvm and for vmware. 2016-04-29 14:06:49 -03:00
Nathan Johnson
7778e24bf7 Log responses from asynchronous api commands
This is mainly useful so that we can see the job ids in the logs
whenever the job is created via the API.
2016-04-27 19:43:57 -05:00
Will Stevens
08f2ec5044 Merge pull request #1263 from rafaelweingartner/lrg-cs-hackday-018
Removed unused code from com.cloud.api.ApiServer**Removed \_ from variables names**: private variables with \_ at the beginning is common in C++ but not in Java.

**Removed unused code from ApiServer:**
- com.cloud.api.ApiServer.getPluggableServices(): unused method;
- com.cloud.api.ApiServer.getApiAccessCheckers(): unused method;

**Methods and variables access level reviewed:**
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String ,Object): this method was private but the annotation @MessageHandler requests public methods, as can be seen in org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class\<?\>), which searches methods with the @MessageHandler annotation and changes
it to be accessible (setAccessible(true)). Thus, there is no reason for handleAsyncJobPublishEvent be a private method and lead some other dev to wrong conclusions about the use of the method;
- Global variables and methods called just by this class (ApiServer) were changed to private.

**Changed variables and methods from static to non-static (if possible):** as some variables/methods are used just by one object of this class, instantiated by Spring, they were changed to non-static.

With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used static methods from ApiServer, were changed from ApiServer.\<staticMethodName\> to \_apiServer.\<methodName\> that refers to the org.apache.cloudstack.api.ApiServerService interface. Thus, methods com.cloud.api.ApiServer.getJSONContentType() and com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be added in the interface (org.apache.cloudstack.api.ApiServerService, interface implemented by class ApiServer).

* pr/1263:
  The goal of this PR is to review com.cloud.api.ApiServer class, with the following actions:

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-18 08:34:53 -04:00
gabrascher
48888e7405 The goal of this PR is to review com.cloud.api.ApiServer class, with the
following actions:

Removed “_” in beginning of global variables names:
Variables was changed from “_<variablename>” to “<variablename>”, as
this convension (private veriables with “_”) is common in C++ but not in
Java.

Removed unused code from ApiServer:
- com.cloud.api.ApiServer.getPluggableServices():
	Unused method.
- com.cloud.api.ApiServer.getApiAccessCheckers():
	Unused method.

Methods and variables access level reviewed:
- com.cloud.api.ApiServer.handleAsyncJobPublishEvent(String, String,
Object):
	This method was private but the annotation @MessageHandler requests
public methods, as can be seen in
org.apache.cloudstack.framework.messagebus.MessageDispatcher.buildHandlerMethodCache(Class<?>),
which searches methods with the @MessageHandler annotation and changes
it to accessible (“setAccessible(true)”). Thus, there is no reason for
handleAsyncJobPublishEvent be a private method.

- Global variables and methods called just by this class (ApiServer)
were changed to private.

Changed variables and methods from static to non static (if possible):
As some variables/methods are used just by one object of this class
(instantiated by springer), they were changed to non static.

With that, calls from com.cloud.api.ApiServlet.ApiServlet() that used
static methods from ApiServer, was changed from
ApiServer.<staticMethodName> to _apiServer.<methodName> that refers to
the org.apache.cloudstack.api.ApiServerService interface. Thus, methods
com.cloud.api.ApiServer.getJSONContentType() and
com.cloud.api.ApiServer.isSecureSessionCookieEnabled() had to be
included in the interface (org.apache.cloudstack.api.ApiServerService,
interface implemented by class ApiServer).

However, com.cloud.api.ApiServer.isEncodeApiResponse() was keept static,
as its call hierarchy would have to be changed (more than planed for
this PR).
2016-04-12 15:46:07 -03:00
nvazquez
9857f41d9a CLOUDSTACK-9298: Add @MappedSuperClass support for persistence inheritance 2016-03-10 06:07:29 -08:00
nvazquez
db54b26fed CLOUDSTACK-9298: Improve ListTemplatesCmd, ListVolumesCmd and ListVMsCmd performance 2016-03-10 06:07:29 -08:00
Rohit Yadav
f280bbe9e4 Merge pull request #830 from sureshanaparti/CLOUDSTACK-8858
CLOUDSTACK-8858: listVolumes API fails for a particular domain with NPE.CLOUDSTACK-8858: listVolumes API fails for a particular domain with NPE.

Summary: listVolumes API fails when volume associated vm instance has NULL or invalid state. Fix the code to guard this situation since this should not block volume listing.

* pr/830:
  CLOUDSTACK-8858: listVolumes API fails for a particular domain with NPE.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-01-27 15:56:07 +01:00
Remi Bergsma
75b68c6829 Merge release branch 4.7 to master
* 4.7:
  Fix unable to setup more than one Site2Site VPN Connection
  FIX S2S VPN rVPC: Check only redundant routers in state MASTER
  PEP8 of integration/smoke/test_vpc_vpn
  Add S2S VPN test for Redundant VPC
  Make integration/smoke/test_vpc_vpn Hypervisor independant
  FIX VPN: non-working ipsec commands
  [UI] MADNESS
  [DB] Add force_encap field to s2s_customer_gateway table
  [ROUTER] Add forceencaps field to python router ipsec config method
  [TEST] unittest needs rework
  [MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
  [CORE] Add Force UDP Encapsulation option to Site2Site VPN
  CLOUDSTACK-9186: Root admin cannot see VPC created by Domain admin user
  CLOUDSTACK-9192: UpdateVpnCustomerGateway is failing
  CLOUDSTACK-6485 prevent ip asignment of private gw iface
  CLOUDSTACK-9204 Do not error when staticroute is already gone
  make both check lines consistent
  CLOUDSTACK-9181 Prevent syntax error in checkrouter.sh
  CLOUDSTACK-9202 Bump ssh timeout
2016-01-16 19:54:41 +01:00
Nitin Kumar Maharana
ba0503d000 CLOUDSTACK-8847: ListServiceOfferings is returning incompatible tagged offerings when called with VM id
Fixed the subset and superset issue.
Added unit test for the same.
2016-01-08 14:17:57 +05:30
Michael Andersen
6da3bc1237 [CORE] Add Force UDP Encapsulation option to Site2Site VPN 2016-01-07 19:27:37 +01:00
nvazquez
8149081658 CLOUDSTACK-9074: API Changes: Add nsxLogicalSwitch and nsxLogicalSwitchPort to listNics 2015-12-24 16:07:06 -03:00
Wei Zhou
24f1396ac1 CLOUDSTACK-9129: list vpc routers by keyword in Infrastructure -> Virtual Routers
and two more changes:
(1) add network name/vpc name in the listRouters response
(2) add network name/vpc id, vpc name in the router details page
2015-12-10 09:14:12 +01:00
Abhinandan Prateek
987fcbd441 CLOUDSTACK-8592: Implement Quota service
Quota service while allowing for scalability will make sure that the cloud is
not exploited by attacks, careless use and program errors. To address this
problem, we propose to employ a quota-enforcement service that allows resource
usage within certain bounds as defined by policies and available quotas for
various entities.  Quota service extends the functionality of usage server to
provide a measurement for the resources used by the accounts and domains using a
common unit referred to as cloud currency in this document. It can be configured
to ensure that your usage won’t exceed the budget allocated to accounts/domain
in cloud currency.  It will let user know how much of the cloud resources he is
using. It will help the cloud admins, if they want, to ensure that a user does
not go beyond his allocated quota. Per usage cycle if a account is found to be
exceeding its quota then it is locked. Locking an account means that it will not
be able to initiat e a new resource allocation request, whether it is more
storage or an additional ip. Needless to say quota service as well as any action
on the account is configurable.

Changes from Github code review:

- Added marvin test for quota plugin API
- removed unused commented code
- debug messages in debug enabled check
- checks for nulls, fixed access to member variables and feature
- changes based on PR comments
- unit tests for UsageTypes
- unit tests for all Cmd classes
- unit tests for all service and manager impls
- try-catch-finally or try-with-resource in dao impls for failsafe db switching
- remove dead code
- add missing quota calculation case (regression fixed)
- replace tabs with spaces in pom.xmls
- quota: though default value for quota_calculated is 0, the usage server
  makes it null while entering usage entries. Flipping the condition so
  as to acocunt for that.
- quotatypes: fix NPE in quota type
- quota framework test fixes
- made statement period configurable
- changed default email templates to reflect the fact that exhausted quota may not result in a locked account
- added quotaUpdateCmd that refreshes quota balances and sends alerts and statements
- report quotaSummary command returns quota balance, quota usage and state for all account
- made UI framework changes to allow for text area input in edit views
- process usage entries that have greater than 0 usage
- orocess quota entries only if tariff is non zero
- if there are credit entries but no balance entry create a dummy balance entry
- remove any credit entries that are before the last balance entry
  when displaying balance statement
- on a rerun the last balance is now getting added

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS
PR: https://github.com/apache/cloudstack/pull/768

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-12-07 23:02:48 +05:30
Daan Hoogland
638f1cf091 Merge pull request #1049 from DaanHoogland/CLOUDSTACK-9047
CLOUDSTACK-9047 rename enumsmake enums adhere to best practice naming conventions

* pr/1049:
  CLOUDSTACK-9046 rename enums to adhere to naming conventions
  CLOUDSTACK-9046 renamed enums in kvm plugin
  CLOUDSTACK-9047 use 'State's only with context   there are more types called 'State'   (or to be called so but now 'state')   So remove imports and prepend their enclosing class/context to them.

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-06 20:17:04 +01:00
Remi Bergsma
7e902cd505 Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
  CLOUDSTACK-9075 - Adds VPC static routes test
  CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
  CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
  CLOUDSTACK-6276 Removing unused parameter in integration test for projects
  CLOUDSTACK-6276 Removing unused parameter in integration test
  CLOUDSTACK-6276 Fixing affinity groups for projects
2015-12-03 20:42:41 +01:00
Remi Bergsma
a1d2c531f6 Merge pull request #1007 from ustcweizhou/dedicated-ip-for-domain
[4.7] CLOUDSTACK-8958: add dedicated ips to domain (account for now)For now, we dedicate ip pool to account, however, other accounts in the same domain cannot fetch the ip from this ip pool.
By dedicating ip pool to domain, accounts in the domain can fetch the public ip from same ip pool.

* pr/1007:
  CLOUDSTACK-8958: throw an exception if project account cannot be found
  CLOUDSTACK-8958: add dedicated ips to domain (account for now)

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-29 09:39:25 +01:00
Patrick Dube
c76d317150 CLOUDSTACK-6276 Fixing affinity groups for projects 2015-11-27 14:43:02 -05:00
Remi Bergsma
a2a72887d2 Merge pull request #1021 from koushik-das/CLOUDSTACK-8485
CLOUDSTACK-8485: listAPIs are taking too long to return results- Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d4734724358df97b6fa4d8c5beb0f447745e
- Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields
- Added annotation (@LogLevel(Log4jLevel.Off)) to sensitive response object fields

Ran the following tests on simulator:

test_vm_life_cycle.py

Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
ok
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
ok
Test migrate VM ... === TestName: test_08_migrate_vm | Status : SUCCESS ===
ok
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 10 tests in 306.429s

OK

test_volumes.py

Download a Volume attached to a VM ... === TestName: test_03_download_attached_volume | Status : SUCCESS ===
ok
Delete a Volume attached to a VM ... === TestName: test_04_delete_attached_volume | Status : SUCCESS ===
ok
Detach a Volume attached to a VM ... === TestName: test_05_detach_volume | Status : SUCCESS ===
ok
Delete a Volume unattached to an VM ... === TestName: test_09_delete_detached_volume | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 4 tests in 184.132s

OK

test_network.py

Test for delete account ... === TestName: test_delete_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for admin account ... === TestName: test_public_ip_admin_account | Status : SUCCESS ===
ok
Test for Associate/Disassociate public IP address for user account ... === TestName: test_public_ip_user_account | Status : SUCCESS ===
ok
Test for release public IP address ... === TestName: test_releaseIP | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 4 tests in 783.726s

OK

test_routers.py

Test router internal advanced zone ... SKIP: Marvin configuration has no host credentials                            to check router services
Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
ok
Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
ok
Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
ok
Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
ok
Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
ok
Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 7 tests in 42.958s

OK (SKIP=1)

test_global_settings.py

test update configuration setting at zone level scope ... === TestName: test_UpdateConfigParamWithScope | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 0.127s

OK

test_resource_detail.py

Test volume detail ... === TestName: test_01_updatevolumedetail | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 11.492s

OK

* pr/1021:
  CLOUDSTACK-8485: listAPIs are taking too long to return results - Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d4734724358df97b6fa4d8c5beb0f447745e - Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields - Added new parameter 'isSensitive' to @Param for marking a field as sensitive in response objects

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-23 12:56:25 +01:00
cirstofolini
1a64c247ad Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses. 2015-11-21 18:31:11 -02:00
Koushik Das
e13df96348 CLOUDSTACK-8485: listAPIs are taking too long to return results
- Removed regex. based search/replace of sensitive data on API response introduced as part of commit b0c6d4734724358df97b6fa4d8c5beb0f447745e
- Added new response serializer to skip sensitive data from getting logged based on annotation present in resposne object fields
- Added new parameter 'isSensitive' to @Param for marking a field as sensitive in response objects
2015-11-20 16:40:53 +05:30
Daan Hoogland
c669c93b05 CLOUDSTACK-9046 rename enums to adhere to naming conventions 2015-11-16 13:41:16 +00:00
Wei Zhou
37301ed454 CLOUDSTACK-8958: add dedicated ips to domain (account for now) 2015-11-16 10:17:40 +01:00
Wei Zhou
cb9bf44538 [UI] Network names are not being displayed in Infra » VRs » router » NICS 2015-11-06 11:05:12 +01:00
ramamurtis
c90723f491 CLOUDSTACK-9006 - ListTemplates API returns result in inconsistent order when called concurrently 2015-10-30 09:44:59 +05:30
Remi Bergsma
535ab51b9a Merge pull request #849 from karuturi/CLOUDSTACK-8816-take2
Cloudstack-8816 some of the events do not have resource uuidsThe key objects in the context map are sometimes String and sometimes object. This causes missing uuids when an entity put in the context map with key entity.toString is queried with key entity

Testing:
manually tested by deploying a vm and checked that the created events in rabbitmq now has uuids.
events before and after the change are update at https://issues.apache.org/jira/browse/CLOUDSTACK-8816?focusedCommentId=14805239

unittests
```
$ mvn -pl :cloud-api test -Dtest=CallContextTest
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.cloudstack.context.CallContextTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.152 sec - in org.apache.cloudstack.context.CallContextTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.445 s
[INFO] Finished at: 2015-09-18T14:58:53+05:30
[INFO] Final Memory: 55M/448M
[INFO] ------------------------------------------------------------------------
```

* pr/849:
  CLOUDSTACK-8816 added missing events
  CLOUDSTACK-8816: fixed missing resource uuid in delete network cmd
  CLOUDSTACK-8816: fixed missing resource uuid in destroy vm event
  Cloudstack-8816: Fixed missing resource uuid in delete snapshot events
  CLOUDSTACK-8816: some of the events do not have resource uuids
  CLOUDSTACK-8816: some of the events do not have resource uuids

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-10-27 19:26:44 +01:00