812 Commits

Author SHA1 Message Date
Will Stevens
121b3d6403 Merge pull request #1567 from exoscale/CLOUDSTACK-9238
CLOUDSTACK-9238: Fix URL length to 2048 for all url fields in VOI will update the PR to add max field length in the API commands too

* pr/1567:
  API: update url field max length
  not needed on host table
  Fix URL length to 2048 for all url fields in VO

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-27 15:20:22 -04:00
Marc-Aurèle Brothier
d0453fc296 not needed on host table 2016-05-27 08:26:09 +02:00
Marc-Aurèle Brothier
a59ee03fd7 Fix URL length to 2048 for all url fields in VO 2016-05-27 08:16:05 +02:00
Will Stevens
05a7beccfe Merge pull request #1494 from nlivens/master_nuage
Remodeling of Nuage VSP Plugin + CLOUDSTACK-9294Hi all,

We've remodeled the Nuage VSP plugin to use the same model as VMWare is using (non-OSS). Before, we had a runtime dependency to the Nuage Client, this has been changed to a compile-time dependency instead because of multiple reasons (build management, readability, maintainability, ...)

We've adapted the code so it now uses model objects defined in the Nuage client instead of passing a list of parameters to the Nuage client. This is a lot more readable, and a lot more maintainable.

I've had a chat with @DaanHoogland about this approach, and he told me that ACS is trying to move away from the whole non-OSS approach. We're looking into the Juniper approach, we would set up a custom maven repository which would host the required dependencies for the Nuage VSP plugin.

Any remarks or suggestions are always welcome :)

* pr/1494:
  Nuage VSP : Extending Marvin test coverage
  Nuage VSP : Fix for NPE while cleaning up account when there are still resources belonging to that account
  CLOUDSTACK-9294 : Make sure to remove VR from VSD when removing the VPC
  CLOUDSTACK-9242 : Remodel Nuage VSP plugin

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-25 22:58:10 -04:00
Nick Livens
4b6179940d CLOUDSTACK-9294 : Make sure to remove VR from VSD when removing the VPC 2016-05-24 11:12:19 +02:00
Kishan Kavala
c12d83601d Bug-ID: CLOUDSTACK-8870: Skip external device usage collection if no external devices exist 2016-05-23 15:23:23 +05:30
Will Stevens
82b702dc9a Merge pull request #1403 from mike-tutkowski/xs-snapshots
Taking fast and efficient volume snapshots with XenServer (and your storage provider)A XenServer storage repository (SR) and virtual disk image (VDI) each have UUIDs that are immutable.

This poses a problem for SAN snapshots, if you intend on mounting the underlying snapshot SR alongside the source SR (duplicate UUIDs).

VMware has a solution for this called re-signaturing (so, in other words, the snapshot UUIDs can be changed).

This PR only deals with the CloudStack side of things, but it works in concert with a new XenServer storage manager created by CloudOps (this storage manager enables re-signaturing of XenServer SR and VDI UUIDs).

I have written Marvin integration tests to go along with this, but cannot yet check those into the CloudStack repo as they rely on SolidFire hardware.

If anyone would like to see these integration tests, please let me know.

JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-9281

Here's a video I made that shows this feature in action:

https://www.youtube.com/watch?v=YQ3pBeL-WaA&list=PLqOXKM0Bt13DFnQnwUx8ZtJzoyDV0Uuye&index=13

* pr/1403:
  Faster logic to see if a cluster supports resigning
  Support for backend snapshots with XenServer

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-20 08:33:07 -04:00
Will Stevens
3f5b3a16dd Merge pull request #1516 from sudhansu7/CLOUDSTACK-9366
CLOUDSTACK-9366: Capacity of one zone-wide primary storage ignoredDisable and Remove Host operation disables the primary storage capacity.

Steps to replicate:
Base Condition: There exists a host and storage pool with same id
Steps:
1. Find a host and storage pool having same id
2. Disable the host
3. CPU(1) and MEMORY(0) capacity in op_host_capacity for above host is disabled
4. STORAGE(3) capacity in op_host_capacity for storage pool with id same as above host is also disabled

RCA:
'host_id' column in 'op_host_capacity' table used for storing both storage pool id (for STORAGE capacity) and host id (MEMORY and CPU). While disabling a HOST we also disable the capacity associated with host.

Ideally while disabling capacity we should only disable MEMORY and CPU capacity, but we are not doing so.

Code Path:
ResourceManagerImpl.doDeleteHost() -> ResourceManagerImpl.resourceStateTransitTo() -> CapacityDaoImpl.updateCapacityState(null, null, null, host.getId(), capacityState.toString())

updateCapacityState is updating disabling all entries which matches the host_id. This will also disable a entry having storage pool id same as that of host id.

Changes:
introduced new capacityType parameter in updateCapacityState method and necessary changes to add capacity_type clause in sql
also fixed incorrect sql builder logic (unused code path for which it is never surfaced )
Added marvin test to  check host and storagepool capacity when host is disabled

Test Result:
```
Before Fix:
mysql> select ohc.host_id, ohc.`capacity_state`,  case capacity_type  when 0 then  'MEMORY'  when 1 then  'CPU'  ELSE  'STORAGE'  END as 'capacity_type' ,  total_capacity, case capacity_type  when 0 then  'HOST'  when 1 then  'HOST' ELSE  'STORAGE POOL' END as 'HOST/STORAGE POOL'  from op_host_capacity ohc where host_id=3;
+---------+----------------+---------------+----------------+-------------------+
| host_id | capacity_state | capacity_type | total_capacity | HOST/STORAGE POOL |
+---------+----------------+---------------+----------------+-------------------+
|       3 | Enabled        | MEMORY        |     8589934592 | HOST              |
|       3 | Enabled        | CPU           |          32000 | HOST              |
|       3 | Enabled        | STORAGE       |  2199023255552 | STORAGE POOL      |
+---------+----------------+---------------+----------------+-------------------+

9 rows in set (0.00 sec)

Disable Host 3 from UI.

mysql> select ohc.host_id, ohc.`capacity_state`,  case capacity_type  when 0 then  'MEMORY'  when 1 then  'CPU'  ELSE  'STORAGE'  END as 'capacity_type' ,  total_capacity, case capacity_type  when 0 then  'HOST'  when 1 then  'HOST' ELSE  'STORAGE POOL' END as 'HOST/STORAGE POOL'  from op_host_capacity ohc where host_id=3;
+---------+----------------+---------------+----------------+-------------------+
| host_id | capacity_state | capacity_type | total_capacity | HOST/STORAGE POOL |
+---------+----------------+---------------+----------------+-------------------+
|       3 | Disabled       | MEMORY        |     8589934592 | HOST              |
|       3 | Disabled       | CPU           |          32000 | HOST              |
|       3 | Disabled       | STORAGE       |  2199023255552 | STORAGE POOL      |
+---------+----------------+---------------+----------------+-------------------+

After Fix:

mysql> select ohc.host_id, ohc.`capacity_state`,  case capacity_type  when 0 then  'MEMORY'  when 1 then  'CPU'  ELSE  'STORAGE'  END as 'capacity_type' ,  total_capacity, case capacity_type  when 0 then  'HOST'  when 1 then  'HOST' ELSE  'STORAGE POOL' END as 'HOST/STORAGE POOL'  from op_host_capacity ohc where host_id=3;
+---------+----------------+---------------+----------------+-------------------+
| host_id | capacity_state | capacity_type | total_capacity | HOST/STORAGE POOL |
+---------+----------------+---------------+----------------+-------------------+
|       3 | Enabled        | MEMORY        |     8589934592 | HOST              |
|       3 | Enabled        | CPU           |          32000 | HOST              |
|       3 | Enabled        | STORAGE       |  2199023255552 | STORAGE POOL      |
+---------+----------------+---------------+----------------+-------------------+
3 rows in set (0.01 sec)

Disable Host 3 from UI.

mysql> select ohc.host_id, ohc.`capacity_state`,  case capacity_type  when 0 then  'MEMORY'  when 1 then  'CPU'  ELSE  'STORAGE'  END as 'capacity_type' ,  total_capacity, case capacity_type  when 0 then  'HOST'  when 1 then  'HOST' ELSE  'STORAGE POOL' END as 'HOST/STORAGE POOL'  from op_host_capacity ohc where host_id=3;
+---------+----------------+---------------+----------------+-------------------+
| host_id | capacity_state | capacity_type | total_capacity | HOST/STORAGE POOL |
+---------+----------------+---------------+----------------+-------------------+
|       3 | Disabled       | MEMORY        |     8589934592 | HOST              |
|       3 | Disabled       | CPU           |          32000 | HOST              |
|       3 | Enabled        | STORAGE       |  2199023255552 | STORAGE POOL      |
+---------+----------------+---------------+----------------+-------------------+
3 rows in set (0.00 sec)

Sudhansus-MAC:cloudstack sudhansu$  nosetests-2.7 --with-marvin --marvin-config=setup/dev/advanced.cfg test/integration/component/maint/test_capacity_host_delete.py

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp//MarvinLogs//Apr_22_2016_22_42_27_X4VBWD. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
===final results are now copied to: /tmp//MarvinLogs/test_capacity_host_delete_9RHSNB===
Sudhansus-MAC:cloudstack sudhansu$ cat /tmp//MarvinLogs/test_capacity_host_delete_9RHSNB/results.txt
test_01_op_host_capacity_disable_host (integration.component.maint.test_capacity_host_delete.TestHosts) ... === TestName: test_01_op_host_capacity_disable_host | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 1 test in 0.168s

OK
```

* pr/1516:
  CLOUDSTACK-9366: Capacity of one zone-wide primary storage ignored

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-20 08:31:46 -04:00
Sudhansu
18a6aa89be CLOUDSTACK-9366: Capacity of one zone-wide primary storage ignored
introduced new capacityType parameter in updateCapacityState method and necessary changes to add capacity_type clause in sql
also fixed incorrect sql builder logic (unused code path for which it is never surfaced )
Added marvin test to  check host and storagepool capacity when host is disabled
Added conditions to ensure the capacity_type is added only when capacity_type length is greater than 0.
Added checks in marvin test to ensure the capacity exists for a host before disabling it.
Added  checks to avoid index out of range exception
2016-05-19 20:24:04 +05:30
Rohit Yadav
1770f3cb86 dynamic-roles: remove unnecessary order by ID
On some MySQL server envs, this may cause a SQL statement error, though
I was unable to reproduce it. Since it's not needed, an order by 'sort_order'
is enough, we can safely remove it.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-18 13:46:18 +05:30
Mike Tutkowski
9d215562eb Faster logic to see if a cluster supports resigning 2016-05-16 07:18:39 -06:00
Mike Tutkowski
2bd035d199 Support for backend snapshots with XenServer 2016-05-13 01:02:04 -06:00
Will Stevens
7e41747c33 Merge pull request #816 from mike-tutkowski/addremovehosts2
Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster

This PR addresses the following JIRA ticket:

https://issues.apache.org/jira/browse/CLOUDSTACK-8813

The problem is that there needs to be notifications sent when a host is added to, about to be removed from, and removed from a cluster.

Such notifications can be used for many purposes. For example, it can allow storage plug-ins to update ACLs on their storage systems. Also, it can allow us to clean up IQNs from ESXi hosts that are no longer needed.

* pr/816:
  CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-05-12 11:07:36 -04:00
Rohit Yadav
07564469e9 CLOUDSTACK-9299: Out-of-band Management for CloudStack
Support access to a host’s out-of-band management interface (e.g. IPMI, iLO,
DRAC, etc.) to manage host power operations (on/off etc.) and querying current
power state in CloudStack.

Given the wide range of out-of-band management interfaces such as iLO and iDRA,
the service implementation allows for development of separate drivers as plugins.
This feature comes with a ipmitool based driver that uses the
ipmitool (http://linux.die.net/man/1/ipmitool) to communicate with any
out-of-band management interface that support IPMI 2.0.

This feature allows following common use-cases:
- Restarting stalled/failed hosts
- Powering off under-utilised hosts
- Powering on hosts for provisioning or to increase capacity
- Allowing system administrators to see the current power state of the host

For testing this feature `ipmisim` can be used:
https://pypi.python.org/pypi/ipmisim

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Out-of-band+Management+for+CloudStack

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-12 11:05:40 +05:30
Mike Tutkowski
dad9e5d868 CLOUDSTACK-8813: Notify listeners when a host has been added to a cluster, is about to be removed from a cluster, or has been removed from a cluster 2016-05-11 08:02:46 -06:00
Rohit Yadav
4347776ac6 CLOUDSTACK-8562: DB-Backed Dynamic Role Based API Access Checker
This feature allows root administrators to define new roles and associate API
permissions to them.

A limited form of role-based access control for the CloudStack management server
API is provided through a properties file, commands.properties, embedded in the
WAR distribution. Therefore, customizing API permissions requires unpacking the
distribution and modifying this file consistently on all servers. The old system
also does not permit the specification of additional roles.

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+Role+Based+API+Access+Checker+for+CloudStack

DB-Backed Dynamic Role Based API Access Checker for CloudStack brings following
changes, features and use-cases:
- Moves the API access definitions from commands.properties to the mgmt server DB
- Allows defining custom roles (such as a read-only ROOT admin) beyond the
  current set of four (4) roles
- All roles will resolve to one of the four known roles types (Admin, Resource
  Admin, Domain Admin and User) which maintains this association by requiring
  all new defined roles to specify a role type.
- Allows changes to roles and API permissions per role at runtime including additions or
  removal of roles and/or modifications of permissions, without the need
  of restarting management server(s)

Upgrade/installation notes:
- The feature will be enabled by default for new installations, existing
  deployments will continue to use the older static role based api access checker
  with an option to enable this feature
- During fresh installation or upgrade, the upgrade paths will add four default
  roles based on the four default role types
- For ease of migration, at the time of upgrade commands.properties will be used
  to add existing set of permissions to the default roles. cloud.account
  will have a new role_id column which will be populated based on default roles
  as well

Dynamic-roles migration tool: scripts/util/migrate-dynamicroles.py
- Allows admins to migrate to the dynamic role based checker at a future date
- Performs a harder one-way migrate and update
- Migrates rules from existing commands.properties file into db and deprecates it
- Enables an internal hidden switch to enable dynamic role based checker feature

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-11 09:45:19 +05:30
Rohit Yadav
c63ea0a240 engine/schema: fix upgrade path to work with MySQL 5.7
Found this issue when using MySQL 5.7 with Ubuntu 16.04 with following settings:

sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
server-id   = 1
innodb_rollback_on_timeout=1
innodb_lock_wait_timeout=600
max_connections=350
log-bin=mysql-bin
binlog-format = 'ROW'

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-25 14:47:22 +05:30
Remi Bergsma
e1ecfad322 Implement upgrade path 4.8.1 -> 4.9.0 2016-01-26 15:21:23 +01:00
Remi Bergsma
43ab98d823 Updating pom.xml version numbers for release 4.9.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 15:12:20 +01:00
Remi Bergsma
95331aad30 Implement upgrade path 4.8.0 -> 4.8.1 2016-01-26 13:42:53 +01:00
Remi Bergsma
32fcc47117 Updating pom.xml version numbers for release 4.8.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-26 09:39:00 +01:00
Remi Bergsma
62f218b7bd Updating pom.xml version numbers for release 4.8.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-20 23:43:35 +01:00
Remi Bergsma
75b68c6829 Merge release branch 4.7 to master
* 4.7:
  Fix unable to setup more than one Site2Site VPN Connection
  FIX S2S VPN rVPC: Check only redundant routers in state MASTER
  PEP8 of integration/smoke/test_vpc_vpn
  Add S2S VPN test for Redundant VPC
  Make integration/smoke/test_vpc_vpn Hypervisor independant
  FIX VPN: non-working ipsec commands
  [UI] MADNESS
  [DB] Add force_encap field to s2s_customer_gateway table
  [ROUTER] Add forceencaps field to python router ipsec config method
  [TEST] unittest needs rework
  [MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
  [CORE] Add Force UDP Encapsulation option to Site2Site VPN
  CLOUDSTACK-9186: Root admin cannot see VPC created by Domain admin user
  CLOUDSTACK-9192: UpdateVpnCustomerGateway is failing
  CLOUDSTACK-6485 prevent ip asignment of private gw iface
  CLOUDSTACK-9204 Do not error when staticroute is already gone
  make both check lines consistent
  CLOUDSTACK-9181 Prevent syntax error in checkrouter.sh
  CLOUDSTACK-9202 Bump ssh timeout
2016-01-16 19:54:41 +01:00
Remi Bergsma
55667896d0 Merge pull request #1317 from michaelandersen/vpn/forceencap
[4.7] ADD Force UDP encapsulation option to Site2Site VPNThis PR adds the option to enable forced UDP encapsulation of ESP packets during a setup of a site2site vpn. This options enforces the 'forceencaps' option in the openswan ipsec config:
https://wiki.strongswan.org/projects/strongswan/wiki/ConnSection

* pr/1317:
  [UI] MADNESS
  [DB] Add force_encap field to s2s_customer_gateway table
  [ROUTER] Add forceencaps field to python router ipsec config method
  [TEST] unittest needs rework
  [MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
  [CORE] Add Force UDP Encapsulation option to Site2Site VPN

Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-16 19:38:42 +01:00
Remi Bergsma
b1034ed624 Merge release branch 4.7 to master
* 4.7:
  CLOUDSTACK-9220 Sort list of domains on Domain tab in UI
  Admin cannot see VMs on port forwarding page
  Fix mariadb related listCapacity bug (CLOUDSTACK-8966)
  CLOUDSTACK-9213 - Split the ACL rules using comma instead of dash.
  CLOUDSTACK-9213 - Formatting the code
2016-01-11 16:22:31 +01:00
Michael Andersen
6da3bc1237 [CORE] Add Force UDP Encapsulation option to Site2Site VPN 2016-01-07 19:27:37 +01:00
Anton Opgenoort
65d39d0346 Fix mariadb related listCapacity bug (CLOUDSTACK-8966)
type bigint(20) with type  varchar does not work well on MariaDB
So forcing it to type decimal
2016-01-07 10:21:13 +01:00
nvazquez
c67637180f CLOUDSTACK-9074: Support Shared Networks in NiciraNVP Plugin 2015-12-24 16:07:06 -03:00
nvazquez
8149081658 CLOUDSTACK-9074: API Changes: Add nsxLogicalSwitch and nsxLogicalSwitchPort to listNics 2015-12-24 16:07:06 -03:00
Remi Bergsma
18fb10439b Implement upgrade path 4.7.1 -> 4.8.0 2015-12-21 23:02:43 +01:00
Remi Bergsma
8f5a2920e8 Updating pom.xml version numbers for release 4.8.0-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 22:09:31 +01:00
Remi Bergsma
9537f5d32b Merge release branch '4.7' into master
* 4.7:
  Debian changelog updated
  Implement upgrade path 4.7.0 -> 4.7.1
  Updating pom.xml version numbers for release 4.7.1-SNAPSHOT
  Implement upgrade path 4.6.2 -> 4.6.3
  Updating pom.xml version numbers for release 4.6.3-SNAPSHOT
2015-12-21 21:57:52 +01:00
Remi Bergsma
04c759390b Implement upgrade path 4.7.0 -> 4.7.1 2015-12-21 20:25:54 +01:00
Remi Bergsma
5b112408c7 Updating pom.xml version numbers for release 4.7.1-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-21 19:55:04 +01:00
Remi Bergsma
2f26a859a9 Updating pom.xml version numbers for release 4.7.0
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-13 21:09:53 +01:00
Daan Hoogland
50c20dab17 Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9134: set device_id as the first device_id not in use instead of nic count
2015-12-13 19:04:22 +01:00
Daan Hoogland
5774b965f3 Merge pull request #1209 from ustcweizhou/free-deviceid
CLOUDSTACK-9134: set device_id as the first device_id not in use instead of nic count
when we restart vpc tiers, the old nics will be removed, and create a new nic.
however, the device_id was set to the nic count, which may be already used.
this commit get the first device_id not in use as the device_id of new nic.

This issue also happen when we add multiple networks to a vm and remove them.

* pr/1209:
  CLOUDSTACK-9134: set device_id as the first device_id not in use instead of nic count

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-13 18:43:30 +01:00
Remi Bergsma
a7b098ff16 Implement 4.6.1 -> 4.6.2 upgrade path 2015-12-13 00:06:02 +01:00
Daan Hoogland
22a5621476 4.6.2 -> 4.7.0 upgrade does not use any scripts above 4.6.1 2015-12-12 21:51:14 +01:00
Remi Bergsma
5147dec4ff Updating pom.xml version numbers for release 4.6.2-SNAPSHOT
Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-12 21:49:37 +01:00
Wei Zhou
acfc19dc82 CLOUDSTACK-9134: set device_id as the first device_id not in use instead of nic count
when we restart vpc tiers, the old nics will be removed, and create a new nic.
however, the device_id was set to the nic count, which may be already used.
this commit get the first device_id not in use as the device_id of new nic.

This issue also happen when we add multiple networks to a vm and remove them.
2015-12-10 14:02:02 +01:00
Abhinandan Prateek
987fcbd441 CLOUDSTACK-8592: Implement Quota service
Quota service while allowing for scalability will make sure that the cloud is
not exploited by attacks, careless use and program errors. To address this
problem, we propose to employ a quota-enforcement service that allows resource
usage within certain bounds as defined by policies and available quotas for
various entities.  Quota service extends the functionality of usage server to
provide a measurement for the resources used by the accounts and domains using a
common unit referred to as cloud currency in this document. It can be configured
to ensure that your usage won’t exceed the budget allocated to accounts/domain
in cloud currency.  It will let user know how much of the cloud resources he is
using. It will help the cloud admins, if they want, to ensure that a user does
not go beyond his allocated quota. Per usage cycle if a account is found to be
exceeding its quota then it is locked. Locking an account means that it will not
be able to initiat e a new resource allocation request, whether it is more
storage or an additional ip. Needless to say quota service as well as any action
on the account is configurable.

Changes from Github code review:

- Added marvin test for quota plugin API
- removed unused commented code
- debug messages in debug enabled check
- checks for nulls, fixed access to member variables and feature
- changes based on PR comments
- unit tests for UsageTypes
- unit tests for all Cmd classes
- unit tests for all service and manager impls
- try-catch-finally or try-with-resource in dao impls for failsafe db switching
- remove dead code
- add missing quota calculation case (regression fixed)
- replace tabs with spaces in pom.xmls
- quota: though default value for quota_calculated is 0, the usage server
  makes it null while entering usage entries. Flipping the condition so
  as to acocunt for that.
- quotatypes: fix NPE in quota type
- quota framework test fixes
- made statement period configurable
- changed default email templates to reflect the fact that exhausted quota may not result in a locked account
- added quotaUpdateCmd that refreshes quota balances and sends alerts and statements
- report quotaSummary command returns quota balance, quota usage and state for all account
- made UI framework changes to allow for text area input in edit views
- process usage entries that have greater than 0 usage
- orocess quota entries only if tariff is non zero
- if there are credit entries but no balance entry create a dummy balance entry
- remove any credit entries that are before the last balance entry
  when displaying balance statement
- on a rerun the last balance is now getting added

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS
PR: https://github.com/apache/cloudstack/pull/768

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-12-07 23:02:48 +05:30
Daan Hoogland
638f1cf091 Merge pull request #1049 from DaanHoogland/CLOUDSTACK-9047
CLOUDSTACK-9047 rename enumsmake enums adhere to best practice naming conventions

* pr/1049:
  CLOUDSTACK-9046 rename enums to adhere to naming conventions
  CLOUDSTACK-9046 renamed enums in kvm plugin
  CLOUDSTACK-9047 use 'State's only with context   there are more types called 'State'   (or to be called so but now 'state')   So remove imports and prepend their enclosing class/context to them.

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-06 20:17:04 +01:00
Daan Hoogland
3de117c8be 4.6.0 upgrade path is to pass 4.6.1 to create the extra view in there 2015-12-04 10:06:19 +01:00
Remi Bergsma
7e902cd505 Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9075 - Uses the same vlan since it should have been already released
  CLOUDSTACK-9075 - Adds VPC static routes test
  CLOUDSTACK-9075 - Covers Private GW ACL with Redundant VPCs
  CLOUDSTACK-9075 - Add method to get list of Physical Networks per zone
  CLOUDSTACK-6276 Removing unused parameter in integration test for projects
  CLOUDSTACK-6276 Removing unused parameter in integration test
  CLOUDSTACK-6276 Fixing affinity groups for projects
2015-12-03 20:42:41 +01:00
Remi Bergsma
9a21873c4a Merge pull request #1134 from pdube/CLOUDSTACK-6276
CLOUDSTACK-6276 Fixing affinity groups for projectsWith some contributions from @resmo and @ustcweizhou.
This closes https://github.com/apache/cloudstack/pull/508

To test manually (need at least 2 hosts):
Create a project
Create an affinity group in that project
Deploy a vm with that affinity group
Deploy a second vm with that affinity group
They should be on different hosts

Ran old and new tests for affinity groups on the simulator

Test create affinity group as admin in project ... === TestName: test_01_admin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as domain admin for projects ... === TestName: test_02_doadmin_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group as user for projects ... === TestName: test_03_user_create_aff_grp_for_project | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) for projects ... === TestName: test_4_user_create_aff_grp_existing_name_for_project | Status : SUCCESS ===
ok
#Delete Affinity Group by id. ... === TestName: test_01_delete_aff_grp_by_id | Status : SUCCESS ===
ok
#Delete Affinity Group by id should fail for user not in project ... === TestName: test_02_delete_aff_grp_by_id_another_user | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_01_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups with more vms than hosts. ... === TestName: test_02_deploy_vm_anti_affinity_group_fail_on_not_enough_hosts | Status : SUCCESS ===
ok
List affinity group for a vm for projects ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm for projects ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id for projects ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name for projects ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id for projects ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name for projects ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group for projects ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 16 tests in 581.706s

OK

Deploy vm as Admin in Affinity Group belonging to regular user (should fail) ... === TestName: test_01_deploy_vm_another_user | Status : SUCCESS ===
ok
Create Affinity Group as admin for regular user ... === TestName: test_02_create_aff_grp_user | Status : SUCCESS ===
ok
List Affinity Groups as admin for all the users ... === TestName: test_03_list_aff_grp_all_users | Status : SUCCESS ===
ok
List Affinity Groups belonging to admin user ... === TestName: test_04_list_all_admin_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing account id and domain id ... === TestName: test_05_list_all_users_aff_grp | Status : SUCCESS ===
ok
List Affinity Groups belonging to regular user passing group id ... === TestName: test_06_list_all_users_aff_grp_by_id | Status : SUCCESS ===
ok
Delete Affinity Group belonging to regular user ... === TestName: test_07_delete_aff_grp_of_other_user | Status : SUCCESS ===
ok
Test create affinity group as admin ... === TestName: test_01_admin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as domain admin ... === TestName: test_02_doadmin_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group as user ... === TestName: test_03_user_create_aff_grp | Status : SUCCESS ===
ok
Test create affinity group that exists (same name) ... === TestName: test_04_user_create_aff_grp_existing_name | Status : SUCCESS ===
ok
Test create affinity group with existing name but within different account ... === TestName: test_05_create_aff_grp_same_name_diff_acc | Status : SUCCESS ===
ok
Test create affinity group of non-existing type ... === TestName: test_06_create_aff_grp_nonexisting_type | Status : SUCCESS ===
ok
Delete Affinity Group by name ... === TestName: test_01_delete_aff_grp_by_name | Status : SUCCESS ===
ok
Delete Affinity Group as admin for an account ... === TestName: test_02_delete_aff_grp_for_acc | Status : SUCCESS ===
ok
Delete Affinity Group which has vms in it ... === TestName: test_03_delete_aff_grp_with_vms | Status : SUCCESS ===
ok
Delete Affinity Group with id which does not belong to this user ... === TestName: test_05_delete_aff_grp_id | Status : SUCCESS ===
ok
Delete Affinity Group by name which does not belong to this user ... === TestName: test_06_delete_aff_grp_name | Status : SUCCESS ===
ok
Delete Affinity Group by id. ... === TestName: test_08_delete_aff_grp_by_id | Status : SUCCESS ===
ok
Root admin should be able to delete affinity group of other users ... === TestName: test_09_delete_aff_grp_root_admin | Status : SUCCESS ===
ok
Deploy VM without affinity group ... === TestName: test_01_deploy_vm_without_aff_grp | Status : SUCCESS ===
ok
Deploy VM by aff grp name ... === TestName: test_02_deploy_vm_by_aff_grp_name | Status : SUCCESS ===
ok
Deploy VM by aff grp id ... === TestName: test_03_deploy_vm_by_aff_grp_id | Status : SUCCESS ===
ok
test DeployVM in anti-affinity groups ... === TestName: test_04_deploy_vm_anti_affinity_group | Status : SUCCESS ===
ok
Deploy vms by affinity group id ... === TestName: test_05_deploy_vm_by_id | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by name ... === TestName: test_06_deploy_vm_aff_grp_of_other_user_by_name | Status : SUCCESS ===
ok
Deploy vm in affinity group of another user by id ... === TestName: test_07_deploy_vm_aff_grp_of_other_user_by_id | Status : SUCCESS ===
ok
Deploy vm in multiple affinity groups ... === TestName: test_08_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy multiple vms in multiple affinity groups ... === TestName: test_09_deploy_vm_multiple_aff_grps | Status : SUCCESS ===
ok
Deploy VM by aff grp name and id ... === TestName: test_10_deploy_vm_by_aff_grp_name_and_id | Status : SUCCESS ===
ok
List affinity group for a vm ... === TestName: test_01_list_aff_grps_for_vm | Status : SUCCESS ===
ok
List multiple affinity groups associated with a vm ... === TestName: test_02_list_multiple_aff_grps_for_vm | Status : SUCCESS ===
ok
List affinity groups by id ... === TestName: test_03_list_aff_grps_by_id | Status : SUCCESS ===
ok
List Affinity Groups by name ... === TestName: test_04_list_aff_grps_by_name | Status : SUCCESS ===
ok
List Affinity Groups by non-existing id ... === TestName: test_05_list_aff_grps_by_non_existing_id | Status : SUCCESS ===
ok
List Affinity Groups by non-existing name ... === TestName: test_06_list_aff_grps_by_non_existing_name | Status : SUCCESS ===
ok
List affinity group should list all for a vms associated with that group ... === TestName: test_07_list_all_vms_in_aff_grp | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupids ... === TestName: test_01_update_aff_grp_by_ids | Status : SUCCESS ===
ok
Update the list of affinityGroups by using affinity groupnames ... === TestName: test_02_update_aff_grp_by_names | Status : SUCCESS ===
ok
Update the list of affinityGroups for vm which is not associated ... === TestName: test_03_update_aff_grp_for_vm_with_no_aff_grp | Status : SUCCESS ===
ok
Update the list of Affinity Groups to empty list ... SKIP: Skip - Failing - work in progress
Update the list of Affinity Groups on running vm ... === TestName: test_05_update_aff_grp_on_running_vm | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 42 tests in 976.432s

OK (SKIP=1)

* pr/1134:
  CLOUDSTACK-6276 Removing unused parameter in integration test for projects
  CLOUDSTACK-6276 Removing unused parameter in integration test
  CLOUDSTACK-6276 Fixing affinity groups for projects

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-03 20:10:16 +01:00
Daan Hoogland
f9775de8ff Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9022: move storage.cleanup related global configurations to StorageManager
  CLOUDSTACK-9022: keep Destroyed volumes for sometime

 Conflicts:
	server/src/com/cloud/storage/StorageManagerImpl.java
2015-12-03 10:35:00 +01:00
Wei Zhou
9077c9a5b4 CLOUDSTACK-9022: keep Destroyed volumes for sometime 2015-11-30 20:43:13 +01:00
Remi Bergsma
a1d2c531f6 Merge pull request #1007 from ustcweizhou/dedicated-ip-for-domain
[4.7] CLOUDSTACK-8958: add dedicated ips to domain (account for now)For now, we dedicate ip pool to account, however, other accounts in the same domain cannot fetch the ip from this ip pool.
By dedicating ip pool to domain, accounts in the domain can fetch the public ip from same ip pool.

* pr/1007:
  CLOUDSTACK-8958: throw an exception if project account cannot be found
  CLOUDSTACK-8958: add dedicated ips to domain (account for now)

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-29 09:39:25 +01:00
Patrick Dube
c76d317150 CLOUDSTACK-6276 Fixing affinity groups for projects 2015-11-27 14:43:02 -05:00