2530 Commits

Author SHA1 Message Date
Wei Zhou
92344c006d CLOUDSTACK-5863: revert volume snapshot for KVM/QCOW2 2015-08-24 11:01:50 +02:00
Rajani Karuturi
e384f2d146 Merge pull request #715 from devdeep/CLOUDSTACK-8687
CLOUDSTACK-8687: Prepare template only on a given storage poolUpdate prepare template api to seed/prepare a template only on a given primary storage. Currently, the prepare template api will seed/prepare a given template on all the primary storage pools in a zone. If however, a user wishes to prepare a template only a particular storage pool, it isn't possible. Updated the api to take storage pool id as an optional parameter. If the pool id is provided then the template is prepared only on the given primary storage pool. Also added unit tests to validate the functionality

* pr/715:
  CLOUDSTACK-8687: Unit tests for validating the prepare template functionality. These tests validate that the templates get scheduled for seeding. Additionally, if a template is already seeded, we do not try and seed it again. Tests also validate that templates are seeded to storage pools which are available.
  CLOUDSTACK-8687: Update prepare template api to seed/prepare a template only on a given primary storage. Currently, the prepare template api will seed/prepare a given template on all the primary storage pools in a zone. If however, a user wishes to prepare a template only a particular storage pool, it isn't possible. Updated the api to take storage pool id as an optional parameter. If the pool id is provided then the template is prepared only on the given primary storage pool

Signed-off-by: Rajani Karuturi <rajanikaruturi@gmail.com>
2015-08-19 15:58:42 +05:30
Devdeep Singh
adf6b588dd CLOUDSTACK-8687: Update prepare template api to seed/prepare a template
only on a given primary storage. Currently, the prepare template api
will seed/prepare a given template on all the primary storage pools in
a zone. If however, a user wishes to prepare a template only a
particular storage pool, it isn't possible. Updated the api to take
storage pool id as an optional parameter. If the pool id is provided
then the template is prepared only on the given primary storage pool
2015-08-19 10:04:05 +05:30
Boris Schrijver
bb819ca222 Renamed internal variables of NicProfile.java to adhere to the naming convention.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
2015-08-18 14:05:00 +02:00
Boris Schrijver
c30ba1df0b Refactored Nic.java for readability.
Changed methodnames according to Nic.java refactor.

Fixed NicVO.java due to regression from Nic.java refactor.

Fixed VmWareGuru.java after Nic.java refactor.

See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
2015-08-18 14:04:48 +02:00
Boris Schrijver
5db3371840 Refactored NicProfile.java for readability.
See issue CLOUDSTACK-8736 for ongoing effort to clean up network code.
2015-08-17 14:12:10 +02:00
Mike Tutkowski
8b0266d12e Merge branch 'pr/547'
* pr/547:
  CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.

Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
2015-08-10 19:00:53 -06:00
Wido den Hollander
178a9f555d CLOUDSTACK-8581: S3, make connection TTL and TCP KeepAlive configureable
Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-07-17 10:07:04 +02:00
Jayapal
140cd7c227 CLOUDSTACK-8324: Fixed config key in multiple places 2015-07-10 09:47:09 +05:30
Likitha Shetty
13a98dd196 CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage.
Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
2015-07-01 10:47:36 +05:30
Rohit Yadav
107595a6a5 CLOUDSTACK-8457: SAML auth plugin improvements for production usage
* Move config options to SAML plugin
  This moves all configuration options from Config.java to SAML auth manager. This
  allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
  Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server

- CLOUDSTACK-8458:
    * On UI show dropdown list of discovered IdPs
    * Support SAML Federation, where there may be more than one IdP
        - New datastructure to hold metadata of SP or IdP
        - Recursive processing of IdP metadata
        - Fix login/logout APIs to get new interface and metadata data structure
        - Add org/contact information to metadata
        - Add new API: listIdps that returns list of all discovered IdPs
        - Refactor and cleanup code and tests

- CLOUDSTACK-8459:
    * Add HTTP-POST binding to SP metadata
    * Authn requests must use either HTTP POST/Artifact binding

- CLOUDSTACK-8461:
    * Use unspecified x509 cert as a fallback encryption/signing key
      In case a IDP's metadata does not clearly say if their certificates need to be
      used as signing or encryption and we don't find that, fallback to use the
      unspecified key itself.

- CLOUDSTACK-8462:
    * SAML Auth plugin should not do authorization
      This removes logic to create user if they don't exist. This strictly now
      assumes that users have been already created/imported/authorized by admins.
      As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
      either HTTP POST or HTTP Artifact binding to transfer the message through a
      user agent (browser in our case). The use of HTTP Redirect was one of the reasons
      why this plugin failed to work for some IdP servers that enforce this.
    * Add new User Source
      By reusing the source field, we can find if a user has been SAML enabled or not.
      The limitation is that, once say a user is imported by LDAP and then SAML
      enabled - they won't be able to use LDAP for authentication
    * UI should allow users to pass in domain they want to log into, though it is
      optional and needed only when a user has accounts across domains with same
      username and authorized IDP server
    * SAML users need to be authorized before they can authenticate
        - New column entity to track saml entity id for a user
        - Reusing source column to check if user is saml enabled or not
        - Add new source types, saml2 and saml2disabled
        - New table saml_token to solve the issue of multiple users across domains and
          to enforce security by tracking authn token and checking the samlresponse for
          the tokens
        - Implement API: authorizeSamlSso to enable/disable saml authentication for a
          user
        - Stubs to implement saml token flushing/expiry

- CLOUDSTACK-8463:
    * Use username attribute specified in global setting
      Use username attribute defined by admin from a global setting
      In case of encrypted assertion/attributes:
      - Decrypt them
      - Check signature if provided to check authenticity of message using IdP's
        public key and SP's private key
      - Loop through attributes to find the username

- CLOUDSTACK-8538:
    * Add new global config for SAML request sig algorithm

- CLOUDSTACK-8539:
    * Add metadata refresh timer task and token expiring
        - Fix domain path and save it to saml_tokens
        - Expire hour old saml tokens
        - Refresh metadata based on timer task
        - Fix unit tests

This closes #489

(cherry picked from commit 20ce346f3acb794b08a51841bab2188d426bf7dc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	client/WEB-INF/classes/resources/messages_hu.properties
	plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
	plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
	ui/scripts/ui-custom/login.js
2015-06-29 12:31:51 +02:00
wilderrodrigues
03dd34039a Decouple the use of updateHostPassword
- The code was hard to maintain because updating a host or all the hosts in a cluster was handled in the same method
   - Created updateHost and updateCluster password in both ResourceManager and ManagementServer interfaces/classes
   - The chck for whihc method to use is done in the API level
   - Started adding the support for KVM host passwd update

No API changes are needed and it will be backwards compatible.

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-06-29 09:59:10 +02:00
wilderrodrigues
4e6fd0f17f Formatting the code of the ManagementService class
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-06-29 09:59:09 +02:00
Daan Hoogland
f5847147c0 findbugs: serializable fields
Signed-off-by: Daan Hoogland <daan@onecht.net>

This closes #533
2015-06-26 15:11:16 +02:00
Daan Hoogland
55c1d9f3fa findbugs: fields of serializable made serializable
Signed-off-by: Daan Hoogland <daan@onecht.net>

This closes #522
2015-06-24 20:20:12 +02:00
Jayapal
69ea932897 CLOUDSTACK-8324: Config drive changes for xenserver 2015-05-28 15:51:50 +05:30
Jayapal
e407986183 CLOUDSTACK-8324: Added config drive support for xenserver 2015-05-28 15:51:50 +05:30
Jayapal
733ac2b728 CLOUDSTACK-8324: Added vm ip fetch logic 2015-05-28 15:42:08 +05:30
Jayapal
70934e8c1a CLOUDSTACK-8324: Skip ip allocation for external dhcp 2015-05-28 15:42:07 +05:30
Koushik Das
1cd2e9bc44 Merge branch 'CLOUDSTACK-8301' of https://github.com/apache/cloudstack 2015-05-22 09:47:59 +05:30
Rene Moser
c43e4db4e0 ListFirewallEgressRulesCmd: add interfaces and minor cleanup
This commit does not implement new functionality:

* Fixes duplicate parameter in API docs.
* Fixes a bunch of typos.
* Add interfaces to make it easier for the FirewallService interface.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #249
2015-05-20 17:00:08 +01:00
Koushik Das
3f7e31ed05 CLOUDSTACK-8301: Enable configuring local storage use for system VMs at zone level
Made system.vm.use.local.storage a zone level configuration.
2015-05-19 21:32:14 +05:30
Devdeep Singh
a99c9d0e68 Implementation for the ability to disable a storage pool for provisioning
... of new volumes. Following changes are implemented 1. Disable or enable a pool with the
updateStoragePool api. A new 'enabled' parameter added for the same. 2. When a
pool is disabled the state of the pool is updated to 'Disabled' in the db. On
enabling it is updated back to 'Up'. Alert is raised when a pool is disabled or
enabled. 3. Updated other storage providers to also honour the disabled state.
4. A disabled pool is skipped by allocators for provisioing of new volumes. 5.
Since the allocators skip a disabled pool for provisioning of volumes, the
volumes are also not listed as a destination for volume migration.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Disabling+Storage+Pool+for+Provisioning

This closes #257

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-05-19 11:16:49 +01:00
Rohit Yadav
1840805aab server: Introduce Unknown Status to be used in AbstractInvestigatorImpl
The PR #211 introduced changes where the abstract investigator testIpAddress()
would return other Status, which previously only returned null, Up or Down. In
this patch we introduce a new Status "Unknown" that replaces null's semantics.

The important changes #211 introduced was the debugging statements as semantically
the changes would work same as the consumers of testIpAddress() method only used
if returned values were Up or Down and in other cases (null, Alert etc) it would
simply continue to loop through the resources being investigated.

Keeping the debug logs, this commit only replaces the previously returned null
values with Status.Unknown and fixed the debug statements to reflect the same.
In case of trapped exceptions too, we return Unknown status but log the exception
we trapped.

server: add null assertions and remove dead code with testIpAddress usage

This closes #222

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 7a1cb28c9f548ac185dcb7c59eb2fadb7d550718)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-05-03 15:16:04 +02:00
Rohit Yadav
df3dea58a2 CLOUDSTACK-6139: Fix regression, allow zone level systemvm localstorage config
From b3f18e7d74a0f09db9977554a6c7648b7edbc33d, the zone level systemvm local
storage setting never worked as it needed to be moved to config depot.

(cherry picked from commit 279efb04324249a2e1a5487b58b40a723baf4600)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	plugins/network-elements/internal-loadbalancer/src/org/apache/cloudstack/network/lb/InternalLoadBalancerVMManagerImpl.java
	server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java
2015-05-01 15:27:40 +02:00
Anshul Gangwar
2133c302f4 CLOUDSTACK-8413: Fixed resource tags on disk are lost when migrate to another storage
During cold volume migration we are duplicating volume entry in volumes table.
When migration is complete, we update the uuid of new entry and expunge the older entry.
This results in removal of resource tags on volume as its resource id still pointing to older volume.
As part of fix while updating uuid for volume, we are updating resource_id for tags also.

This closes #194
2015-04-30 14:34:20 +05:30
Rajani Karuturi
0b8355920e Merge branch 'volume-upload' into master
This closes #206
2015-04-29 11:12:53 +05:30
Rajani Karuturi
d5bb2f16fc volume upload: added state transition for a failed volume delete 2015-04-16 17:42:18 +05:30
KC Wang
01864ef77c CLOUDSTACK-6697: bigswitch networking plugin update
1. provide compatibility with the Big Cloud Fabric (BCF) controller
   L2 Connectivity Service in both VPC and non-VPC modes
2. virtual network terminology updates: VNS --> BCF_SEGMENT
3. uses HTTPS with trust-always certificate handling
4. topology sync support with BCF controller
5. support multiple (two) BCF controllers with HA
6. support VM migration
7. support Firewall, Static NAT, and Source NAT with NAT enabled option
8. add VifDriver for Indigo Virtual Switch (IVS)

This closes #151

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-04-09 13:02:02 +05:30
wilderrodrigues
57d0931b49 Adding more tests to cover other possibilities concerning different versions of the XenServer
Adding a better algorithm in order to corver deeper inheritance form the command classes
Removing some unsued variables.
2015-03-31 16:49:48 +02:00
Daan Hoogland
6a821730ce Boolean -> boolean 2015-03-25 22:20:40 +01:00
Daan Hoogland
de4445845e trailing whitespace 2015-03-25 22:06:26 +01:00
Daan Hoogland
048212b117 findbugs: Boolen function should not return null 2015-03-25 21:46:47 +01:00
Rohit Yadav
3c429ee6b5 Merge remote-tracking branch 'sbp/feature/persisten-systemvm-redundant-vpc-REBASE'
This closes #118

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-17 15:44:57 +05:30
wilderrodrigues
e350f1e76f Turn a single VPC into a redundant VPC
- when restarting a VPC, the user can check the option Make Redundant in order to change
    the VPC offering and make it redundant
2015-03-16 11:39:57 +01:00
wilderrodrigues
88129adac1 Implement the new VPC restart
- behaves just like network restart: if clean up is checked, all routers destroyed; if 1 router is gone and no
    clean up, only 1 new router created.
2015-03-16 11:39:52 +01:00
wilderrodrigues
40eb579115 TK-3135 fixes for VPC restart
* removed the "is redundant" flag form the addVpcRouterToGuestNetwork() method
* removed the "is redundant" flag from the removeVpcRouterFromGuestNetwork() method
* changed the path of the master.py file in the keepalived.conf.temp file
* the call to routerDao.addRouterToGuestNetwork() in the VpcRouterDeploymentDefinition is not needed. That step will be performed once a VM is created
  - In addition, when restarting a VPC the routers will have the guest net configured, if any exists.

* Pushing the POM.xml as well, to use the old Jetty for now. Could not fix the logging problem. Will replace the POM with master version after VPC is done.
2015-03-16 11:39:17 +01:00
Antonio Fornie
09bd847040 Vpc redundancy enabled. Including Vpc & Vpc Offering creation. Marvin tests and UI. 2015-03-16 11:38:05 +01:00
Rajani Karuturi
843f6b1691 CLOUDSTACK-5236 : ability to identify where the user is from (ex. LDAP)
Added a source column to the user table.

Source now has only two values UNKNOWN,LDAP with UNKNOWN being the
default and is an enum is com.cloud.User.

When the source is UNKNOWN, the old method of authenticating against all
the available authenticators is used. If a source is available, only
that particular authenticator will be used.

added overloaded methods in AccountService to createUserAccount and
createUser with source specified.

(cherry picked from commit 5da733072e838c830720909ef5ba27ef6ce6d0b0)
2015-03-16 14:53:53 +05:30
Funs
c27c69438b hypervisors: add OVM3 plugin that supports OVM 3.2.1/3.3.x
This is a plugin that puts in ovm3 support ranging from 3.3.1 to 3.3.2. Basic
functionality is in here, advanced networking etc..

Snapshots only work when a VM is stopped now due to the semantics of OVM's raw
image implementation (so snapshots should work on a storage level underneath the
hypervisor shrug)

This closes #113

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-12 11:33:42 +05:30
Rohit Yadav
a75ff23131 CLOUDSTACK-8273: fix baremetal account creation
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit e000646790597cb6c245311f57761b41d23a4e34)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	plugins/hypervisors/baremetal/src/com/cloud/baremetal/manager/BaremetalVlanManagerImpl.java
2015-02-27 15:55:09 +05:30
Pierre-Yves Ritschard
1dc0611bcd ensure agent reconciliation triggers events
(cherry picked from commit 0ba75bcebde4e8a4773c7b31e5d8b88bf748a131)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-02-18 11:15:44 +05:30
Rohit Yadav
1a6df6f978 CLOUDSTACK-7908: Add user_id column to vm_instance table
Design Document:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Allow+VM+listing+by+User+ID

- Adds column to VMInstance DAO
- Adds column in vm_instance table
- Adds column in the UserVMJoinVO
- Adds default admin user which has UID = 2
- Adds migration path that sets user_id to first user of the accountId that
  owns the vm in vm_instance table
- Add arg on list VMs API to query by userId, add support in query layer
- Refactor VMInstanceVO and child classes to accept userId
- Add code to let service layer pass userId if loggedIn user belongs to same
  account as the owner executing an API call or use first user from owner account
- In case of CPVM and SSVM use system user ID
- Fix unit tests and spring injections

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-02-16 15:02:38 +05:30
Daan Hoogland
b586439c9f CLOUDSTACK-8197 create MinVRVersion config item 2015-02-03 13:18:15 +01:00
Koushik Das
6ff8752fd0 volume-upload: gc thread to clean partially uploaded files
Fixed multiple issues in the template code path
2015-01-31 17:25:19 +05:30
Daan Hoogland
ebd7a7be5a state description c&p fix
cherry-picked from 766f8216a0bc96343a074c66757872052dd7b233, because of merge conflicts
2015-01-24 13:54:29 +01:00
Rohit Yadav
f9a8111e17 CLOUDSTACK-8151: Add removeRawUsageRecords API to cleanup old cloud_usage data
Call removeRawUsageRecords with interval (> 0) and it will clean up cloud_usage
table by removing records older than interval no. of days from today (current date)
and in case it runs when the job exec time is near, it will fail alerting
user to try again after a 15 min window.

There is an issue with async job scheduler, if this API were async it tries
to search and remove job from cloud_usage.async_job table and fails which is
why this API is sync and extends BaseCmd.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-23 19:38:32 +05:30
Koushik Das
cd02ceb7a1 volume-upload: gc thread to clean partially uploaded files
Added state transitions from error/abandoned to destroyed
2015-01-19 18:01:29 +05:30
Koushik Das
053c160b0f volume upload: management server polling and upload status from agent
MS polling logic to query status for templates that are uploaded
2015-01-16 18:03:36 +05:30
Koushik Das
1f1c96d2ee volume upload: management server polling and upload status from agent
- Added config parameters for upload monitoring interval and operation timeout
- Some updates to the volume state machine
2015-01-16 15:28:17 +05:30