2765 Commits

Author SHA1 Message Date
Wido den Hollander
d1f76a2a84 CLOUDSTACK-8628: kvm: Disable Fencing when no NFS storage pools are present
On NFS we write a heartbeat, but without those we can not safely
fence off a host.

If we fence without knowing about a heartbeat we can cause a split-brain situation.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-07-17 12:33:11 +02:00
Wido den Hollander
4b61f5a17a CLOUDSTACK-8581: Make the code which initiates the S3TO object more readable
All arguments are now on a own line which makes the code easier to read

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-07-17 10:07:04 +02:00
Wido den Hollander
178a9f555d CLOUDSTACK-8581: S3, make connection TTL and TCP KeepAlive configureable
Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-07-17 10:07:04 +02:00
Likitha Shetty
f5ed824ea2 CLOUDSTACK-8598. CS reports volume migration as successful but the volume is not migrated in vCenter.. For the following disk operations - migration, snapshot creation, resize, detach and template creation, CS should do an exact disk match between volume path and vCenter disk name. If the exact matching fails to find a disk, CS should fall back to the old method of partial matching on a trimmed disk.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #543
2015-07-16 10:44:58 +02:00
Funs
ca68bb0ef7 xenserver: Improved message for peeples to actually see what the problem is.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #576
2015-07-15 18:24:39 +05:30
Daan Hoogland
e62d952375 coverity 1311706: to generic exeption handling on xapi call
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
2015-07-14 13:41:09 +02:00
Daan Hoogland
c53959513e findbugs: create wrapper to deal with static field
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>

This closes #573
2015-07-14 11:50:39 +02:00
Daan Hoogland
10853bfa8a adhere to naming conventions for enums
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>
2015-07-14 11:50:39 +02:00
Rohit Yadav
76a10351a5 CLOUDSTACK-8457: SAML UI enhancements
Add option to authorize SAML SSO for user when adding user. Appends a domain to
user/account name if global setting is enabled, useful in case of multiple IDP
server. By default the setting is set to false (keep as it is).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #575

(cherry picked from commit 7d11c7bc703be00c91ea267695a33756ebb14b57)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-07-14 10:20:30 +05:30
Wido den Hollander
63688e70a0 kvm: More verbose logging around storage pools in the Agent 2015-07-13 16:53:26 +02:00
Wido den Hollander
24f8e66e93 kvm: Write Libvirt instead of LibVirt 2015-07-13 16:22:33 +02:00
Wido den Hollander
7c37ba73a3 kvm: Add more verbose logging about what is happening with Libvirt connections 2015-07-13 16:21:41 +02:00
Rohit Yadav
42940a8828 CLOUDSTACK-8622: Reinstate working sessions in browser
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
  and the API parameters. In both cases, if either or both are passed they should
  match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
  to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user

Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
  admin, domain-admin and user

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #574
This closes #308
2015-07-10 17:25:07 +05:30
Jayapal
deb694c201 CLOUDSTACK-8324: Added migration changes for config drive 2015-07-10 09:39:27 +05:30
Daan Hoogland
dae2e05f3c coverity 1291768: file input stream not severe unless repetitive (property file usually aren't)
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-07-09 21:30:23 +02:00
Daan Hoogland
6376503fe3 coverity 1308118: more resources being opened only one closed on exception nothing was closed
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-07-09 21:30:23 +02:00
Daan Hoogland
bcc8182f2f findbugs: write to static from instance these are the trivial cases of ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD as reported by findbugs
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>

This closes #539
2015-07-09 15:01:58 +02:00
weingartner
25e9918c57 Amend last commit, used 4 spaces in code formatting instead of 1 tab character.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #560
2015-07-09 13:37:59 +02:00
weingartner
50c4836c9f Cleaned class “com.cloud.hypervisor.xenserver.resource.XcpOssResource” that seemed to be forgotten during the evolution of the ACS. It was removed a few methods that were already coded properly in its parent class “com.cloud.hypervisor.xenserver.resource.CitrixResourceBase”. It was also removed some methods that seemed to cause weird behaviors. The methods removed/fixed are detailed as follows: • com.cloud.hypervisor.xenserver.resource.XcpOssResource.fillHostInfo(Connection, StartupRoutingCommand) – it was removed, because it always added the string “, hvm” to the host capabilities. Therefore, if one uses XCP hypervisor it could cause a lot of trouble when deploying HVM virtual machines in an environment that has PV and HVM clusters. The method is already properly coded in parent class. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.launchHeartBeat(Connection) – It was removed. It was not performing anything and always returns a true value. The method of parent class is properly coded and works for XCP environments. The heartbeat plugin exists in XCP environment. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.initializeLocalSR(Connection) – it was removed. The method of the parent class works properly for XCP environments. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.createPatchVbd(Connection, String, VM) – It was removed. This method causes a bug in XCP environments, because of its half-implementation, it was not possible to migrate system VMs. The parent class implementation works properly for XCP. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.execute(NetworkUsageCommand) – removed, hence it was already coded into parent class and its respective wrappers (“com.cloud.hypervisor.xenserver.resource.wrapper.xcp.XcpServerNetworkUsageCommandWrapper”). BTW: I noticed that the class XcpServerNetworkUsageCommandWrapper and XenServer56NetworkUsageCommandWrapper are almost the same, with the exception that XenServer56NetworkUsageCommandWrapper deals with VPC. I believe that those wrappers could be converted into one, and moved to parent. I am not doing that here because I do not have a XCP environment with advanced networking to test it. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.executeRequest(Command) – removed, hence it is not needed anymor. • com.cloud.hypervisor.xenserver.resource.XcpOssResource.execute(StopCommand) – I did not understand that method. It seemed weird and its removal did not change any behavior of the environment I tested it with.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-09 13:37:59 +02:00
Daan Hoogland
31c6b14dd4 Merge branch 'improvement/CLOUDSTACK-8607' change the host password on the host itself via the updateHostPassword API
- Make sure the new password replaces the old one in the queue
    - Updated the patch files for XenServer
    - Updated the script path on LibvirtComputing class
    - Adding update_host_passwd to VRScripts
    - Add implementation to CitrixUpdateHostPasswordCommandWrapper
    - Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
    - Adding update_host_passwd.sh script
    - Adding the host IP address as an instance variable on UpdateHostPasswordCommand
    - Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
    - Make sure doUpdateHostPassword() doesn't get called if flag is set to false
    - Do not update XenServer hosts if the cluster ID is not informed

    This closes #559
2015-07-06 16:15:22 +02:00
wilderrodrigues
e348e76eee CLOUDSTACK-8607 - Improving unit tests
- Adding more unit tests as suggested by Daan and Rajani.
   - Mocking SshHelper and the Script classes, used by Citrix and Libvirt resources respectively.
2015-07-06 14:58:17 +02:00
Rajani Karuturi
f1a64902cc Merge branch 'PR-REVIEW-335'
This closes #335
2015-07-06 11:48:38 +05:30
wilderrodrigues
86297e70be CLOUDSTACK-8607 - Make sure the new password replaces the old one in the queue
- Added log info to show details of the operation
   - Renamed the addPwdToQueue to replaceOldPasswdInQueue
2015-07-04 11:33:52 +02:00
wilderrodrigues
efa34361df CLOUDSTACK-8607 - Changed update script to return exit code based on the result
- Changed location of the update_host_passwd script
   - Updated the patch files for XenServer
   - Updated the script path on LibvirtComputing class
   - Removed the hostIP from the LibvirtUpdateHostPasswordCommandWrapper execute() method
2015-07-03 14:29:57 +02:00
Kishan Kavala
3a19a8916e BUG-ID:CLOUDSTACK-7472: kvmclock fix for LXC is not required after fixing CLOUDSTACK-8177
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #557
2015-07-03 15:21:18 +05:30
wilderrodrigues
0dd02ce043 CLOUDSTACK-8607 - Adding support to update host passwd on XenServer hypervisors
- Adding update_host_passwd to VRScripts
   - Add accessor method to host password on CitrixResourceBase
   - Add implementation to CitrixUpdateHostPasswordCommandWrapper
   - Improve testUpdateHostPasswordCommand() unit test on CitrixRequestWrapperTest
   - Add line to patch files on xenserver directory

Concerning the LibVirt change:

   - I forgot to assing the return of the getDefaultHypervisorScriptsDir() method to the hypervisorScriptsDir variable
2015-07-03 10:24:44 +02:00
Sudhansu
b593657503 BUG-ID: CLOUDSTACK-8484 - Hosts without tag are not listed while
listing the hosts for migration for instance with tag

While preparing the suitable hosts we are accidentally removing the
incompatible (host does not have host tag) hosts from otherhost list(
incorrect use of List.retainAll).
2015-07-03 11:41:55 +05:30
wilderrodrigues
47c7a1083f CLOUDSTACK-8607 - Adding update_host_passwd.sh script
- Modifying the LibvirtUpdateHostPasswordCommandWrapper in order to execute the script on the host
   - Adding the script path to LibvirtComputingResource
   - Adding the host IP address as an instance variable on UpdateHostPasswordCommand
   - Improving the Unit Test (LibvirtComputingResourceTest) to get it covering the new code
2015-07-02 14:54:51 +02:00
wilderrodrigues
96ad6f6ccd CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Change the retry() method to return Answer.createUnsupportedCommandAnswer(command) instead of throwing an exception

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #546
2015-07-01 15:04:58 +02:00
wilderrodrigues
c6013303e9 CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Increasing test coverage
     - resource package: 86.7%
     - utils package: 95.1%
     - wrapper package: 97.5%

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:58 +02:00
wilderrodrigues
644458f5ac CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Adding remaining command wrappers

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:58 +02:00
wilderrodrigues
b54126918b CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Adding NiciraNvpCreateLogicalRouterCommandWrapper
   - Adding NiciraNvpDeleteLogicalSwitchPortCommandWrapper
   - Adding NiciraNvpFindLogicalSwitchPortCommandWrapper
   - Adding NiciraNvpUpdateLogicalSwitchPortCommandWrapper
   - Decoupling private methods from NiciraNvpResource

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:57 +02:00
wilderrodrigues
b4ce81ab6c CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Adding NiciraNvpDeleteLogicalSwitchPortCommandWrapper

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:57 +02:00
wilderrodrigues
361ab5ddbd CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Adding NiciraNvpCreateLogicalSwitchPortCommandWrapper
   - Removing unsued field from NiciraNvpResourceTest

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:57 +02:00
wilderrodrigues
28c1da969d CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Adding command wrappers NiciraNvpCreateLogicalSwitchCommandWrapper and NiciraNvpDeleteLogicalSwitchCommandWrapper
   - Refactoring the retry mechanism
   - Applying the new retry mechanism to current wrappers and old methods in NiciraNvpResource
   - Adding 2 tests
   - Fixing the testRetries() in NiciraNvpResourceTest class

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:57 +02:00
wilderrodrigues
27c9651b32 CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Refactoring NiciraNvpResource
   - Added NiciraNvpUtilities and NiciraNvpCreateLogicalSwitchCommandWrapper
   - Removing 1 execute methods form NiciraNvpResource
   - Added 1 unit test

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:56 +02:00
wilderrodrigues
6a51294d19 CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Refactoring NiciraNvpResource
   - Added NiciraNvpMaintainCommandWrapper
   - Removing 1 execute methods form NiciraNvpResource
   - Added 1 unit test

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:56 +02:00
wilderrodrigues
0d943d5426 CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Refactoring NiciraNvpResource
   - Added NiciraNvpRequestWrapper
   - Removing 1 execute methods form NiciraNvpResource
   - Added 1 unit test

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:56 +02:00
wilderrodrigues
fc478604e8 CLOUDSTACK-8590 - Refactoring NiciraNVP resource
- Adding the NiciraNvpWrapper
  - This class will keep track of all Wrappers of the Nicira NVP Plugin

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-01 15:04:56 +02:00
Likitha Shetty
9463356ca6 CLOUDSTACK-8600. Upon detaching a disk, delete VM folder if empty.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #545
2015-07-01 14:53:23 +02:00
jeff
93b201d43a Allow custom command role ACL files on classpath in Static Role API Checker.
This commit has a small refactoring of cloud-plugin-acl-static-role-based
to allow it to read files on the classpath that might have a different name
than "commands.properties". It also allows more than one file to be read from.

Rationale: Third-party plugins may want to keep their API command access level
configuration separate from the main file so as to reduce configuration
maintenance work during packaging and deployments.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #354
2015-07-01 14:43:29 +02:00
Likitha Shetty
299c07c3e1 CLOUDSTACK-8597. Failed to migrate volume from zone-wide to cluster-wide storage. While live migrating a volume, CS chooses the endpoint to perform the migration by selecting any host that has the storage containing the volume mounted on it. Instead, if the volume is attached to a running VM, the endpoint chosen by CS should be the host that contains the VM.
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #541
2015-07-01 12:33:26 +02:00
Rohit Yadav
02c3d14f6e saml2: Use UTF-8 based charset for encoding bytes to strings
This fixes the fingbug issue DM_DEFAULT_ENCODING found in SAMLUtils

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-06-29 17:46:35 +02:00
Rohit Yadav
3ff92e883e database: fix upgrade paths from 4.5.2 to 4.6.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-06-29 12:58:47 +02:00
Rohit Yadav
107595a6a5 CLOUDSTACK-8457: SAML auth plugin improvements for production usage
* Move config options to SAML plugin
  This moves all configuration options from Config.java to SAML auth manager. This
  allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
  Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server

- CLOUDSTACK-8458:
    * On UI show dropdown list of discovered IdPs
    * Support SAML Federation, where there may be more than one IdP
        - New datastructure to hold metadata of SP or IdP
        - Recursive processing of IdP metadata
        - Fix login/logout APIs to get new interface and metadata data structure
        - Add org/contact information to metadata
        - Add new API: listIdps that returns list of all discovered IdPs
        - Refactor and cleanup code and tests

- CLOUDSTACK-8459:
    * Add HTTP-POST binding to SP metadata
    * Authn requests must use either HTTP POST/Artifact binding

- CLOUDSTACK-8461:
    * Use unspecified x509 cert as a fallback encryption/signing key
      In case a IDP's metadata does not clearly say if their certificates need to be
      used as signing or encryption and we don't find that, fallback to use the
      unspecified key itself.

- CLOUDSTACK-8462:
    * SAML Auth plugin should not do authorization
      This removes logic to create user if they don't exist. This strictly now
      assumes that users have been already created/imported/authorized by admins.
      As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
      either HTTP POST or HTTP Artifact binding to transfer the message through a
      user agent (browser in our case). The use of HTTP Redirect was one of the reasons
      why this plugin failed to work for some IdP servers that enforce this.
    * Add new User Source
      By reusing the source field, we can find if a user has been SAML enabled or not.
      The limitation is that, once say a user is imported by LDAP and then SAML
      enabled - they won't be able to use LDAP for authentication
    * UI should allow users to pass in domain they want to log into, though it is
      optional and needed only when a user has accounts across domains with same
      username and authorized IDP server
    * SAML users need to be authorized before they can authenticate
        - New column entity to track saml entity id for a user
        - Reusing source column to check if user is saml enabled or not
        - Add new source types, saml2 and saml2disabled
        - New table saml_token to solve the issue of multiple users across domains and
          to enforce security by tracking authn token and checking the samlresponse for
          the tokens
        - Implement API: authorizeSamlSso to enable/disable saml authentication for a
          user
        - Stubs to implement saml token flushing/expiry

- CLOUDSTACK-8463:
    * Use username attribute specified in global setting
      Use username attribute defined by admin from a global setting
      In case of encrypted assertion/attributes:
      - Decrypt them
      - Check signature if provided to check authenticity of message using IdP's
        public key and SP's private key
      - Loop through attributes to find the username

- CLOUDSTACK-8538:
    * Add new global config for SAML request sig algorithm

- CLOUDSTACK-8539:
    * Add metadata refresh timer task and token expiring
        - Fix domain path and save it to saml_tokens
        - Expire hour old saml tokens
        - Refresh metadata based on timer task
        - Fix unit tests

This closes #489

(cherry picked from commit 20ce346f3acb794b08a51841bab2188d426bf7dc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	client/WEB-INF/classes/resources/messages_hu.properties
	plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
	plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
	ui/scripts/ui-custom/login.js
2015-06-29 12:31:51 +02:00
wilderrodrigues
527d6ee77b Making possible to update KVM hosts password in the database via the API
- Make sure the doUpdateHostPassword() method returns the result from the answer, not always true
   - Added the LibvirtUpdateHostPasswordCommandWrapper class, which will handle the changes in the KVM agent
     That's will be further implemented in the future. The command will avoid the agent to complain about Unsupported command
   - Added a test to make sure the current implementation of the comment works
     - If changes in the future, it will also require changes. In that way, we make sure nobody will break it

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #527
2015-06-29 09:59:12 +02:00
Daan Hoogland
8a8849354c findbugs: masking fields removed 2015-06-24 12:24:12 +02:00
Daan Hoogland
8a99df77b6 findbugs volumeInfo is known to not be null
Signed-off-by: Daan Hoogland <daan@onecht.net>

This closes #515
2015-06-23 23:27:04 +02:00
Daan Hoogland
e8984d16ed findbugs: move assert to a place where it still makes sense
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-06-23 23:27:03 +02:00
Daan Hoogland
c0a1009740 findbugs: (un)boxing of long from string 2015-06-22 13:18:14 +02:00