Problem: Currently tags cannot be applied to snapshot when it is being created but through separate “create tags” API calls. For snapshot policies tags cannot be set either at creation or through “create tags” API.
Root Cause: The “create snapshots” API does not support adding tags during creation and it can only be done through “create tags” API. Snapshot policy as a resource does not support tags and no tags can be set for them through any API.
Solution: Tag support for snapshot policy has been added. Snapshot policy with tags when executed will produce snapshots containing the same tags from snapshot policy.
Following APIs have been updated:
Both “create snapshotpolicy” and “create snapshot” now accepts “tags” as a new parameter. The expected format for “tags” parameter is similar to parameter “tags” in “create tags“ API.
Deletion support for tags associated with snapshots policy has been added to “delete snapshotpolicies” API.
Tags set for snapshot policies are added to the Response of “list snapshotpolicies“ API.
UI support for setting tags to snapshots and snapshot policy is provided through the corresponding menus with a new section in each form to set tags.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Problem: Users don't know what keys/values to enter for template and VM details.
Root Cause: The feature does not exist that can list possible details and options.
Solution: Based on the possible VM and template details handled by the
codebase, those details were refactored and a list API is introduced
that can return users those details along with possible values. When
users add details now, they will be presented with a list of key details
and their possible options if any.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Problem: Custom compute offering does not allow setting min and max values for CPU and VRAM for custom VMs.
Root Cause: Custom compute offerings cannot be created with a given range of CPU number and memory instead it allows only fixed values.
Solution: createServiceOffering API has been modified to allow setting a defined range for CPU number and memory. Also, UI form for compute offering creation is provided with a new field named 'compute offering type’ with values - Fixed, Custom Constrained, Custom Constrained. It will allow the creation of compute offerings either with a fixed CPU speed and memory for fixed compute offering, or with a range of CPU number and memory for custom constrained compute offering or without predefined CPU number, CPU speed and memory for custom unconstrained compute offering.
To allow the user to set CPU number, CPU speed and memory during VM deployment, UI form for VM deployment has been modified to provide controls to change these values. These controls are depicted in screenshots below for custom constrained and custom unconstrained compute offering types.
Sample API calls using cmk to create a constrained service offering and deploying a VM using it,
create serviceoffering name=Constrained displaytext=Constrained customized=true mincpunumber=2 maxcpunumber=4 cpuspeed=400 minmemory=256 maxmemory=1024
deploy virtualmachine displayname=ConstrainedVM serviceofferingid=60f3e500-6559-40b2-9a61-2192891c2bd6 templateid=8e0f4a3e-601b-11e9-9df4-a0afbd4a2d60 zoneid=9612a0c6-ed28-4fae-9a48-6eb207af29e3 details[0].cpuNumber=3 details[0].memory=800
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
* Add jQuery ui stylings to the project
* Adapt stylings to jquery ui stylings
- remove unnecessary (empty) style definitions
* Check if jquery autocomplete is initialized before destroying it
* Use dialog modal option instead of setting z-index styling
- set minimum heigth of dialog
* Swap removeAttr() for prop() since it's not longer recommended as of jQuery 3.0
* Fix reccuring snapshots modal view
* Further style improvements
* More improvements on the instance-wizard (and multi-wizard in general)
* Raise input hints over modal z-index
* Adapt/fix stylings
- center buttons of create form modals
- fix width of select menu in service offerings
- fix width of modal in Network -> VPC -> configure -> static NAT -> aquire new Ip -> enable static nat
* Adapt width of dynamic-input container
This removes CA certificate button from UI as this is not used and
confuses both admin and users how to use it. This may be re-introduced
in future when a use-case can be justified.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Update jquery, jquery ui, jquery validate and flot to current versions
update jquery to 3.3.1
update jqueryui to 1.12.1
update jquery validate to 1.17.0
update jquery flot to 0.8.3
* Replace deprecated removed jquery functions
* Fix initial tab content loading in detailView
* Fix logout for new jquery version
* Fix tooltip detail displaying for new JQuery version
* Fix view all trigger in detailView for new JQuery version
* Fix breadcrumb click event handler for JQuery update
* Fix displaying of preselected zone in instanceWizard for new jQuery verion
Fix UI overlay issue in snapshot view on master with new jquery and other changes. This problem was created because of the jQuery-UI upgrade.
This fixes: #2830
In the Add Zone Wizard, when a user lands on the Physical Network page, by default the first Physical Network gets initialized with the 3 Traffic Types: Guest, Management and Public. When a user drags the Management or Public Traffic Types from the first Physical Network to a second or nth Physical Network and then clicks the next button to go to the next screen, and then decides to click the previous button to go back to the Physical Network screen, the UI initializes the physicalNetwork again and thus moves these Traffic Types back to their original position in the first Physical Network as if it is the first time the user navigated to this page.
A fix was made so that when a user clicks the previous button to go back to the Physical Network screen, it does not initialize the physicalNetwork again, therefore leaving the user-defined Traffic Type configuration as it was before the next button was clicked.
* [CLOUDSTACK-9261] Upgrate jQuery-UI to 1.11 (JQuery UI 1.8.4 prone to XSS)
* fix problems in the UI for lbCertificatePolicy and StaticNAT
* force jenkins build
* Fix about dialog
* Fix position of network service offering
CloudStack volumes and templates are one single virtual disk in case of XenServer/XCP and KVM hypervisors since the files used for templates and volumes are virtual disks (VHD, QCOW2). However, VMware volumes and templates are in OVA format, which are archives that can contain a complete VM including multiple VMDKs and other files such as ISOs. And currently, Cloudstack only supports Template creation based on OVA files containing a single disk. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, only the first disk is attached to the new instance and other disks are ignored.
Similarly with uploaded volumes, attaching an uploaded volume that contains multiple disks to a VM will result in only one VMDK to being attached to the VM.
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Support+OVA+files+containing+multiple+disks
This behavior needs to be improved in VMWare to support OVA files with multiple disks for both uploaded volumes and templates. i.e. If a user creates a template from a OVA file containing more than 1 disk and launches an instance using this template, the first disk should be attached to the new instance as the ROOT disk and volumes should be created based on other VMDK disks in the OVA file and should be attached to the instance.
Signed-off-by: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit is related to the Saml2 user authentication plugin.
The user can list its domains. His/Her domains are presented as the
following example:
username/subdomainA
username/subdomainB
However, if a user has two subdomains of the same name Dom1/subdomainA
and Dom2/subdomainA, the list is presented as follows:
username/subdomainA
username/subdomainA
With this commit it shows the full domain name to avoid such cases.
Thus, the domains will be presented as follows:
username/Dom1/subdomainA
username/Dom2/subdomainA
This introduces a new certificate authority framework that allows
pluggable CA provider implementations to handle certificate operations
around issuance, revocation and propagation. The framework injects
itself to `NioServer` to handle agent connections securely. The
framework adds assumptions in `NioClient` that a keystore if available
with known name `cloud.jks` will be used for SSL negotiations and
handshake.
This includes a default 'root' CA provider plugin which creates its own
self-signed root certificate authority on first run and uses it for
issuance and provisioning of certificate to CloudStack agents such as
the KVM, CPVM and SSVM agents and also for the management server for
peer clustering.
Additional changes and notes:
- Comma separate list of management server IPs can be set to the 'host'
global setting. Newly provisioned agents (KVM/CPVM/SSVM etc) will get
radomized comma separated list to which they will attempt connection
or reconnection in provided order. This removes need of a TCP LB on
port 8250 (default) of the management server(s).
- All fresh deployment will enforce two-way SSL authentication where
connecting agents will be required to present certificates issued
by the 'root' CA plugin.
- Existing environment on upgrade will continue to use one-way SSL
authentication and connecting agents will not be required to present
certificates.
- A script `keystore-setup` is responsible for initial keystore setup
and CSR generation on the agent/hosts.
- A script `keystore-cert-import` is responsible for import provided
certificate payload to the java keystore file.
- Agent security (keystore, certificates etc) are setup initially using
SSH, and later provisioning is handled via an existing agent connection
using command-answers. The supported clients and agents are limited to
CPVM, SSVM, and KVM agents, and clustered management server (peering).
- Certificate revocation does not revoke an existing agent-mgmt server
connection, however rejects a revoked certificate used during SSL
handshake.
- Older `cloudstackmanagement.keystore` is deprecated and will no longer
be used by mgmt server(s) for SSL negotiations and handshake. New
keystores will be named `cloud.jks`, any additional SSL certificates
should not be imported in it for use with tomcat etc. The `cloud.jks`
keystore is stricly used for agent-server communications.
- Management server keystore are validated and renewed on start up only,
the validity of them are same as the CA certificates.
New APIs:
- listCaProviders: lists all available CA provider plugins
- listCaCertificate: lists the CA certificate(s)
- issueCertificate: issues X509 client certificate with/without a CSR
- provisionCertificate: provisions certificate to a host
- revokeCertificate: revokes a client certificate using its serial
Global settings for the CA framework:
- ca.framework.provider.plugin: The configured CA provider plugin
- ca.framework.cert.keysize: The key size for certificate generation
- ca.framework.cert.signature.algorithm: The certificate signature algorithm
- ca.framework.cert.validity.period: Certificate validity in days
- ca.framework.cert.automatic.renewal: Certificate auto-renewal setting
- ca.framework.background.task.delay: CA background task delay/interval
- ca.framework.cert.expiry.alert.period: Days to check and alert expiring certificates
Global settings for the default 'root' CA provider:
- ca.plugin.root.private.key: (hidden/encrypted) CA private key
- ca.plugin.root.public.key: (hidden/encrypted) CA public key
- ca.plugin.root.ca.certificate: (hidden/encrypted) CA certificate
- ca.plugin.root.issuer.dn: The CA issue distinguished name
- ca.plugin.root.auth.strictness: Are clients required to present certificates
- ca.plugin.root.allow.expired.cert: Are clients with expired certificates allowed
UI changes:
- Button to download/save the CA certificates.
Misc changes:
- Upgrades bountycastle version and uses newer classes
- Refactors SAMLUtil to use new CertUtils
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit contains following changes
(1) add CPU CORE information in op_host_capacity
(2) add capacity name in the CapacityResponse
(3) add allocatedCapacity for CPU/MEMORY/CPU CORE for zones
(4) sort CapacityResponse by zonename and CapacityType
Reviewed-By: Rashmi Dixit
Problem: All the hosts suitable for VM Migration are not shown in the UI. This could
confuse the user as the target host might never be shown in the UI.
Root Cause: The API (findHostsForMigration) always returned page 1 results which would
be always <= default.page.size global parameter. Therefore, in case of large
no. of hosts where the result can map to multiple pages, this issue would arise.
Solution: 1. Replace drop-down with listView widget.
2. Allow lazy-loading of records on listView's scroll.
3. Show additional parameters (CPU/Memory used) to assist admin in decision making.
4. Provide 'Search by host name' to limit the results.
Added change where if there are no hosts found, an empty row with message will
appear.
This commit includes three changes:
(1) Revert commit 13bf1ec5da9891eca98ef5dab61bf74b6db9530b to show the 'root disk size' field
(2) Set the default value from '1' to empty. This also changes the default value of customized service offerings from '1' to empty.
(3) show 'root disk size' field only when users choose a KVM template, as cloudstack supports deployvm with rootdisksize on KVM only.
CLOUDSTACK-9229: Autoscale policy creation failing in VPC due to zone id missing in createAutoScaleVmProfileAutoscale policy creation is failing on VPC while creating autoscalevmprofile since zoneid parameter is missing.
Steps to Reproduce:
================
1. Create VPC Network offering with NS as service provider for LB
2. Create VPC using offering which has NetScaler as LB provider
3. Add a tier using the network offering created in Step1
4. Launch a VM
5. Acquire a public IP
6. Create an auto scale policy with all required parameters on an LB rule
Result:
======
Fails during autoscalevmprofile creation with the following error:
GET command=createAutoScaleVmProfile&response=json&serviceofferingid=05ac866f-604c-43e2-a48b-47e83ef7c4f1&templateid=3cdd83c4-ad27-11e5-9eaf-42407779c24b&destroyvmgraceperiod=30&counterparam%5B0%5D.name=snmpcommunity&counterparam%5B0%5D.value=public&counterparam%5B1%5D.name=snmpport&counterparam%5B1%5D.value=161&_=1451458073663
DEBUG [o.a.c.a.BaseCmd] (catalina-exec-10:ctx-58bcf6cf ctx-f5123e30) (logid:ae6742c8) Ignoring paremeter fordisplay as the caller is not authorized to pass it in
INFO [c.c.a.ApiServer] (catalina-exec-10:ctx-58bcf6cf ctx-f5123e30) (logid:ae6742c8) Unable to execute API command autoscalevmprofile due to missing parameter zoneid
Expected Result:
=============
Autoscale policy creation should be successful and the "min" number of VMs specified in the policy should be deployed.
Following should be the params for autoscalevmprofile creation:
command=createAutoScaleVmProfile&response=json&zoneid=0b43fff3-c069-417b-ac0c-e4dc46b407ea&serviceofferingid=0
5ac866f-604c-43e2-a48b-47e83ef7c4f1&templateid=3cdd83c4-ad27-11e5-9eaf-42407779c24b&destroyvmgraceperiod=30&counterparam%5B0%5D
.name=snmpcommunity&counterparam%5B0%5D.value=public&counterparam%5B1%5D.name=snmpport&counterparam%5B1%5D.value=161
Fix:
===
Added a conditon to check whether the zoneid in networks is undefined,
If that is undefined, it gets the zoneid from ipaddresses argument.
Fixed a localization issue.
* pr/1334:
CLOUDSTACK-9229: Autoscale policy creation failing in VPC due to zoneid missing in createAutoScaleVmProfile
Signed-off-by: Remi Bergsma <github@remi.nl>
- Implement Search boxes in all Metrics views
- Fix threshold calculations for host and storage pool metrics view
- Consider overcommit ratios for calculation allocated thresholds
- Save/pass context while navigating across resources and metrics view
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Added a conditon to check whether the zoneid in networks is undefined,
If that is undefined, it gets the zoneid from ipaddresses argument.
Fixed a localization issue.
Implements various metrics views based on a listView based widget that has following
properties:
- vertically and horizontally scrollable with pagination/infinite scrolling
- sortable columns (client side)
- groupable/collapsible columns
- alternate row coloring
- refresh button to refresh views
- threshold table cell coloring
- panel/breadcrumb navigation
- quick view action column
- translatable labels
- sorts after metrics is refreshed, if a column was previously sorted
- sorts after adding rows on infinite scrolling if a column was pre-sorted
- Metrics views: Zones, Clusters, Hosts, Instances, Storage pools, Volumes
- Resource filtering/navigation: Zones->Clusters->Hosts->Instances->Volumes,
Storage Pool->Volumes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- l10n for the SSH Key Pairs behavior
- l10n for Autoscaling / LB sections
- l10n for Reset password
- l10n on some strings for the installation Wizard
- l10n on some strings in VPN/VPC section
- l10n on Service offerings sections
- improve some FR translations
[CLOUDSTACK-8427] Add missing localization for some labels in Web UI- on new functionnality: upload volume/template from Local
- on the zone configuration wizard for the "Next" button
- update French messages properties from transifex
- Improve some French translations ("Tlverser" for "Upload")
* pr/686:
Add missing localization for some labels in Web UI - on new functionnality: upload volume/template from Local - on the zone configuration wizard for the "Next" button - update French messages properties from transifex - Improve some French translations ("Téléverser" for "Upload")
Signed-off-by: Remi Bergsma <github@remi.nl>
SAML authorized accounts might be across various domains, this allows for
switching of accounts only in case of SAML authenticated user accounts across
other accounts with the same SAML uid/username.
Moves the previous switch account logic to its own ui-custom module
(cherry picked from commit 1065661cd50c8d43bf65644a13d164b96732b011)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/index.jsp
- on new functionnality: upload volume/template from Local
- on the zone configuration wizard for the "Next" button
- update French messages properties from transifex
- Improve some French translations ("Téléverser" for "Upload")
This uses local selector to access the DOM elements, the previous commit
would find a global element which is not in the context and fail changing the
element in the opened dialog.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#583
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
and the API parameters. In both cases, if either or both are passed they should
match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user
Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
admin, domain-admin and user
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This closes#574
This closes#308
* Move config options to SAML plugin
This moves all configuration options from Config.java to SAML auth manager. This
allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server
- CLOUDSTACK-8458:
* On UI show dropdown list of discovered IdPs
* Support SAML Federation, where there may be more than one IdP
- New datastructure to hold metadata of SP or IdP
- Recursive processing of IdP metadata
- Fix login/logout APIs to get new interface and metadata data structure
- Add org/contact information to metadata
- Add new API: listIdps that returns list of all discovered IdPs
- Refactor and cleanup code and tests
- CLOUDSTACK-8459:
* Add HTTP-POST binding to SP metadata
* Authn requests must use either HTTP POST/Artifact binding
- CLOUDSTACK-8461:
* Use unspecified x509 cert as a fallback encryption/signing key
In case a IDP's metadata does not clearly say if their certificates need to be
used as signing or encryption and we don't find that, fallback to use the
unspecified key itself.
- CLOUDSTACK-8462:
* SAML Auth plugin should not do authorization
This removes logic to create user if they don't exist. This strictly now
assumes that users have been already created/imported/authorized by admins.
As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
either HTTP POST or HTTP Artifact binding to transfer the message through a
user agent (browser in our case). The use of HTTP Redirect was one of the reasons
why this plugin failed to work for some IdP servers that enforce this.
* Add new User Source
By reusing the source field, we can find if a user has been SAML enabled or not.
The limitation is that, once say a user is imported by LDAP and then SAML
enabled - they won't be able to use LDAP for authentication
* UI should allow users to pass in domain they want to log into, though it is
optional and needed only when a user has accounts across domains with same
username and authorized IDP server
* SAML users need to be authorized before they can authenticate
- New column entity to track saml entity id for a user
- Reusing source column to check if user is saml enabled or not
- Add new source types, saml2 and saml2disabled
- New table saml_token to solve the issue of multiple users across domains and
to enforce security by tracking authn token and checking the samlresponse for
the tokens
- Implement API: authorizeSamlSso to enable/disable saml authentication for a
user
- Stubs to implement saml token flushing/expiry
- CLOUDSTACK-8463:
* Use username attribute specified in global setting
Use username attribute defined by admin from a global setting
In case of encrypted assertion/attributes:
- Decrypt them
- Check signature if provided to check authenticity of message using IdP's
public key and SP's private key
- Loop through attributes to find the username
- CLOUDSTACK-8538:
* Add new global config for SAML request sig algorithm
- CLOUDSTACK-8539:
* Add metadata refresh timer task and token expiring
- Fix domain path and save it to saml_tokens
- Expire hour old saml tokens
- Refresh metadata based on timer task
- Fix unit tests
This closes#489
(cherry picked from commit 20ce346f3acb794b08a51841bab2188d426bf7dc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Conflicts:
client/WEB-INF/classes/resources/messages_hu.properties
plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
ui/scripts/ui-custom/login.js
1. provide compatibility with the Big Cloud Fabric (BCF) controller
L2 Connectivity Service in both VPC and non-VPC modes
2. virtual network terminology updates: VNS --> BCF_SEGMENT
3. uses HTTPS with trust-always certificate handling
4. topology sync support with BCF controller
5. support multiple (two) BCF controllers with HA
6. support VM migration
7. support Firewall, Static NAT, and Source NAT with NAT enabled option
8. add VifDriver for Indigo Virtual Switch (IVS)
This closes#151
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
