10535 Commits

Author SHA1 Message Date
Nicolas Vazquez
a1a9fb8977
KVM: Enhancements for direct download feature (#3374)
* Add revoke certificates API

* Add background task to sync certificates

* Fix marvin test and revoke certificate

* Fix certificate sent to hypervisor was missing headers

* Fix background task for uploading certificates to hosts
2019-07-18 02:39:00 -03:00
Paul Angus
e15c3112e4
Merge pull request #3248 from shapeblue/storage-offering-domains-zones
Enable service offerings to be scoped to domain(s) and zone(s)
2019-07-16 10:25:35 +01:00
Abhishek Kumar
98e84e372f server: fix public IP association/disassociation to new network (#3489)
Fixes #3321

This changes removes exception throwing while associating an IP address to a new isolated network which is in Allocated state. And it allows disassociating an IP address when it is used for source NAT purpose but network is in allocated state.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-14 17:23:11 +05:30
Anurag Awasthi
9e6ee7c63e server: Add support for new heuristics based VM Deployement for admins (#3454)
Currently an admin can choose which host a VM is to be started on.
They should be able to 'override' the allocation algorthm to a greater
or lesser extent at will, and be able to choose the pod, cluster or host
that they wish a new VM to be deployed in.

DeployVirtualMachine API has been extended with additional, optional
parameters podid and clusterid that will be passed to and used in the
deployment planner, when selecting a viable host. If the user supplies
a pod, a suitable host in the given pod will be selected. If the user
supplies a cluster, a suitable host in the given cluster will be selected.

Based on the parameter supplied and on passing validation, the VM will
then be deployed on the selected host, cluster or pod.
2019-07-13 12:52:48 +05:30
Rohit Yadav
97df52956c
master: travis and trillian smoketests fixes and stabilisation (#3476)
Fix failing test, add more component tests, optimise and refactor test jobs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-12 17:27:49 +05:30
Abhishek Kumar
cb8f58b706 server: fix for user account able to list child domain n/w offering
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-12 13:39:25 +05:30
Abhishek Kumar
7010f855b5 Merge branch 'master' into storage-offering-domains-zones 2019-07-12 10:21:51 +05:30
Spaceman1984
b509e086f9 api: Allowing template owner to download template (#3475)
Removed the download icon when a template is not extractable.

Modified the api to allow a user from the same account as the template, to change the extractable attribute on the template.

Fixes #3400
2019-07-09 12:42:41 +05:30
Rohit Yadav
e4ddee6fb9
network: allow icmp code 16 in firewall rules (#3468)
This allows for icmp code range 0-16.

Type 9 Router advertisement reference:
https://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml#icmp-parameters-codes-9/#table-icmp-parameters-ext-classes

Fixes #3349

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 23:10:02 +05:30
Anurag Awasthi
f1614aa7c2 api: allow copy tags from template/iso image to VM via deployVM API (#3297)
Support copy tags from template/iso image to VM from deploy vm command. Allow creation of tags from the source template/iso image to vm when deploy vm command creates virtual machine.

Fixes: #3048

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 17:08:25 +05:30
Wei Zhou
6780930ecc server: Add option 'details' to listProjects and listAccounts (#3331)
If there are many projects and accounts, listing projects/accounts will take long time getting the resource limitation and resource count in the process. However resource count/limitation are not needed sometimes.

Add an option 'details' to listProjects and listAccounts. If you do not need the resource count/limitation, please add details=min to api call. The api execution time will be reduced significantly.
2019-07-04 16:26:04 +05:30
Wei Zhou
452b48ea0c server: reduce execution time while listing project if projects have many resource tags (#3306)
If projects have many resource tags, it will take a long time to list projects.
Remove resource tags information from project_view will fix it the issue.

Fixes #3178
2019-07-03 11:41:53 +05:30
Abhishek Kumar
18439ca84b server, api, ui: filtering network offerings for a domain while create network
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 15:17:31 +05:30
Rohit Yadav
4449556aba Merge remote-tracking branch 'origin/4.12' 2019-07-01 14:38:30 +05:30
Rohit Yadav
0649821a21 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-01 14:37:40 +05:30
Wei Zhou
9ac32aae38 server: VPC redundant vrs run on same hypervisor (#3421)
For VPC supports redundant VRs, when start the second VR, the pod/cluster/host of first VR should be added to avoid list. This provides higher availability.

The network VRs have the same process already.
2019-06-29 12:09:36 +05:30
Abhishek Kumar
fc8381549d Merge branch 'master' into storage-offering-domains-zones 2019-06-28 17:33:41 +05:30
Abhishek Kumar
8c4efaf6a2 server, ui: domain-admin update compute and disk offering access
Behaviour followed while updating disk, compute offerings by domain-admins,
- Domain-admins cannot change zones for offerings specified for domains/subdomains.
- Domain-admins can chnage domains(within their subdomains) for the offerings specified for their domains/subdomains.
- Domain-admins cannot change name, display text, sort-key for offerings specified for their domains/subdomains and also other domains which are not child domain for admin.

Fixes UI unidentified button bug for Update offering access form

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-27 13:32:19 +05:30
Abhishek Kumar
58474530f6 api: snapshot, snapshotpolicy tag support (#3228)
Problem: Currently tags cannot be applied to snapshot when it is being created but through separate “create tags” API calls. For snapshot policies tags cannot be set either at creation or through “create tags” API.

Root Cause: The “create snapshots” API does not support adding tags during creation and it can only be done through “create tags” API. Snapshot policy as a resource does not support tags and no tags can be set for them through any API.

Solution: Tag support for snapshot policy has been added. Snapshot policy with tags when executed will produce snapshots containing the same tags from snapshot policy.

Following APIs have been updated:

Both “create snapshotpolicy” and “create snapshot” now accepts “tags” as a new parameter. The expected format for “tags” parameter is similar to parameter “tags” in “create tags“ API.
Deletion support for tags associated with snapshots policy has been added to “delete snapshotpolicies” API.
Tags set for snapshot policies are added to the Response of “list snapshotpolicies“ API.
UI support for setting tags to snapshots and snapshot policy is provided through the corresponding menus with a new section in each form to set tags.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-27 09:21:08 +05:30
Rohit Yadav
14bff7bd03
server: export granular volume bytes and iops metrics (#3259)
Problem: The VM metrics has aggregated volume bytes read/write and iops metrics but not on per volume basis.
Root Cause: The volume stats sub-system is not used to export the metrics, the support is not available for VMware.
Solution: Use the volume stats sub-system and DB table to export the metrics via the listVolumes and listVolumeMetrics API, and implement support for VMware and fix issue with network and disk metrics in the VM metrics view.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 09:18:10 +05:30
Rohit Yadav
9f4f2c5348
api: instance and template details are free text (#3240)
Problem: Users don't know what keys/values to enter for template and VM details.
Root Cause: The feature does not exist that can list possible details and options.
Solution: Based on the possible VM and template details handled by the
codebase, those details were refactored and a list API is introduced
that can return users those details along with possible values. When
users add details now, they will be presented with a list of key details
and their possible options if any.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 09:14:47 +05:30
EK
f9998e418c server: warn on migration of volumes within the same storage pool (#3424)
Added an if statement that catches the destination poolid as an invalid parameter if it is similer to the current poolid. 

Fixes #3291
2019-06-27 09:06:27 +05:30
Abhishek Kumar
0badbe99f5 server: fix for delete compute offerings by domain admin
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-26 15:30:05 +05:30
Abhishek Kumar
12218e398d Merge branch 'master' into storage-offering-domains-zones
schema and server changes for VPC offering to honour changes from f653e61 in master
2019-06-26 13:27:00 +05:30
Rohit Yadav
0833cf1dd7
server: fix potential NPE while ldap authentication (#3418)
This fixes a potential NPE when a mapped account is not found and
moving of user to the mapped account is performed. This will now
throw a more information exception than NPE.

Fixes #2853

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-26 10:27:21 +05:30
Anurag Awasthi
f653e6149c server: support sort_key for vpc_offerings table (#3268)
Fixes #2742

UI Supported ordering VPC Offerings but the API did not have that
support implemented. This makes the change in updateVPCOfferings
and listVPCOfferings API calls, along with necessary database
changes for supporting sorting of VPC Offerings.
2019-06-26 08:02:19 +05:30
Abhishek Kumar
5628bd1fc7 Merge branch 'master' into storage-offering-domains-zones 2019-06-26 02:24:48 +05:30
Abhishek Kumar
8cff58d3b8 Merge branch 'master' into storage-offering-domains-zones 2019-06-26 02:03:25 +05:30
Abhishek Kumar
62fc132730 server: fix for delete service offering long value parse bug
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-26 01:54:03 +05:30
Abhishek Kumar
9aa61e5731 server: fix for update VPC offering zomain, zone mixup
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-26 01:53:03 +05:30
Abhishek Kumar
1b79466dd9 server: allow disk offering selection for volume from snapshot (#3246)
Problem: Volume created from a snapshot does not show its disk offering.

Root Cause: The volume created from a snapshot of a root disk does not have a disk offering therefore the disk offering of the created volume from the snapshot is empty.

Solution: Refactored createVolume API and extended UI to allow user to select a disk offering while creating a volume using a root disk volume snapshot. For creating volumes using data disk volume snapshot, the disk offering given by the snapshot will be assigned. Disk offering selection in the UI form for volume creation from snapshot is depicted in screenshot below.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-25 22:00:24 +05:30
Rohit Yadav
6f1fc18332 Revert "Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT"
This reverts commit 5bfad44ef41bae2ce6c67d8b848242c1667d73f4 because
we'll need another RC on latest 4.11 branch towards 4.11.3.0.
2019-06-25 21:31:05 +05:30
Nicolas Vazquez
a75444a585
KVM: DPDK live migrations (#3365)
* DPDK live migrations

* Remove DPDK created ports if VM migration fails or prepare migration fails

* Rename DPDK classes lowercase
2019-06-25 12:23:09 -03:00
Andrija Panic
2c70c7357d server: Fix volume.stats.interval description (#3415)
Tested manually - value is in miliseconds, not seconds for the setting volume.stats.interval
2019-06-25 15:20:28 +05:30
Abhishek Kumar
baf7e0a6c6 server, api: allowing domain(s), zone(s) overwrite while updating offerings
'domainid' and 'zoneid' param of update*Offering APIs has been made string type.
For associating multiple domains, zones to an offering, a comma-separated list of domains and zones can be passes.
To make a domain specific offering public, a value of 'public' can be given for domainid param.
To make a zone specific offering available for all zones, a value of 'all' can be given for zoneid param.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-23 20:04:01 +05:30
Paul Angus
5bfad44ef4 Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-06-22 10:21:02 +01:00
Abhishek Kumar
c556abf969 server: overwrite associated domain(s), zones(s) with offering on update
On update*Offering API call, supplied domain(s) and zone(s) will overwrite current domains and zones associated with the offering.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-21 15:00:58 +05:30
Rohit Yadav
202777e52a
server: publish volume resize event for volumes (#3219)
Problem: The disk offering change is not reflected in cloud_usage database table.
Root Cause: The resizeVolume API does not publish the volume disk offering change event to the
cloud_usage database table.
Solution: This issue has been fixed by refactoring the resizeVolume API to publish this disk offering change for volumes that either in Allocated or Ready state.

Moves the method that published events for volumes in Ready state from
the VolumeStateListener class to the orchestrateResizeVolume method in
the VolumeApiService.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-21 13:28:08 +05:30
Rohit Yadav
2c32d60e44
server: return usage description with resource names and UUIDs (#3344)
The usage record descriptions have CloudStack's internal integer IDs
which makes it difficult for users to read their usages. This PRs
introduces a new API boolean flag `oldformat` which when set to true
would return the older description format, otherwise by default
listUsageRecords will process and return description with names and
UUIDs of resources.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-21 03:48:33 +05:30
Rohit Yadav
61ebc6a4a6
api: Set network name as part of the network usage response (#3234)
Problem: Network name is not part of the network usage response
Root Cause: Code does not set the network name
Solution: Set the network name for network usage type usage records in the API response

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-20 21:39:15 +05:30
Rohit Yadav
26a5d50760
Merge branch 'master' into storage-offering-domains-zones 2019-06-20 15:09:48 +05:30
Rohit Yadav
90cd8aa73d
server: add support for sorting zones in UI/API (#3242)
Problem: Not able to configure a sort order for the zones that are listed in various views in the UI.

Root Cause: There is no mechanism to accept sort key for existing zones or UI widget, that would allow to listing zones in the UI in a certain order.

Solution: The order of zones in listed in various views in the UI can now be configured through the newly added “sort_key” field added for the zone. It can be set using updateZone API by providing “sort_key” parameter for a zone, or by reordering the items in the zones list in the UI. UI has been updated to show ordering controls in zones list view. Database changes include updating table “data_center” by adding “sort_key” column (containing integer values and defaults to zero).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-19 18:03:56 +05:30
Dingane Hlaluku
0d6cae6339 volume: fix volume metrics view from returning sensitive info to end user (#3222)
Problem: The listVolumeMetrics API response does not honor the volume detail visibility restrictions set for normal users and returns sensitive information which should only be visible to the root admin.

Root Cause: The listVolumeMetrics API response extends the ListVolumesByAdmin API internally and this results in a full display view response that is only meant for the root admin.

Solution: This has been fixed by rectifying the API response to not show ‘physical size’, 'storage type', and ‘storage pool’ information. The UI has also been fixed to hide these columns for normal users.
2019-06-19 17:04:26 +05:30
Rohit Yadav
0e87040eb2
network: allow ability to specify if network's ipaddress usage need to be hidden (#3235)
Problem: Admins don’t want to charge for IP address usage on certain (shared) networks.
Root Cause: There is no flag or detail for admins to provide using UI or API when creating networks to specify if they want IP address usage of the network hidden.
Solution: A new boolean hideipaddressusage flag is added to the createNetwork API and a checkbox in the ‘Add guest network’ UI for the root admins to specify if they want the shared network’s IP address usage to be hidden in the listUsageRecords API response. The provided flag is saved as the ‘hideIpAddressUsage’ detail in the cloud.network_details table for the network. For existing (shared) networks, root admins can also specify the same boolean API parameter hideipaddressusage with the updateNetwork API request to configure the behaviour for an existing network. When the detail/flag is true, the IP address usage for the (shared) network is not exported in the listUsageRecords API response. The listNetworks API response will include the details of a network for root admin only. (note usage is still recorded in the usage database but not return by the listUsageRecords API)

The API flag works for any kind of network via the API, but the checkbox is only shown while creating shared networks in the UI.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-19 10:25:59 +05:30
Abhishek Kumar
cf347c89ea Merge branch 'master' into storage-offering-domains-zones 2019-06-18 12:52:34 +05:30
Anurag Awasthi
1c963bac13 ui: Add more info for creating volume snapshots (#3390)
Add more details in console logs and event details.
Also show popup on successful completition of snapshots.

Fixes: #3327
2019-06-13 15:50:10 +05:30
Rohit Yadav
f30d716452
cloudstack: fix forward merge issues (#3394)
- Fixes tests path from old layout to standard maven in src/test/java/
- Removed duplicate SnapshotManagerImpl at old path `server/src/com...`

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-12 16:38:58 +05:30
Rohit Yadav
aecfe6d500
server: save GUID for KVM cluster (#3398)
When a KVM host is added to a cluster, the cluster GUID is null. In case
the KVM host fails to be added, the GUID is not set to null and if any
other hosts are added an exception is thrown by the resource manager
that does not allow addition of hosts to a cluster with existing hosts
whose GUID is null.

In case of KVM, other hosts may be added in parallel therefore this
restriction can be safely removed.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-12 12:47:24 +05:30
Anurag Awasthi
c4f6c43852 api: Add to listRouters response the VR scripts version (#3372)
Add missing scriptsversion field to list router api call response.

Fixes #3357
2019-06-10 22:50:56 +05:30
Paul Angus
51124b7b35 Updating pom.xml version numbers for release 4.11.3.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-06-10 16:15:05 +01:00