32110 Commits

Author SHA1 Message Date
Nicolas Vazquez
a1a9fb8977
KVM: Enhancements for direct download feature (#3374)
* Add revoke certificates API

* Add background task to sync certificates

* Fix marvin test and revoke certificate

* Fix certificate sent to hypervisor was missing headers

* Fix background task for uploading certificates to hosts
2019-07-18 02:39:00 -03:00
Nicolas Vazquez
a1faf09a2c ui: Fix SystemVMs public range dedication (#3495)
Fix small UI issue when dedicating public IR range for system VMs:
Unable to execute API command createvlaniprange due to invalid value. Invalid parameter domainid value=undefined due to incorrect long value format, or entity does not exist or due to incorrect parameter annotation for the field in api cmd class.

Fixes: #3485
2019-07-17 20:13:40 +05:30
Paul Angus
e15c3112e4
Merge pull request #3248 from shapeblue/storage-offering-domains-zones
Enable service offerings to be scoped to domain(s) and zone(s)
2019-07-16 10:25:35 +01:00
Abhishek Kumar
98e84e372f server: fix public IP association/disassociation to new network (#3489)
Fixes #3321

This changes removes exception throwing while associating an IP address to a new isolated network which is in Allocated state. And it allows disassociating an IP address when it is used for source NAT purpose but network is in allocated state.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-14 17:23:11 +05:30
Anurag Awasthi
9e6ee7c63e server: Add support for new heuristics based VM Deployement for admins (#3454)
Currently an admin can choose which host a VM is to be started on.
They should be able to 'override' the allocation algorthm to a greater
or lesser extent at will, and be able to choose the pod, cluster or host
that they wish a new VM to be deployed in.

DeployVirtualMachine API has been extended with additional, optional
parameters podid and clusterid that will be passed to and used in the
deployment planner, when selecting a viable host. If the user supplies
a pod, a suitable host in the given pod will be selected. If the user
supplies a cluster, a suitable host in the given cluster will be selected.

Based on the parameter supplied and on passing validation, the VM will
then be deployed on the selected host, cluster or pod.
2019-07-13 12:52:48 +05:30
Rohit Yadav
97df52956c
master: travis and trillian smoketests fixes and stabilisation (#3476)
Fix failing test, add more component tests, optimise and refactor test jobs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-12 17:27:49 +05:30
Abhishek Kumar
6d516891ec ui: fix for multiple network offering listing
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-12 13:51:39 +05:30
Abhishek Kumar
cb8f58b706 server: fix for user account able to list child domain n/w offering
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-12 13:39:25 +05:30
Abhishek Kumar
b40fdbb2f7 api: fix account deletion event description (#3483)
Fix account deletion event description.
Fixes #3109

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-12 13:10:30 +05:30
Sid Kattoju
1bf4dd39ca kvm: use IDE as the bus type for root disks and VIRTIO for data disks on platforms without support for para virtualization when using managed storage (#3319)
This change addresses #3089. There was an issue when disks were being added with bus type IDE when creating windows VMs from ISOs. It is not possible to select bus type when creating a VM with an ISO. The bus type is inferred based on the platform emulator string provided to the KVM agent. Currently when creating a VM with managed storage (ex: Solidfire) and OS type string Windows*, all disks are added as IDE. Qemu currently does not support multiple IDE controllers and this configuration results in VMs that cannot be started. This issue does not occur when using NFS as the storage provider due to logic in that KVM agent that makes all data volumes (non root) use a virtio controller for file based disk. Similar logic was added for raw physical disks so that managed storage has the same behavior as NFS. In addition specific versions were removed from the code that guesses the disk controller to be used based on the platform emulator string since most modern operating systems support virtio.

Fixes #3089
2019-07-12 12:35:23 +05:30
Abhishek Kumar
7010f855b5 Merge branch 'master' into storage-offering-domains-zones 2019-07-12 10:21:51 +05:30
Abhishek Kumar
5df11b5c43 ui: fix custom offerings selection in upload volume form (#3479)
Removed duplicate code for diskOffering field.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-11 15:14:53 +05:30
Wei Zhou
b7988a3e5f vr: Fix vpc router in UNKNOWN state (#3465)
If there are more than 10 vpc tiers or public ip subnets in a VPC, eth1X will be added in vpc router.
The redundant state is UNKNOWN in this case.
2019-07-11 01:48:37 +05:30
Spaceman1984
b509e086f9 api: Allowing template owner to download template (#3475)
Removed the download icon when a template is not extractable.

Modified the api to allow a user from the same account as the template, to change the extractable attribute on the template.

Fixes #3400
2019-07-09 12:42:41 +05:30
Anurag Awasthi
c88540d5a5 framework/db: Fix bug in counting items for search query (#3457)
The implementation of GenericBaseDao#searchAndCount() can result in
incorrect count. This happens because the implementation of the
GenericBaseDao#getCount method excludes the groupBy components of the
search queries. This means the count returned will always be larger than
the actual count when not considering pagination.

The change was brought in b0ce8fd which also fails to explain the rationale.

Further investigation of the getCount usage reveal they are always
accompanied by search queries which include groupBy components via
GenericBaseDao#searchIncludingRemoved method.

```
Current code diff between search and count methods is as follows -
diff --git a/framework/db/src/main/java/com/cloud/uddtils/db/GenericDaoBase.java b/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
@@ -371,7 +371,7 @@ public abstract class GenericDaoBase<T, ID extends Serializable> extends Compone
             clause = null;
         }

-        final StringBuilder str = createPartialSelectSql(sc, clause != null, enableQueryCache);
+        final StringBuilder str = createCountSelect(sc, clause != null);
@@ -384,19 +384,12 @@ public abstract class GenericDaoBase<T, ID extends Serializable> extends Compone
             }
         }

-        List<Object> groupByValues = addGroupBy(str, sc);
-        addFilter(str, filter);
-
+        // we have to disable group by in getting count, since count for groupBy clause will be different.
+        //List<Object> groupByValues = addGroupBy(str, sc);
         final TransactionLegacy txn = TransactionLegacy.currentTxn();
-        if (lock != null) {
-            assert (txn.dbTxnStarted() == true) : "As nice as I can here now....how do you lock when there's no DB transaction?  Review your db 101 course from college.";
-            str.append(lock ? FOR_UPDATE_CLAUSE : SHARE_MODE_CLAUSE);
-        }
-
@@ -410,20 +403,19 @@ public abstract class GenericDaoBase<T, ID extends Serializable> extends Compone

+            /*
             if (groupByValues != null) {
                 for (Object value : groupByValues) {
                     pstmt.setObject(i++, value);
                 }
             }
+             */

-            if (s_logger.isDebugEnabled() && lock != null) {
-                txn.registerLock(pstmt.toString());
-            }
             final ResultSet rs = pstmt.executeQuery();
             while (rs.next()) {
-                result.add(toEntityBean(rs, cache));
+                return rs.getInt(1);
             }
-            return result;
+            return 0;
--
2.17.1
```

The fix is to update the way we setup query for counting with group by
params.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-08 17:05:47 +05:30
Richard Lawley
d70f574a7e plugins: fix removing SRX port forwarding rules, improve add/remove logic (#3393)
This PR partially fixes the logic around port forwarding rules on the Juniper SRX plugin. The code in the plugin is based on JunOS 10, which is very old. The changes here should not break compatibility, but should enable the plugin to be used on newer devices. Note that an additional change to a script file is required to be able to add port forwarding rules, but as this PR was targetted for 4.11.3, I thought it best not to include this change as it might break compatibility for anyone still using JunOS 10.

I've made the logic better and consistent for adding/removing static nat and port forwarding rules - these were multi-step processes which did not check each individual step. This would aid in manually fixing rules in case of further problems.

I've also improved the logging for communication with the SRX by stripping out the Apache header before sending it, and indicating the name of the template filename in use.

To be able to add port forwarding rules, the <dst-port> tags in dest-nat-rule-add.xml must be changed to <low>.

Fixes: #3379
2019-07-08 15:46:12 +05:30
Rohit Yadav
e4ddee6fb9
network: allow icmp code 16 in firewall rules (#3468)
This allows for icmp code range 0-16.

Type 9 Router advertisement reference:
https://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml#icmp-parameters-codes-9/#table-icmp-parameters-ext-classes

Fixes #3349

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 23:10:02 +05:30
Anurag Awasthi
f1614aa7c2 api: allow copy tags from template/iso image to VM via deployVM API (#3297)
Support copy tags from template/iso image to VM from deploy vm command. Allow creation of tags from the source template/iso image to vm when deploy vm command creates virtual machine.

Fixes: #3048

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 17:08:25 +05:30
Rohit Yadav
c93630f125
travis: use explicit change directory and use -pl to build rat check (#3472)
This tries to fix build failures seen in job 1 of Travis. Also fixes a pylint issue.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 15:47:44 +05:30
Rohit Yadav
7ed3dc33b3
schema: add support for XenServer 7.1.2 (LTS) (#3467)
Fixes #3461

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 13:41:17 +05:30
HuaDream
7d90652dfa quota: fix issue of QuotaType name (#3463)
Fix issue of QuotaType name.
This bug causes Quota - Summary > Statement to show the incorrect name of VM_DISK_BYTES_WRITE.
2019-07-05 13:40:17 +05:30
Wei Zhou
6780930ecc server: Add option 'details' to listProjects and listAccounts (#3331)
If there are many projects and accounts, listing projects/accounts will take long time getting the resource limitation and resource count in the process. However resource count/limitation are not needed sometimes.

Add an option 'details' to listProjects and listAccounts. If you do not need the resource count/limitation, please add details=min to api call. The api execution time will be reduced significantly.
2019-07-04 16:26:04 +05:30
Anurag Awasthi
a0097d83da engine/schema: count Starting along with Running VMs for user dispersing planner (#3462)
Consider running and starting VMs when considering load ona host for VM deployement for more accurate dispersion.

Fixes: #3442
2019-07-04 16:21:16 +05:30
Rohit Yadav
92773ea9fd Merge remote-tracking branch 'origin/4.12' 2019-07-04 15:32:53 +05:30
Rohit Yadav
47f5c656ac Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 15:32:01 +05:30
Rohit Yadav
35b914570d
travis: use openjdk8 and xenial (ubuntu 16.04) (#3466)
This fixes Travis to use openjdk8 and Xenial (Ubuntu 16.04) as base
distro.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 15:30:18 +05:30
EK
9f18b5b620 ui: allow for the VM Hostname to be edited when VM is switched off (#3412)
Allow for VM name (hostname) to be edited when VM is switched off.
2019-07-04 14:49:32 +05:30
smlshn
f3aa147806 ui: sort list of templates, serviceOfferings, diskOfferings etc in the deploy VM wizard (#3336)
Adds functionality to sort the data that is available on each in the deploy VM wizard's step by their suitable fields:

affinityGroups by name
sshkeyPairs by name
vpcs by name

Fixes: #3050
2019-07-04 14:29:17 +05:30
Rohit Yadav
75e4882f32 Merge remote-tracking branch 'origin/4.12' 2019-07-04 14:01:26 +05:30
Rohit Yadav
f0d7942162 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 13:56:24 +05:30
Paul Angus
b72aadb65d Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-07-03 09:28:09 +01:00
Paul Angus
f855e6d3b9 Merge branch '4.11.3.0-RC20190625T2045' into 4.11 2019-07-03 09:10:24 +01:00
Wei Zhou
452b48ea0c server: reduce execution time while listing project if projects have many resource tags (#3306)
If projects have many resource tags, it will take a long time to list projects.
Remove resource tags information from project_view will fix it the issue.

Fixes #3178
2019-07-03 11:41:53 +05:30
Nicolas Vazquez
4c28a2bcaa ui: Fix wrap text for accounts on project view (#3451)
Fix wrap text for account names on the project view, as it cannot be fully displayed.
2019-07-02 17:28:28 +05:30
Wei Zhou
047003315b utils: reverse ip addresses of a nic returned by java to get the first ip address (#3449)
Java methods getInterfaceAddresses() returns ip addresses in reverse order as "ip addr show"
If there are multiple IPs assigned to a management interface, the last ip will be used as management ip in cloudstack. We need to reverse the ip addresses to get the first ip that makes more sense.

Fixes #3311
2019-07-02 13:16:18 +05:30
Anurag Awasthi
4185452366 ui: Fix sorting bug in UI Code (#3445)
Current master has sorting broken and the order reverses as opposed to
what is set if sortkey.algorithm is set to true. The problem lies in the
way the update APIs were being called by the UI. The code was agnostic
to a global config that backend uses to set the order of the entities
in the corresponding list APIs.

We need to make UI aware of the global config and instead of changing
sign of sort key pass proper numbers so that DB isn't confusing when
some tables have positive sortkey and some have negative.

The fix is in 2 steps-
1) Make use of sortkey name in place where it's relevant. Mere row index
is not sufficient.
2) Reverse the sortkey if we are sorting by descending (when global
config is false)
2019-07-02 12:53:10 +05:30
Anurag Awasthi
14bf2e98f3 ui: Don't pass account name when in project mode (#3435)
When users try to list instances in project mode from Account > SSHKeyPairs > Key > View Instances, we don't need to include sshkeypairs account name into the API Param. It's redundant and causes error.

Fixes #2895
2019-07-02 12:51:44 +05:30
Rohit Yadav
2ecd5ec804
systemvm: don't fork to curl to save password (#3437)
This fixes to avoid forking curl to save password but instead call
a HTTP POST url directly within Python code. This may reduce bottleneck
during high VM launches that require passwords.

Fixes #3182

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-01 23:53:44 +05:30
Abhishek Kumar
9df8d7bc21 ui: create L2, Isolated network - filter offerings for domain
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 18:24:39 +05:30
Abhishek Kumar
d75a26d38b ui: fix for delete VPC offering bug
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 17:04:54 +05:30
Abhishek Kumar
18439ca84b server, api, ui: filtering network offerings for a domain while create network
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 15:17:31 +05:30
Rohit Yadav
4449556aba Merge remote-tracking branch 'origin/4.12' 2019-07-01 14:38:30 +05:30
Rohit Yadav
0649821a21 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-01 14:37:40 +05:30
Abhishek Kumar
5766ddb4d1 ui: fix for create network offering domain, zone selection bug
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 13:51:42 +05:30
Wei Zhou
9ac32aae38 server: VPC redundant vrs run on same hypervisor (#3421)
For VPC supports redundant VRs, when start the second VR, the pod/cluster/host of first VR should be added to avoid list. This provides higher availability.

The network VRs have the same process already.
2019-06-29 12:09:36 +05:30
Rohit Yadav
58c18fc09f
debian: install file as cloudstack-management dependency (#3436)
This install `file` package that provides the `file` tool used by
cloud-install-sys-tmplt script, as part of cloudstack-management
package installation on Ubuntu distros.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-29 00:24:10 +05:30
Rohit Yadav
c160e7532a
ui: fix LB protocol bug (#3438)
Fixes #3252

This fixes labels for missing LB protocol(s).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-29 00:23:36 +05:30
Abhishek Kumar
fc8381549d Merge branch 'master' into storage-offering-domains-zones 2019-06-28 17:33:41 +05:30
Anurag Awasthi
541d280e3b ui: Bug fix for distinguishing between string and map type tags in forms (#3441)
Some APIs consume 'tags' param as string and some consume as maps.
Since each API can have at most one 'tags' param the extraction of
map based tags should only happen when strings based tags are not
extracted from the form serialization.
2019-06-28 17:05:24 +05:30
Abhishek Kumar
54cdab2921 ui: fix for create vpc offering domain selection
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-28 12:45:35 +05:30