32101 Commits

Author SHA1 Message Date
Abhishek Kumar
cb8f58b706 server: fix for user account able to list child domain n/w offering
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-12 13:39:25 +05:30
Abhishek Kumar
7010f855b5 Merge branch 'master' into storage-offering-domains-zones 2019-07-12 10:21:51 +05:30
Abhishek Kumar
5df11b5c43 ui: fix custom offerings selection in upload volume form (#3479)
Removed duplicate code for diskOffering field.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-07-11 15:14:53 +05:30
Wei Zhou
b7988a3e5f vr: Fix vpc router in UNKNOWN state (#3465)
If there are more than 10 vpc tiers or public ip subnets in a VPC, eth1X will be added in vpc router.
The redundant state is UNKNOWN in this case.
2019-07-11 01:48:37 +05:30
Spaceman1984
b509e086f9 api: Allowing template owner to download template (#3475)
Removed the download icon when a template is not extractable.

Modified the api to allow a user from the same account as the template, to change the extractable attribute on the template.

Fixes #3400
2019-07-09 12:42:41 +05:30
Anurag Awasthi
c88540d5a5 framework/db: Fix bug in counting items for search query (#3457)
The implementation of GenericBaseDao#searchAndCount() can result in
incorrect count. This happens because the implementation of the
GenericBaseDao#getCount method excludes the groupBy components of the
search queries. This means the count returned will always be larger than
the actual count when not considering pagination.

The change was brought in b0ce8fd which also fails to explain the rationale.

Further investigation of the getCount usage reveal they are always
accompanied by search queries which include groupBy components via
GenericBaseDao#searchIncludingRemoved method.

```
Current code diff between search and count methods is as follows -
diff --git a/framework/db/src/main/java/com/cloud/uddtils/db/GenericDaoBase.java b/framework/db/src/main/java/com/cloud/utils/db/GenericDaoBase.java
@@ -371,7 +371,7 @@ public abstract class GenericDaoBase<T, ID extends Serializable> extends Compone
             clause = null;
         }

-        final StringBuilder str = createPartialSelectSql(sc, clause != null, enableQueryCache);
+        final StringBuilder str = createCountSelect(sc, clause != null);
@@ -384,19 +384,12 @@ public abstract class GenericDaoBase<T, ID extends Serializable> extends Compone
             }
         }

-        List<Object> groupByValues = addGroupBy(str, sc);
-        addFilter(str, filter);
-
+        // we have to disable group by in getting count, since count for groupBy clause will be different.
+        //List<Object> groupByValues = addGroupBy(str, sc);
         final TransactionLegacy txn = TransactionLegacy.currentTxn();
-        if (lock != null) {
-            assert (txn.dbTxnStarted() == true) : "As nice as I can here now....how do you lock when there's no DB transaction?  Review your db 101 course from college.";
-            str.append(lock ? FOR_UPDATE_CLAUSE : SHARE_MODE_CLAUSE);
-        }
-
@@ -410,20 +403,19 @@ public abstract class GenericDaoBase<T, ID extends Serializable> extends Compone

+            /*
             if (groupByValues != null) {
                 for (Object value : groupByValues) {
                     pstmt.setObject(i++, value);
                 }
             }
+             */

-            if (s_logger.isDebugEnabled() && lock != null) {
-                txn.registerLock(pstmt.toString());
-            }
             final ResultSet rs = pstmt.executeQuery();
             while (rs.next()) {
-                result.add(toEntityBean(rs, cache));
+                return rs.getInt(1);
             }
-            return result;
+            return 0;
--
2.17.1
```

The fix is to update the way we setup query for counting with group by
params.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-08 17:05:47 +05:30
Richard Lawley
d70f574a7e plugins: fix removing SRX port forwarding rules, improve add/remove logic (#3393)
This PR partially fixes the logic around port forwarding rules on the Juniper SRX plugin. The code in the plugin is based on JunOS 10, which is very old. The changes here should not break compatibility, but should enable the plugin to be used on newer devices. Note that an additional change to a script file is required to be able to add port forwarding rules, but as this PR was targetted for 4.11.3, I thought it best not to include this change as it might break compatibility for anyone still using JunOS 10.

I've made the logic better and consistent for adding/removing static nat and port forwarding rules - these were multi-step processes which did not check each individual step. This would aid in manually fixing rules in case of further problems.

I've also improved the logging for communication with the SRX by stripping out the Apache header before sending it, and indicating the name of the template filename in use.

To be able to add port forwarding rules, the <dst-port> tags in dest-nat-rule-add.xml must be changed to <low>.

Fixes: #3379
2019-07-08 15:46:12 +05:30
Rohit Yadav
e4ddee6fb9
network: allow icmp code 16 in firewall rules (#3468)
This allows for icmp code range 0-16.

Type 9 Router advertisement reference:
https://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml#icmp-parameters-codes-9/#table-icmp-parameters-ext-classes

Fixes #3349

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 23:10:02 +05:30
Anurag Awasthi
f1614aa7c2 api: allow copy tags from template/iso image to VM via deployVM API (#3297)
Support copy tags from template/iso image to VM from deploy vm command. Allow creation of tags from the source template/iso image to vm when deploy vm command creates virtual machine.

Fixes: #3048

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 17:08:25 +05:30
Rohit Yadav
c93630f125
travis: use explicit change directory and use -pl to build rat check (#3472)
This tries to fix build failures seen in job 1 of Travis. Also fixes a pylint issue.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 15:47:44 +05:30
Rohit Yadav
7ed3dc33b3
schema: add support for XenServer 7.1.2 (LTS) (#3467)
Fixes #3461

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-05 13:41:17 +05:30
HuaDream
7d90652dfa quota: fix issue of QuotaType name (#3463)
Fix issue of QuotaType name.
This bug causes Quota - Summary > Statement to show the incorrect name of VM_DISK_BYTES_WRITE.
2019-07-05 13:40:17 +05:30
Wei Zhou
6780930ecc server: Add option 'details' to listProjects and listAccounts (#3331)
If there are many projects and accounts, listing projects/accounts will take long time getting the resource limitation and resource count in the process. However resource count/limitation are not needed sometimes.

Add an option 'details' to listProjects and listAccounts. If you do not need the resource count/limitation, please add details=min to api call. The api execution time will be reduced significantly.
2019-07-04 16:26:04 +05:30
Anurag Awasthi
a0097d83da engine/schema: count Starting along with Running VMs for user dispersing planner (#3462)
Consider running and starting VMs when considering load ona host for VM deployement for more accurate dispersion.

Fixes: #3442
2019-07-04 16:21:16 +05:30
Rohit Yadav
92773ea9fd Merge remote-tracking branch 'origin/4.12' 2019-07-04 15:32:53 +05:30
Rohit Yadav
47f5c656ac Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 15:32:01 +05:30
Rohit Yadav
35b914570d
travis: use openjdk8 and xenial (ubuntu 16.04) (#3466)
This fixes Travis to use openjdk8 and Xenial (Ubuntu 16.04) as base
distro.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 15:30:18 +05:30
EK
9f18b5b620 ui: allow for the VM Hostname to be edited when VM is switched off (#3412)
Allow for VM name (hostname) to be edited when VM is switched off.
2019-07-04 14:49:32 +05:30
smlshn
f3aa147806 ui: sort list of templates, serviceOfferings, diskOfferings etc in the deploy VM wizard (#3336)
Adds functionality to sort the data that is available on each in the deploy VM wizard's step by their suitable fields:

affinityGroups by name
sshkeyPairs by name
vpcs by name

Fixes: #3050
2019-07-04 14:29:17 +05:30
Rohit Yadav
75e4882f32 Merge remote-tracking branch 'origin/4.12' 2019-07-04 14:01:26 +05:30
Rohit Yadav
f0d7942162 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-04 13:56:24 +05:30
Paul Angus
b72aadb65d Updating pom.xml version numbers for release 4.11.4.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-07-03 09:28:09 +01:00
Paul Angus
f855e6d3b9 Merge branch '4.11.3.0-RC20190625T2045' into 4.11 2019-07-03 09:10:24 +01:00
Wei Zhou
452b48ea0c server: reduce execution time while listing project if projects have many resource tags (#3306)
If projects have many resource tags, it will take a long time to list projects.
Remove resource tags information from project_view will fix it the issue.

Fixes #3178
2019-07-03 11:41:53 +05:30
Nicolas Vazquez
4c28a2bcaa ui: Fix wrap text for accounts on project view (#3451)
Fix wrap text for account names on the project view, as it cannot be fully displayed.
2019-07-02 17:28:28 +05:30
Wei Zhou
047003315b utils: reverse ip addresses of a nic returned by java to get the first ip address (#3449)
Java methods getInterfaceAddresses() returns ip addresses in reverse order as "ip addr show"
If there are multiple IPs assigned to a management interface, the last ip will be used as management ip in cloudstack. We need to reverse the ip addresses to get the first ip that makes more sense.

Fixes #3311
2019-07-02 13:16:18 +05:30
Anurag Awasthi
4185452366 ui: Fix sorting bug in UI Code (#3445)
Current master has sorting broken and the order reverses as opposed to
what is set if sortkey.algorithm is set to true. The problem lies in the
way the update APIs were being called by the UI. The code was agnostic
to a global config that backend uses to set the order of the entities
in the corresponding list APIs.

We need to make UI aware of the global config and instead of changing
sign of sort key pass proper numbers so that DB isn't confusing when
some tables have positive sortkey and some have negative.

The fix is in 2 steps-
1) Make use of sortkey name in place where it's relevant. Mere row index
is not sufficient.
2) Reverse the sortkey if we are sorting by descending (when global
config is false)
2019-07-02 12:53:10 +05:30
Anurag Awasthi
14bf2e98f3 ui: Don't pass account name when in project mode (#3435)
When users try to list instances in project mode from Account > SSHKeyPairs > Key > View Instances, we don't need to include sshkeypairs account name into the API Param. It's redundant and causes error.

Fixes #2895
2019-07-02 12:51:44 +05:30
Rohit Yadav
2ecd5ec804
systemvm: don't fork to curl to save password (#3437)
This fixes to avoid forking curl to save password but instead call
a HTTP POST url directly within Python code. This may reduce bottleneck
during high VM launches that require passwords.

Fixes #3182

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-01 23:53:44 +05:30
Abhishek Kumar
9df8d7bc21 ui: create L2, Isolated network - filter offerings for domain
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 18:24:39 +05:30
Abhishek Kumar
d75a26d38b ui: fix for delete VPC offering bug
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 17:04:54 +05:30
Abhishek Kumar
18439ca84b server, api, ui: filtering network offerings for a domain while create network
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 15:17:31 +05:30
Rohit Yadav
4449556aba Merge remote-tracking branch 'origin/4.12' 2019-07-01 14:38:30 +05:30
Rohit Yadav
0649821a21 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-07-01 14:37:40 +05:30
Abhishek Kumar
5766ddb4d1 ui: fix for create network offering domain, zone selection bug
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-07-01 13:51:42 +05:30
Wei Zhou
9ac32aae38 server: VPC redundant vrs run on same hypervisor (#3421)
For VPC supports redundant VRs, when start the second VR, the pod/cluster/host of first VR should be added to avoid list. This provides higher availability.

The network VRs have the same process already.
2019-06-29 12:09:36 +05:30
Rohit Yadav
58c18fc09f
debian: install file as cloudstack-management dependency (#3436)
This install `file` package that provides the `file` tool used by
cloud-install-sys-tmplt script, as part of cloudstack-management
package installation on Ubuntu distros.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-29 00:24:10 +05:30
Rohit Yadav
c160e7532a
ui: fix LB protocol bug (#3438)
Fixes #3252

This fixes labels for missing LB protocol(s).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-29 00:23:36 +05:30
Abhishek Kumar
fc8381549d Merge branch 'master' into storage-offering-domains-zones 2019-06-28 17:33:41 +05:30
Anurag Awasthi
541d280e3b ui: Bug fix for distinguishing between string and map type tags in forms (#3441)
Some APIs consume 'tags' param as string and some consume as maps.
Since each API can have at most one 'tags' param the extraction of
map based tags should only happen when strings based tags are not
extracted from the form serialization.
2019-06-28 17:05:24 +05:30
Abhishek Kumar
54cdab2921 ui: fix for create vpc offering domain selection
Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-28 12:45:35 +05:30
Philipp Bankonier
044e4e4b21 ui: re-add custom css (#3431)
This readds the custom.css file to overwrite the default cloudstack styles.
Currently this is only possible by adding them to the custom.scss in the repository and compile them into the cloudstack3.css.
Now there is an empty custom.css created when compiling the scss source. This custom.css can be overwritten with the custom styles afterwards.

This functionality was broken by #3328.
2019-06-27 19:02:40 +05:30
Andrija Panic
4ce0562cff
Merge pull request #3432 from shapeblue/config_another_cleanup
Remove additional line from config as storagepool isn't available for users
2019-06-27 13:28:08 +02:00
Anurag Awasthi
d967993a0a Remove additional line from config as storagepool isn't available for users 2019-06-27 16:53:37 +05:30
Rohit Yadav
23a8b74e4f
engine/schema: add guest-os support and mappings for XenServer 7.6 (#3427)
This adds hypervisor capabilities and mappings for XenServer 7.6

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 16:13:21 +05:30
Abhishek Kumar
8c4efaf6a2 server, ui: domain-admin update compute and disk offering access
Behaviour followed while updating disk, compute offerings by domain-admins,
- Domain-admins cannot change zones for offerings specified for domains/subdomains.
- Domain-admins can chnage domains(within their subdomains) for the offerings specified for their domains/subdomains.
- Domain-admins cannot change name, display text, sort-key for offerings specified for their domains/subdomains and also other domains which are not child domain for admin.

Fixes UI unidentified button bug for Update offering access form

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-06-27 13:32:19 +05:30
Anurag Awasthi
95a509a1ed ui: Update config file with tables that users can see (#3429)
This is a purely documentation change of config file.
The cleanup was needed post merge of PR #3258
2019-06-27 12:51:29 +05:30
Abhishek Kumar
58474530f6 api: snapshot, snapshotpolicy tag support (#3228)
Problem: Currently tags cannot be applied to snapshot when it is being created but through separate “create tags” API calls. For snapshot policies tags cannot be set either at creation or through “create tags” API.

Root Cause: The “create snapshots” API does not support adding tags during creation and it can only be done through “create tags” API. Snapshot policy as a resource does not support tags and no tags can be set for them through any API.

Solution: Tag support for snapshot policy has been added. Snapshot policy with tags when executed will produce snapshots containing the same tags from snapshot policy.

Following APIs have been updated:

Both “create snapshotpolicy” and “create snapshot” now accepts “tags” as a new parameter. The expected format for “tags” parameter is similar to parameter “tags” in “create tags“ API.
Deletion support for tags associated with snapshots policy has been added to “delete snapshotpolicies” API.
Tags set for snapshot policies are added to the Response of “list snapshotpolicies“ API.
UI support for setting tags to snapshots and snapshot policy is provided through the corresponding menus with a new section in each form to set tags.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-27 09:21:08 +05:30
Rohit Yadav
14bff7bd03
server: export granular volume bytes and iops metrics (#3259)
Problem: The VM metrics has aggregated volume bytes read/write and iops metrics but not on per volume basis.
Root Cause: The volume stats sub-system is not used to export the metrics, the support is not available for VMware.
Solution: Use the volume stats sub-system and DB table to export the metrics via the listVolumes and listVolumeMetrics API, and implement support for VMware and fix issue with network and disk metrics in the VM metrics view.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 09:18:10 +05:30
Rohit Yadav
9f4f2c5348
api: instance and template details are free text (#3240)
Problem: Users don't know what keys/values to enter for template and VM details.
Root Cause: The feature does not exist that can list possible details and options.
Solution: Based on the possible VM and template details handled by the
codebase, those details were refactored and a list API is introduced
that can return users those details along with possible values. When
users add details now, they will be presented with a list of key details
and their possible options if any.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-27 09:14:47 +05:30