32174 Commits

Author SHA1 Message Date
Rohit Yadav
ff23131701 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-05 10:00:45 +05:30
Rohit Yadav
8fb388e931
router: support multi-homed VMs in VPC (#3373)
This does not remove VM entries in dbags when hostnames match. The
current codebase already removes entry when a VM is stopped/removed so
we don't need to handle lazy removal. This will allow a VM on
multiple-tiers in a VPC to get dns/dhcp rules as expected.

This also fixes the issue of dhcp_release based on a specific interface and
removes dhcp/dns entry when a nic is removed on a guest VM.

Fixes #3273

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-05 08:47:05 +05:30
Richard Lawley
41f569e8a8 router: Fix rule duplication with non-VPC static NAT rules (#3366)
The VR code has provision for inserting rules at the top or bottom by specifying "front" as the second parameter to self.fw.append. However, there are a number of cases where someone has been unaware of this and added a rule with the pattern self.fw.append(["mangle", "", "-I PREROUTING".... This causes the code to check for the rule already being present to fail, and duplicate rules end up being added.

This PR fixes two of these cases which apply to adding static NAT rules. I am aware of more of these cases, but I don't have the ability to easily test the outcome of fixing them. I'm happy to add these in if you're confident that the automated tests will be sufficient. Searching for "-I (case sensitive) finds these.

The code for dealing with "front" is included below to show that this shouldn't have any ill effects:

if fw[1] == "front":
    cpy = cpy.replace('-A', '-I')

Fixes #3177
2019-06-05 02:21:03 +05:30
nvazquez
50869fec92 Merge branch '4.12' 2019-06-04 09:49:39 -03:00
nvazquez
bea627a52e Merge branch '4.11' into 4.12 2019-06-04 09:06:09 -03:00
Vladimir Melnik
c94ee1454d kvm: suspend a VM before snapshot deletion (see PR #3193) (#3194)
To make sure that a qemu2-image won't be corrupted by the snapshot deletion procedure which is being performed after copying the snapshot to a secondary store, I'd propose to put a VM in to suspended state.

Additional reference: https://bugzilla.redhat.com/show_bug.cgi?id=920020#c5

Fixes #3193
2019-06-04 16:04:45 +05:30
Rohit Yadav
42501ceecf
ssvm: apply MTU value on storage/management nic if available (#3370)
If mtu= value is defined in the parameters received by the SSVM agent
per the secstorage.vm.mtu.size setting, it applies the MTU setting on
eth1 which is the storage/management nic.

Fixes #3369

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-04 15:53:51 +05:30
Nicolas Vazquez
12c850ed2f
KVM: Improvements on upload direct download certificates (#2995)
* Improvements on upload direct download certificates

* Move upload direct download certificate logic to KVM plugin

* Extend unit test certificate expiration days

* Add marvin tests and command to revoke certificates

* Review comments

* Do not include revoke certificates API
2019-06-04 03:08:31 -03:00
Rohit Yadav
3e4c55c424 Merge remote-tracking branch 'origin/4.12' 2019-06-03 18:56:54 +05:30
Rohit Yadav
3f2399f107 server: Fix forward merge missing import causing build failure
This fixes forward merge regression that missed an import and causes
build failure in b2b99ca63eecab6c0be40015ee54c37b55e07bac

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 18:55:25 +05:30
Rohit Yadav
cd6e84659e Revert "Fix template size for managed storage / refactor cloud-install-sys-tmplt and createtmplt.sh (#3346)"
This reverts commit 6946f41784423788e71441255e67c2b9b1c3633d because it
adds regression to deployment and testing. Testing against VMware, KVM
and XenServer confirm failure to deploy an environment where systemvms
don't come online.
2019-06-03 17:29:43 +05:30
Rohit Yadav
1c5244a0c5 Merge remote-tracking branch 'origin/4.12' 2019-06-03 17:17:51 +05:30
Rohit Yadav
b2b99ca63e Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 17:15:41 +05:30
Nicolas Vazquez
c9ce3e2344 router: Persistent DHCP leases file on VRs and cleanup /etc/hosts on VM deletion (#3351)
Since the CloudStack virtual router was redesigned on version 4.6 it has been observed that the DHCP leases file is not persistent across network operations. This causes conflicts on guest VMs static IPs, causing these static IPs to not be renewed by the DHCP server running on isolated and VPC networks' virtual routers (dnsmasq). On stopping or destroying a VM, its dhcp/dns records are not removed from the virtual router causing ghost effects.

Fixes #3272
Fixes #3354

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-06-03 17:04:16 +05:30
Richard Lawley
2484527cae srx: Fix removing static NAT rules with Juniper SRX (#3310)
Fixed the logic for deleting static NAT rules on a Juniper SRX device. Previously the private (trust) rule was not being removed.

Fixes #3309
2019-06-03 16:51:23 +05:30
Sven Vogel
6946f41784 Fix template size for managed storage / refactor cloud-install-sys-tmplt and createtmplt.sh (#3346)
* refactor cloud-install-sys-tmplt and createtmplt.sh

* move qemu-img check to kvm / add more comments if qemu-img not available / set virtual size only for qcow2

* add comments to the failed and get options block / rename vars in failed block / typo Fix

*  add comments to the failed and get options block / rename vars in failed block / typo Fix to cloudtmplt.sh
2019-06-03 09:30:45 +02:00
Abhishek Kumar
6c6f40fd4d server: fix for public IP addresses filtering (#3368)
Added missing parameter in listPublicIpAddresses API.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-06-03 12:52:19 +05:30
Abhishek Kumar
852a228321 server: fix for incorrect createVPCOffering argument
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-31 13:00:33 +05:30
Rohit Yadav
28d7eef21f Merge remote-tracking branch 'origin/4.12' 2019-05-31 12:37:31 +05:30
Rohit Yadav
fb555b11ae Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-31 12:36:45 +05:30
Richard Lawley
2f268fbb52 systemvm: fix VR issues with Multiple Public Subnets (#3361)
This PR resolves 2 issues related to Virtual Routers with multiple public interfaces, and works around a third.

- Fixes #3353 - Adds missing throw routes for eth0/eth1 to eth3+ when there are >1 public IPs
- Fixes #3168 - Incorrect marks set on some static NAT rules (some code references were changed from hex(int(interfacenum)) to hex(100 + int(interfacenum)) - this change just adds the remaining ones
- Fixes #3352 - Work around that sends Gratuitous ARP messages when a HA VR becomes master to work around the problem of the MAC address being different between HA VRs. If that issue is fixed properly (i.e. a database entry for the subsequent interfaces so they can be static) then this is unnecessary, though should not cause any problems.
2019-05-31 12:35:42 +05:30
Rohit Yadav
4ba67b4c59 Merge remote-tracking branch 'origin/4.12' 2019-05-31 12:32:11 +05:30
Rohit Yadav
3737b41dbc Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-31 12:31:27 +05:30
ustcweizhou
b8522c97cb server: allow dedicate ip range to a domain if ips are used by an accout in the domain (#3206)
when we dedicate public ip range to a domain but some ips are used by an account in the domain,
the operation should be allowed but actually fails for now.
It is because cloudstack check if ips are used by same account by account name,
However, accountName is null when dedicate public ip range to a domain.

Modify the code to check account id only when dedicate ip range to account.
2019-05-31 12:24:33 +05:30
ustcweizhou
bd78030385 server: update dhcp configurations in vrs while update default nic of running vms (#3205)
In virtual routers, there are different dnsmasq settings for default nic and non-default nic on vm.
We need to update dhcp informations on network vrs when default nic is changed.

For example, if 172.16.1.135 is non-default nic of vm VPC1-001-001, then

root@r-22-VM:~# cat /etc/dhcphosts.txt
02:00:1d:15:00:05,set:172_16_1_135,172.16.1.135,VPC1-001-001,710h
root@r-22-VM:~# cat /etc/dhcpopts.txt
172_16_1_135,3
172_16_1_135,6
172_16_1_135,15

If it is default nic,then

root@r-22-VM:~# cat /etc/dhcpopts.txt
root@r-22-VM:~# cat /etc/dhcphosts.txt
02:00:1d:15:00:05,172.16.1.135,VPC1-001-001,757h

Fixes #3201
2019-05-31 12:23:55 +05:30
Rohit Yadav
8c387f9de6
vmware: fix potential NPE when memory hotplug capability is checked (#3362)
This fixes potential NPE case when memory hotpluggability is checked
based on the guest OS descriptor.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-31 10:27:36 +05:30
Andrija Panic
fd4309cc3d server: increase POST timeout for local template upload (#3356)
Increase template upload POST timeout for local template upload to 1 hr or 60 minutes.
2019-05-31 09:32:59 +05:30
Abhishek Kumar
1323036bba refactoring: fixed comment typo
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-30 16:55:28 +05:30
Abhishek Kumar
f78b99cc8f refactoring: code review comment changes
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-30 16:42:14 +05:30
Andrija Panic
964430ab23 vmware: update vmware reservations settings description (#3358)
We do NOT always reserve VMware CPU/RAM resources - only when "vmware.reserve.cpu" or "vmware.reserve.mem" setting is set to TRUE - AND we do so, irrelevant if overprovisioning is active or not. Verified for both system VMs and user VMs.
2019-05-30 16:17:48 +05:30
Anurag Awasthi
3bf4e5c498 ui: configurable branding, keyboard list and hide-able columns through a new config.js file (#3258)
We want to support hiding table columns, specifically in metrics table, through config file so that users can make the relevant bits hidden as per their organization. Current work will support the metrics table but can be extended to any table with minimal work in future.

Config file will take the key of the metrics column from metrics.js file for the sake of minimal changes and simplicity of development.

Problem: The keyboard list in the UI is not consistent across views such as in the instance wizard and in the register template form. There is also no way to custom about url/text and doc title and help URL in the UI.
Root Cause: The list is hardcoded in the UI allowing no centralised configuration.
Solution: Introduce a new config.js file installed at the /usr/share/cloudstackmanagement/webapp/config.js location. The config.js allows configurable keyboard list, about url/text, doc title, and help URL.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-30 15:32:37 +05:30
Rohit Yadav
6d82e63e53
Merge branch 'master' into storage-offering-domains-zones 2019-05-30 15:04:32 +05:30
Rohit Yadav
0f4b1f511f Merge remote-tracking branch 'origin/4.12' 2019-05-30 12:31:26 +05:30
Rohit Yadav
7ef921de16 Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 12:30:54 +05:30
ustcweizhou
f6f381fc68 ui: fix enable static nat only towards first nic and not on any other interface (#3338)
When enable static nat in a vpc on UI, it only lists the primary and secondary ips of first nic of a vm, no matter which vpc tier is selected. The same issue happens when add a vm to load balancer.

Fixes #3334
2019-05-30 11:44:52 +05:30
Rohit Yadav
c072dda88e Merge remote-tracking branch 'origin/4.12' 2019-05-30 11:41:38 +05:30
Rohit Yadav
31e677f0fa Merge remote-tracking branch 'origin/4.11' into 4.12 2019-05-30 11:40:19 +05:30
smlshn
f1efcc1af6 ui: reset multiselect actions when refreshing listView in Instance page (#3359)
Enables the toolbar to reset to its initial state after any multiSelectAction completed.

Fixes #3337
2019-05-30 11:38:14 +05:30
ustcweizhou
8e43d258f3 server: Fail to restart VPC with cleanup if there are multiple public IPs in different subnet" (#3342)
If there are multiple IPs in different subnet assigned to a VPC, after restarting VPC with cleanup, the VRs will be FAULT state.

Step to reproduce:
(1) create vpc, source nat IP is 10.11.118.X
(2) assign two public IPs in other subnet to this VPC. 10.11.119.X and 10.11.119.Y
(3) deploy two vms in the vpc, and enable static nat 10.11.119.X and 10.11.119.Y to these two vms
(4) restart vpc with cleanup. There are more than 1 nic allocated for 10.11.119 to new VRs

Logs as below:
2019-05-10 14:12:24,652 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.118.157-vlan://untagged
2019-05-10 14:12:24,676 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119
2019-05-10 14:12:24,699 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119
2019-05-10 14:12:24,723 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-36:ctx-839f6522 job-652 ctx-35fb4667) (logid:1ab7aa37) Allocating nic for vm VM[DomainRouter|r-85-VM] in network Ntwk[200|Public|1] with requested profile NicProfile[0-0-null-10.11.119.110-vlan://119

This is a regression issue caused by commit 1d382e0
2019-05-30 11:33:03 +05:30
dahn
910b08f72b server: fix duplicate tag exception as CloudRuntimeException (#3348)
See #3339: a runtime exception is thrown but it should be converted to an error return. Wrapping it in a CloudRuntimeException should do the trick.

Fixes #3339
2019-05-30 11:25:52 +05:30
Nicolas Vazquez
501aa7cd91
DPDK vHost User mode selection (#3153)
* DPDK vHost User mode selection

* SQL text field and DPDK classes refactor

* Fix NullPointerException after refactor

* Fix unit test

* Refactor details type
2019-05-29 08:36:33 -03:00
Rohit Yadav
4e8f14975a
ui: add memory used column in instance metrics view (#3243)
This adds memory used column in the instance metrics view. Also fixes
a bug for VMware, due to which incorrect memory usage was returned.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-28 20:35:37 +05:30
Spaceman1984
de18613206 User allowed to tag project created by him (#3323) 2019-05-28 10:32:39 +02:00
ustcweizhou
798b79fa5b kvm: disable cpu features if feature starts with '-' (#3335)
When I use SandyBridge as custom cpu in my testing, vm failed to start due to following error:
```
org.libvirt.LibvirtException: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: avx, xsave, aes, tsc-deadline, x2apic, pclmuldq
```

With this patch, it works with the following setting in agent.properties:
```
  guest.cpu.mode=custom
  guest.cpu.model=SandyBridge
  guest.cpu.features=-avx -xsave -aes -tsc-deadline -x2apic -pclmuldq
```

vm cpu is defined as below:
```
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>SandyBridge</model>
    <feature policy='disable' name='avx'/>
    <feature policy='disable' name='xsave'/>
    <feature policy='disable' name='aes'/>
    <feature policy='disable' name='tsc-deadline'/>
    <feature policy='disable' name='x2apic'/>
    <feature policy='disable' name='pclmuldq'/>
  </cpu>
```
2019-05-27 18:43:38 +05:30
Abhishek Kumar
d1090c0577 server: fix for inactive service offering for VM (#3320)
Fixes #3315
Currently, the code was allowed to change service offering for VM to a deleted or inactive service offering. Added check for it to throw an exception.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-27 18:27:23 +05:30
Gabriel Beims Bräscher
25c4f7fc08 kvm: Remove code that generated /var/lib/libvirt/images/null on target host (#3280)
This commit simplifies the generateDestPath method and fixes an issue where an extra file, named as 'null', was created on the target storage pool during VM local storage volume migration. Without this fix, the VM is migrated and there is no data loss; however, 193 KB is allocated for the unused file named as 'null' and the file stays on the target storage.
2019-05-27 18:15:29 +05:30
Philipp Bankonier
e8c1deb25d ui: Fix ip and ip cidr column sorting in tables (#3199)
This fixes the sorting of ip addresses and ip cidrs.
Currently they get evaluated as a numeric value via parseFloat and sorted as numeric values.
For example "10.32.123.123" got parsed to 10.32.
2019-05-27 18:09:56 +05:30
Abhishek Kumar
46a32795bc ui: instance settings visibility (#3244)
This change allows instance Settings tab to be visible but inaccessible when instance is running. A warning is shown when user tries to access Settings for a running instance and tab content is greyed out.
It also allows some admin defined instance settings/details to be made static for user. User will be able to see them in instance settings tab but cannot change their values as action buttons are disabled and greyed out. This can be achieved by providing a comma-separated list details for global settings key 'user.vm.readonly.ui.details'. A new value 'readonlyuidetails' has been added in UserVMResponse for UI manipulate editing functionality of settings/details.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 12:28:15 +02:00
Abhishek Kumar
179d229bd7 ui: fixed create offering domain multi-selection
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-05-24 14:50:57 +05:30
Sven Vogel
9d02b3679b packaging: set correct logrotate file permissions for cloudstack-agent (#3347)
Correct the permissions on cloudstack-agent logrotate file (/etc/logrotate.d/).
2019-05-24 12:43:08 +05:30