32675 Commits

Author SHA1 Message Date
Pearl Dsilva
0d487fc8c9
support for data migration of incremental snaps on xen (#4395)
* support for handling incremental snaps (on DB entries) on xen

* Addressed comments

* Update NfsSecondaryStorageResource.java

adjusted space in comment/ log

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-18 02:15:10 +05:30
Rodrigo D. Lopez
c222d0bf60
enable update tags on disk offerings (#4194) 2020-10-16 09:22:42 +02:00
Rodrigo D. Lopez
d676ffa0a0
Handle with VM snapshot events (#4251)
Co-authored-by: Rodrigo <rodrigo@scclouds.com.br>
2020-10-16 09:19:44 +02:00
Gabriel Beims Bräscher
4c936b4690
Re-add affinity group (#4405) 2020-10-16 09:16:19 +02:00
Daan Hoogland
644731edad Merge branch '4.14' 2020-10-15 18:02:00 +00:00
Wei Zhou
f7a3701c7a
server: fix issue that vm guest os type is reset after updatetemplate (#4377) 2020-10-15 17:54:59 +00:00
Wei Zhou
be5df8d9ed
kvm: fix wrong VM CPU usage (#4381) 2020-10-15 13:57:59 +02:00
Rakesh
6c88e9afb3
Dont add host back after agent service restart (#4228) 2020-10-14 16:49:39 +00:00
Rohit Yadav
d6db47618d Merge remote-tracking branch 'origin/4.14' 2020-10-14 16:06:57 +05:30
Abhishek Kumar
5b419b90d0
vmware: use hotPlugMemoryIncrementSize only for valid value (#4348)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-10-14 16:05:57 +05:30
davidjumani
16bb05351a
api: Making id a required field in scalekubernetesclustercmd (#4404) 2020-10-14 15:55:22 +05:30
Wei Zhou
55f07030cb
plugins: Host is counted twice if it has multiple host tags in Prometheus exporter (#4383)
* Hosts are counted twice if it has multiple host tags in Prometheus exporter

* Import HostVO and inject HostDao
2020-10-14 15:51:15 +05:30
Wei Zhou
40724e66e8
debian/control: add uuid-runtime to cloudstack-common, ufw/apparmor to cloudstack-agent (#4382) 2020-10-14 15:50:11 +05:30
Wei Zhou
2acd87c41e
server: Add global configuration vm.serviceoffering.cpu.cores.max and vm.serviceoffering.ram.size.max (#4379)
vm.serviceoffering.cpu.cores.max and vm.serviceoffering.ram.size.max
2020-10-14 15:48:35 +05:30
Pearl Dsilva
cdcc914995
scripts: creates /var/run/cloud folder for creation of lock file (#4373)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-10-14 15:45:22 +05:30
Rakesh
2333d97098
plugins: Consider maintenance mode as offline for promethues stats (#4366)
If the resource state of hypervisor in "Maintenance" then it
should be considered as offline even though the agent state
is "Up". Since its in maintenance mode, it cant be used to
allocate VM's and hence can't be considered towards resource
allocation
2020-10-14 15:42:02 +05:30
Rakesh
191dbf7ea7
plugins: Export dedicated host stats to prometheus (#4365)
We should have the metrics for the hosts which are dedicated to certain domains.
We should also be able to see cpu/memory/storage currently used per domain

> How Has This Been Tested?
Enable prometheus server
Add 127.0.0.1 as allowed Ip so that you can fetch metrics from prometheus

Now fetch the endpoint
# http http://127.0.0.1:9595/metrics | grep cloudstack_host_is_dedicated
cloudstack_host_is_dedicated{zone="mgt122-10",hostname="node11",ip="10.13.122.11"} 1
# http http://127.0.0.1:9595/metrics | grep cloudstack_host_dedicated_to_account
cloudstack_host_dedicated_to_account{zone="mgt122-10",hostname="node11",ip="10.13.122.11"} 1
2020-10-14 15:41:10 +05:30
Daan Hoogland
b65282eaa9 Merge branch '4.14' 2020-10-14 10:54:35 +02:00
Gabriel Beims Bräscher
2e32a3a94a
List VMs by Security Group & HA (#4397) 2020-10-14 09:26:47 +02:00
Wei Zhou
94f9f89736
server: Fix some cpuspeed issues while create service offering (#4376) 2020-10-13 11:32:16 +02:00
davidjumani
aab8df09aa
Fixing searchAndCount searchAndDistinctCount when sc is null (#4374) 2020-10-13 11:30:04 +02:00
Spaceman1984
1bbb29d034
Fixed vm-templates not being removed from primary storage with storag… (#4389)
* Fixed vm-templates not being removed from primary storage with storage garbage collection

* Update vmware-base/src/main/java/com/cloud/hypervisor/vmware/mo/VirtualMachineMO.java

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>

* Var name

Co-authored-by: sureshanaparti <12028987+sureshanaparti@users.noreply.github.com>
2020-10-12 15:54:12 +05:30
Abhishek Kumar
d3be14d4c8
hypervisor: Add Citrix Hypervisor 8x product name support (#4271)
* changes for supporting Citrix Hypervisor product name

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-10-12 15:50:43 +05:30
Spaceman1984
479ef8aafa
VMware: match hardware version for worker VM when taking a snapshot (#4321)
* Add hardware version to worker VM

* Added worker VM hardware version when creating a template from a volume and migrating a detached volume

* Add null parameter back that was removed by merge
2020-10-09 17:06:07 +02:00
davidjumani
cde7fbff05
createaccountcmd: Improving account param description (#4354) 2020-10-07 14:46:33 +05:30
Gabriel Beims Bräscher
0f3f2a0937
oobm: Retry redfish requests (#4352)
It is not common, but HTTP requests can fail due to connection issues. In order to mitigate such situations and also improve logging, this PR enhances the Redfish request handling by adding an execution flow for re-trying HTTP requests; the retry happens only if the global settings redfish.retries is set to 1 or more retries; default is of 2 (two). One can disable the retries by setting redfish.retries to 0 (zero).
2020-09-30 16:35:17 +05:30
Rohit Yadav
b3bafffff3 Merge remote-tracking branch 'origin/4.14' 2020-09-29 14:33:58 +05:30
Abhishek Kumar
9391fa9b6b
cks: assorted fixes, test refactoring (#4269)
Fixes #4265
Fixes for:

template selection
network cleanup
capacity check
specific patch version during deployment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-29 14:31:41 +05:30
Wei Zhou
98c51a6d3d
server: check guest os preference of last host when start a vm (#4338)
If vm has last host_id specified, cloudstack will try to start vm on it at first.
However, host tag is checked, but guest os preference is not checked.

for new vm, it will be deployed to the preferred host as we expect.

Fixes: #3554 (comment)
2020-09-29 12:45:29 +05:30
slavkap
d213a4d956
systemd: Binding listening socket to all address for remote debug (#4345)
Since Java 9 the notation 'address=port' only applies to localhost.
For remote debug you have to explicitly specify that you want to listen
to all IP addresses (e.g. address=*8000)
2020-09-29 12:26:14 +05:30
davidjumani
e0a842cf3a
test: Changing test_pvlan vlan id to prevent conflict with env config (#4340) 2020-09-29 10:58:00 +05:30
Rohit Yadav
86fcb14238 Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:20:29 +05:30
Rohit Yadav
1efe6e2df0 Merge remote-tracking branch 'origin/4.13' into 4.14
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 12:13:23 +05:30
Gabriel Beims Bräscher
d6152b37ad
server: Broadcast URI not set to vxlan, but vlan (Fix #3040) (#4190)
This PR sets properly Broadcast URI to vxlan://vxlan_id when the physical network is of VXLAN.

Fixes: #3040
2020-09-24 11:09:50 +05:30
Abhishek Kumar
ca1e02fab9
vmware: search unmanaged instances using hypervisor name (#4328)
VMware code keeps a cache of existing VMs on a hypervisor host using cloud.vm.internal.name property of the VM. Searching for unmanaged instances/VMs on a host might not return an expected result when this property differs from the actual name of the VM.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-24 11:01:33 +05:30
Rohit Yadav
4f8b88baea
vmware: while plugging in nics get existing sorted nic devices (#4336)
In large environments, with VR having multiple nics when plugging in
nic, it must get existing nics by sorted device ID otherwise it may
cause incorrect nic plugging/order.

Fixes #4246

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-24 10:59:39 +05:30
davidjumani
d657ef7d5b
debian: Changing dependency from python3-distutils to python3-distutils-extra (#4305)
* Changing dependency from python3-distutils to python3-distutils-extra

* Update debian/control

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-23 15:30:17 +05:30
Wei Zhou
534dd475d8
agent: Compare indirect agent lb algorithm when cloudstack agent connects (#4335)
Compare not only the list of management servers but also the lb algorithm when agent connects.

Fixes: #3895
2020-09-23 12:48:16 +05:30
Rohit Yadav
81c0bec765 Merge remote-tracking branch 'origin/4.14' 2020-09-23 10:50:24 +05:30
Rohit Yadav
766eab8cab Merge remote-tracking branch 'origin/4.13' into 4.14 2020-09-23 10:49:19 +05:30
Lucas Granet
ab02cf7078 router: adding "data-server" dns entry in /etc/hosts (#4319)
The DNS entry "data-server" was not added in /etc/hosts.

Since the VR is now considered as a "dhcpsrvr" (?), we need to apply this commit to add this DNS entry.
/etc/hosts is fully rewritten by this script.

Fixes: #4308
(cherry picked from commit dc65f31f9f3cb47240946c8c1cced44a7ecf9640)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-09-23 10:48:44 +05:30
Wei Zhou
00ceafe47f
securitygroup: Ubuntu 20.04 fix systemvm cannot start up (#4303)
* security_group.py: fix SyntaxWarning: "is" with a literal.

2020-04-27 09:43:54,172 DEBUG [kvm.resource.LibvirtComputingResource] (Agent-Handler-2:null) (logid:c33ba330) /usr/share/cloudstack-common/scripts/vm/network/security_group.py:513: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if rules is None or rules is "":
/usr/share/cloudstack-common/scripts/vm/network/security_group.py:522: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if rules is None or rules is "":
/usr/share/cloudstack-common/scripts/vm/network/security_group.py:823: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if brName is None or brName is "":

* Ubuntu 20.04: Fix systemvm cannot start up

in Ubuntu 16.04:

root@node13:~# bridge -o link show
2: eth0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
5: vnet0 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloud0 state forwarding priority 32 cost 100
6: vnet1 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
7: vnet2 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100

root@node13:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1
eth0

root@node13:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1 |cut -d ":" -f1
eth0

in Ubuntu 20.04:

root@node62:~# bridge -o link show
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
10: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloud0 state forwarding priority 32 cost 100
11: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100
12: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cloudbr0 state forwarding priority 32 cost 100

root@node62:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1
ens3:

root@node62:~# bridge -o link show | awk '/master cloudbr0 / && !/^[0-9]+: vnet/ {print $2}' | head -1 |cut -d ':' -f1
ens3

* security_group.py: use 'if not' instead
2020-09-22 20:56:54 +05:30
davidjumani
ead9a34b3d
Disabling managing firewall - cloudstack-setup-management (#4239)
* Adding message to ensure ports are open

* Removing configuring iptables

* Fixing merge conflict
2020-09-22 17:46:21 +05:30
davidjumani
c06e7ded3c
systemvm: update novnc v1.2.0 (#4323)
Update noVNC v1.2.0, add support for clipboard, explicit button toolbar and resize screensize
2020-09-22 17:42:30 +05:30
Lucas Granet
dc65f31f9f
router: adding "data-server" dns entry in /etc/hosts (#4319)
The DNS entry "data-server" was not added in /etc/hosts.

Since the VR is now considered as a "dhcpsrvr" (?), we need to apply this commit to add this DNS entry.
/etc/hosts is fully rewritten by this script.

Fixes: #4308
2020-09-22 13:07:56 +05:30
Pearl Dsilva
cfbb4ff3dd
schema: change upgrade path to 4.14 (from 4.13) and intensify check (#4331)
* change upgrade path to 4.14 (from 4.13) and intensify check

* extracted check

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-22 09:40:51 +05:30
Andrija Panic
238eccc317
packaging: Minor message update (#4333)
adding quotes, to fix the "servers" to "server's"
2020-09-21 14:12:52 +05:30
Pearl Dsilva
90e72b1e40
vmware: Create template from detached data-disks on VMWare (#4294)
Creation of templates from detached data disks results in a Null Pointer Exception on VMWare, as it expects the volume to be attached to a VM.
To fix this behavior and make it consistent with other hypervisors, creation of the template from the volume in case not attached to a VM is facilitated by creating a worker VM, attaching the disk to the worker VM, creating the template from it, and then destroying the VM.

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-18 08:25:17 +05:30
Pearl Dsilva
82b6971258
server: Handle listProjects API to list projects with user as members when listAll=true (#4316)
* added defensive checks for avoiding NPE and list projects API fix

* list projects with account name provided to not include users in the account in response

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-09-17 10:20:34 +05:30
Abhishek Kumar
87e08f8224
cks: fix logging exception (#4309)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-09-17 10:17:07 +05:30