34697 Commits

Author SHA1 Message Date
Nicolas Vazquez
a5372a98dc
Fix storage cleanup corner case preventing VM deletion (#5575)
* Fix storage cleanup corner case

* Improve deletion

* Refactor
2021-10-16 00:09:54 -03:00
Hoang Nguyen
26b5d9423a
fix the error of not being able to read the length of numeric (#5577) 2021-10-14 14:20:29 +05:30
Pearl Dsilva
e73445e5d5
api: Fix response object for various APIs (#5573) 2021-10-13 22:26:53 -03:00
Wei Zhou
6b757d502e
CKS: use cluster-autoscaler-standard.yaml in kubernetes repo (#5574) 2021-10-13 12:46:52 -03:00
Pearl Dsilva
1b464b4d06
api: Fix RestartNetwork response type (#5571) 2021-10-13 11:03:59 -03:00
Rohit Yadav
ad8f3750c9
engine/schema: add unique constraint for sshkeys UUID column (#5565)
This adds unique constraints much like other tables, instead of using
query that maybe incompatible with older 5.x mysql servers.

Fixes #5564

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-10-12 10:08:30 -03:00
Pearl Dsilva
4ed916d639
UI: Restrict viewing project invitation options when configuration is disabled (#5572) 2021-10-12 14:23:21 +05:30
Hoang Nguyen
daf29a085e
fix display ip address allow input (#5569) 2021-10-12 14:22:58 +05:30
Pearl Dsilva
79e6fe47c8
Fix warning caused due to duplicate declaration of plugin - pom.xml (#5568) 2021-10-12 10:10:53 +05:30
sureshanaparti
152159b13a
kvm: Add the source disk format for disk conversion/copy using 'qemu-img convert', when specified explicitly. (#5561)
This PR adds the source disk format (in KVM) for disk conversion/copy using 'qemu-img convert', when specified explicitly

Fixes: #5516
2021-10-11 12:00:49 +05:30
sureshanaparti
07efd7c19d
Updated storage type of the volume, in the volume response, based on the underlying storage pool. (#5560) 2021-10-09 21:22:20 -03:00
Nicolas Vazquez
45967cffb2
marvin: Use deploy as is for Vmware tests in test_data (#5557)
This PR updates the test data after fixes for Vmware 7 smoke tests issues
2021-10-08 12:59:49 +05:30
Gabriel Beims Bräscher
404e264caf
CloudStack fails to migrate VM with volume when there are datadisks attatched (#5410)
* Check if should map volume in createStoragePoolMappingsForVolumes

* Invert conditional at internalCanHandle
2021-10-08 11:50:37 +05:30
Wei Zhou
798d7bebf6
VR: skip dhcp/dns health check in some cases (#5554)
this fixes #5536
2021-10-08 08:22:32 +05:30
Abhishek Kumar
2a42a01728
xcp-ng: fix vm boot options (#5543)
* xcp-ng: fix vm boot options

Use value boot option values from VM details directly similar to other hypervisor plugins instead of relying on boot properties explicitly set for VirtualMachineTO.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2021-10-07 13:22:23 +05:30
davidjumani
6ac834a358
Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements (#4329)
Adding AutoScaling support for cks
Kubernetes PR : kubernetes/autoscaler#3629
Also replaces CoreOS with Debian
Fixes #4198

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-10-06 21:17:41 +05:30
sureshanaparti
e4beb1f955
Add empty config value for scope based config setting (#5551) 2021-10-05 18:49:33 -03:00
sureshanaparti
e198edfaf8
Report the PowerFlex/ScaleIO disk copy failure during volume migration and fail the migration (#5542)
* Report the PowerFlex/ScaleIO disk copy failure during volume migration and fail the migration.

* Code improvements

* Addressed review comments
2021-10-05 14:19:52 -03:00
div8cn
52a9dbdcd2
kvm available memory calculation optimization (#5540)
* Update LibvirtComputingResource.java

* Update LibvirtComputingResource.java

* Update LibvirtComputingResource.java

re

* Update MemStat.java

* Update MemStat.java

* Update MemStatTest.java
2021-10-05 12:34:28 -03:00
Harikrishna
cd4e7e031a
Fix resize volume and migrate volume to update volume path if DRS is applied on volume in datastore cluster (#5539)
* Fix resize volume and migrate volume to update volume path if DRS is applied on volume in datastore cluster

* Change in constructors

* Naming changes

* Remove commented code

* Refactor code for more readability

* Addressed review comments on code refactor
2021-10-04 20:58:25 -03:00
Abhishek Kumar
6e216dd0d1
vmware, network: add maclearning option (#5471)
* vmware, network: add maclearning option

Adds option for specifying MAC Learning property for network offering (useful for VMware Distributed Virtual Portgroup). Added global config - network.mac.learning for the default value.
MAC Learning is supported for DV portgroups for VMware Distributed vSwitches v6.6.0+ and vSphere 6.7+

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix warning msg

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-10-04 20:00:45 -03:00
dahn
ad4def5d1b
an inject annotation short (#5547)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-10-04 21:20:31 +02:00
dahn
3574d8d20b
parallel nic adding (#5541)
* trace nics additions

* work queue patch for network to add

* add secondary key to job

* logging improvements and naming of field(s)

* several naming corrections

* extra check if net already exists for vm

* placeholder job with secondary object

* constraint on entering the same job multiple times

* error handling/warning message

* review comments applied

Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Wei Zhou <wei.zhou@shapeblue.com>
2021-10-04 12:21:21 -03:00
SadiJr
93c0b602db
[UI] Edit backup offering (#5546)
* Allow edit backup offering

* Address reviews

Co-authored-by: SadiJr <sadi@scclouds.com.br>
2021-10-04 12:12:51 -03:00
Wei Zhou
9f5ac89c9a
VR: fix data-server if shared network has multiple ip ranges (#5530)
* VR: fix data-server if shared network has multiple ip ranges

This fixes #5518

* Update PR #5530: fix nameserver in vm with IP in additional IP ranges

without this change
```
root@r-757-VM:~# cat /etc/dnsmasq.d/cloud.conf
listen-address=127.0.0.1,10.10.12.31,10.10.13.19
dhcp-range=set:interface-eth0-0,10.10.12.31,static
dhcp-option=tag:interface-eth0-0,15,cs1cloud.internal
dhcp-option=tag:interface-eth0-0,6,10.10.12.31,10.0.32.1,8.8.8.8
dhcp-option=tag:interface-eth0-0,3,10.10.12.254
dhcp-option=tag:interface-eth0-0,1,255.255.255.0
dhcp-range=set:interface-eth0-1,10.10.13.19,static
dhcp-option=tag:interface-eth0-1,15,cs1cloud.internal
dhcp-option=tag:interface-eth0-1,6,10.10.12.31,10.0.32.1,8.8.8.8 <<< nameserver 10.10.12.31
dhcp-option=tag:interface-eth0-1,3,10.10.13.254
dhcp-option=tag:interface-eth0-1,1,255.255.255.0
```

with this change
```
root@r-757-VM:~# cat /etc/dnsmasq.d/cloud.conf
listen-address=127.0.0.1,10.10.12.31,10.10.13.19
dhcp-range=set:interface-eth0-0,10.10.12.31,static
dhcp-option=tag:interface-eth0-0,15,cs1cloud.internal
dhcp-option=tag:interface-eth0-0,6,10.10.12.31,10.0.32.1,8.8.8.8
dhcp-option=tag:interface-eth0-0,3,10.10.12.254
dhcp-option=tag:interface-eth0-0,1,255.255.255.0
dhcp-range=set:interface-eth0-1,10.10.13.19,static
dhcp-option=tag:interface-eth0-1,15,cs1cloud.internal
dhcp-option=tag:interface-eth0-1,6,10.10.13.19,10.0.32.1,8.8.8.8 <<< nameserver 10.10.13.19
dhcp-option=tag:interface-eth0-1,3,10.10.13.254
dhcp-option=tag:interface-eth0-1,1,255.255.255.0
```

* Update #5530: add 'localise-queries' to /etc/dnsmasq.conf
2021-10-04 11:40:25 +02:00
Leo (Hsueh Yu-Min)
72a1c0e7f1
[KVM] Add MV Settings for virtual GPU hardware type and memory (#5513)
* KVM: Add MV Settings for virtual GPU hardware type and memory

* fix method createVideoDef argument in test package

* add available options for KVM virtual GPU hardware VM setting

* fix videoRam default value

* fix _videoRam is 0, it will use default provided by libvirt
2021-10-04 09:55:32 +05:30
Wei Zhou
669ab73efe
server: check service offering (storage) tags when reallocate a ROOT disk (#5501)
* server: check service offering (storage) tags when reallocate a ROOT disk

* server: resize volumes in Allocated state
2021-10-03 19:45:59 -03:00
Daniel Augusto Veronezi Salvador
e92ce916f3
network: Remove logic that creates gap for multiple 'source NAT' in VR (#5532)
In ACS, when a VPC has more than one public IP and a user tries to use the non `source NAT` IP with some feature/option (like `static NAT`, `port forwarding`, `VPN` and others), ACS adds the public IP (used for the feature/option) as `source NAT` in the `iptables` of the VR.

Example:
- VPC has one public IP `192.168.0.50` and it is defined as the `source NAT`.
- If we assign another public IP `192.168.0.51` to the VPC and use it to execute `port forwarding`, ACS will automatically add `192.168.0.51` as `source NAT` too.

Co-authored-by: GutoVeronezi <daniel@scclouds.com.br>
2021-10-03 14:55:28 +05:30
Wei Zhou
09fce75047
OVS/GRE: bug fixes (#5446)
* OVS/GRE: fix failed to fetch cloudbr* info on host

* OVS/GRE: move ovstunnel.py to python3

* OVS: add test_privategw_acl_ovs_gre.py for automatic testing

(1) VR <-> VM should work
(2) Private GW should work

* OVS: fix an issue on xenserver

ovs bridges are deleted by xenserver/ovs automatically
```
[root@ref-trl-1797-x-M7-wei-zhou-xs2 ~]# grep -r xapi7 /var/log/ |grep del-br
/var/log/xensource.log:Sep 15 07:13:44 ref-trl-1797-x-M7-wei-zhou-xs2 xcp-networkd: [ info|localhost|611 |org.xen.xapi.xenops.classic events D:4a3d931cd89f|network_utils] /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
/var/log/daemon.log:Sep 15 07:13:45 ref-trl-1797-x-M7-wei-zhou-xs2 ovs-vsctl: ovs|00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --timeout=20 -- --if-exists del-br xapi7
```

which results that xe network exists but bridge does not exist, and operation stuck for 20 minutes at
```
2021-09-15 16:06:56    DEBUG [root] #### VMOPS enter  create_tunnel ####
2021-09-15 16:06:56    DEBUG [root] Creating tunnel from host 2 to host 1 with GRE key 2116
2021-09-15 16:06:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', '--timeout=0', 'wait-until', 'bridge', 'xapi7', '--', 'get', 'bridge', 'xapi7', 'name']
2021-09-15 16:26:56    DEBUG [root] bridge xapi7 for creating tunnel - VERIFIED
2021-09-15 16:26:56    DEBUG [root] Executing:['/usr/bin/ovs-vsctl', 'add-port', 'xapi7', 't2116-2-1', '--', 'set', 'interface', 't2116-2-1', 'type=gre', 'options:key=2116', 'options:remote_ip=10.0.34.230']
```

* OVS/GRE: component test test_vpc_distributed_routing_offering.py passes on xenserver

* OVS: support Shared network with GRE protocol

* update #5446: use lang3.StringUtils

* OVS/GRE: VPC with distributed routing on kvm (still not working)
2021-10-03 14:47:52 +05:30
Abhishek Kumar
4a42e7ef9e
vmware, ui: update portgroup on network update (#5470)
Enhanced update network form in the UI.
On network offering change for an isolated network,

- VMware portgroup should be updated accordingly.
- VMs on the network should be placed on the correct VMware portgroup based on the network rate, https://docs.cloudstack.apache.org/en/latest/adminguide/service_offerings.html#network-throttling.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-10-03 14:41:42 +05:30
SadiJr
965a47fdfd
Create UpdateBackupOffering API (#5511)
* Create UpdateBackupOffering API

* Address reviews

* Address reviews

* Address reviews

Co-authored-by: SadiJr <sadi@scclouds.com.br>
2021-10-01 08:30:25 -03:00
Harikrishna
df0c0045b5
Fix export snapshot and template to secondary storage to export only required disk (#5510)
* Fix export snapshot and export template to secondary storage in VMware to export only one required disk

* Move clone operation into virtual machine mo

* Code refactored for readability

* Added disk key check even for successful clone operation

* Delete dettached disks from cloned VM and added few logs
2021-09-30 12:44:02 -03:00
Pearl Dsilva
32051fb5fa
Fix permission issue during Diagnostic service garbage collection (#5504) 2021-09-30 08:18:23 -03:00
Hoang Nguyen
75922bb727
remove duplicate endipv6 item in shared network (#5537) 2021-09-30 10:19:37 +05:30
Hoang Nguyen
d08e2bcf3a
UI - Fixes modal width by device screen (#5526)
* fix modal width by screen width

* fix test unit fail

* fix overlaps dialogs
2021-09-30 10:19:10 +05:30
Wei Zhou
ea643a64e5
server: cannot deploy/start vm if service offering has multiple tags (#5521) 2021-09-29 14:59:48 +05:30
Rakesh
dcc02e0fbb
Enable account settings to be visible under domain settings (#4215)
* Enable account settings to be visible under domain settings

All the account settings can't be configured under domain
level settings right now.
By default, if account setting is not configured then
its value will be taken from global setting.
Add a global setting "enable.account.settings.for.domain"
so that if its enabled then all the account level settings
will be visible under domain levelsettings also.
If account level setting is configured then that value will
be considered else it will take domain scope value. If
domain scope value is not configured then it will pick
it up from global setting.

If domain level setting is not configured then by default
the value will be taken from global setting
Add another global setting "enable.domain.settings.for.child.domain"
so that when its true, if a value for domain setting is not
configured then its parent domain value is considered until
it reaches ROOT domain. If no value is configured till ROOT
domain then global setting value will be taken.

Also display all the settings configured under the domain level
in list domains api response

* rename variables

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-09-29 10:29:20 +02:00
DK101010
9ef7355399
ui: show datastore cluster option in vmware zone deployment (#5522)
Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
2021-09-29 09:52:38 +05:30
Pearl Dsilva
37d22646b2
simulator: Add support to scale a VM (#5515)
* simulator: Add support to scale VM

* make list a constant
2021-09-29 09:47:58 +05:30
slavkap
75c15fd0d6
Allow storage plugins to get storage/volume stats without sending commands to hosts (#4826)
* Allow storage plugins to use storage/volume stats

Allow the rest of the storage plugins, not only ScaleIO to implement and
use the storage/volume statistics directly instead of sending cmd to
hosts

* Get the store driver from pool instead of passing it as argument

required change from code review to pick the store driver from pool not
to pass it as argument
2021-09-28 16:17:34 -03:00
Pearl Dsilva
e573d0ddcb
Allow users (User accout Role) to delete / archive events (#5520) 2021-09-28 09:45:04 -03:00
Rohit Yadav
b955b38ccb Merge remote-tracking branch 'origin/4.15' into main 2021-09-27 16:17:49 +05:30
Abhishek Kumar
25d522fb3d
server: add vm boot details for start vm api (#5469)
Add vm boot details for start vm api

Fixes #5466

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-09-27 16:16:27 +05:30
Rakesh
a9c42fd0cd
Provide option to force delete the project (#4617)
* Provide a cleanup flag so that the project will
be deleted only when there are no resources left
in the project. If users click on delete project
by mistake then everything is deleted.

* fix travis failures

Co-authored-by: Rakesh Venkatesh <rakeshv@apache.org>
2021-09-26 01:02:29 -03:00
nicolas
9de5ef9f2b
Merge branch '4.15' into main 2021-09-24 11:32:38 -03:00
Wei Zhou
6aa75cf671
test_vpc_redundant.py: reduce sleep time from 1 hour to 21 mins (#5503) 2021-09-24 11:31:47 -03:00
SadiJr
c3a1c7862f
backup: Improve Veeam Plugin logs (#5455)
* Improve Veeam Plugin logs

* Address reviews

* Address reviews

Co-authored-by: SadiJr <sadi@scclouds.com.br>
2021-09-24 17:25:27 +05:30
Wei Zhou
06f602c7df
tools/docker: Upgrade to ubuntu 20.04 , MySQL 8 and python3 (#5507)
* tools/docker: Upgrade to ubuntu 20.04 , MySQL 8 and python3

* Update tools/docker/README.md

* docker simulator: run 'npm rebuild node-sass' before 'npm install'
2021-09-24 17:17:27 +05:30
Pearl Dsilva
d1f375f35a
marvin: Refactor - cleanup of resource after test run (#5505) 2021-09-24 13:42:15 +05:30
Pearl Dsilva
74bb80687d
resource limit: Fix resource limit check on VM start (#5428)
* resource limit: Fix resource limit check on VM start

* add check to validate if cpu/memory are within limits for custom offering + exception handling

* unit tests

Co-authored-by: utchoang <hoangnm@unitech.vn>
2021-09-24 09:51:16 +05:30