33907 Commits

Author SHA1 Message Date
Rohit Yadav
3a82392941 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-03-06 15:59:50 +05:30
Rohit Yadav
41c2e944a6 ui: Remove make redundant option for isolated networks
Fixes #4052

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-06 15:59:02 +05:30
Hoang Nguyen
03bfe9a2a3
ui: FIX error in "Port forward" and "Load Balancing" (#4746)
* fix error show in fetching port forwarding tab

* clear all list vm when tier empty
2021-03-06 15:10:07 +05:30
Rohit Yadav
f511babc16
api: remove account from listProjects API response (#4743)
The `account` is no longer set in the listProjects API response that is
still mentioned in the API docs. API consumers should now use the
`owner` key from the listProjects API response which returns a list of
owners (accounts and users).

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-06 15:09:06 +05:30
Wei Zhou
df4103f0d1
novnc: Add source IP check (#4736)
* novnc: Add client IP check for novnc console in cloudstack 4.16

* novnc ip check : Fix restart CPVM or mgt server does not update novnc param

* novnc ip check: move to method
2021-03-06 15:08:34 +05:30
Wei Zhou
58a3f90a51
server: fix failed to remove template/iso if upload from local fails (#4626)
* server: fix failed to remove template/iso if upload from local fails

When upload template/iso/volume from local fails, the install_path will not be a full path of file so removing it will fail.

```
mysql> select install_path from template_store_ref;
+--------------------------------------------------------------------+
| install_path                                                       |
+--------------------------------------------------------------------+
| template/tmpl/1/3/805f4763-248e-40ec-b79a-b868cc480d0a.qcow2       |
| template/tmpl/1/4/c7e32c9e-5e72-3726-85cf-aa5ccd84118d.qcow2       |
| template/tmpl/2/201/bc4f4f08-138a-31b8-af1a-d4450eff7982.qcow2     |
| template/tmpl/2/202                                                |
| template/tmpl/2/203/203-2-d47f8cde-a2a8-31e7-a826-2628ad98a6c8.iso |
| template/tmpl/2/204                                                |
| template/tmpl/5/205                                                |
| template/tmpl/2/206                                                |
| template/tmpl/2/207                                                |
| template/tmpl/2/208                                                |
| template/tmpl/2/209                                                |
| template/tmpl/2/210                                                |
+--------------------------------------------------------------------+
12 rows in set (0.00 sec)

mysql> select install_path from volume_store_ref;
+---------------------------------------------------------+
| install_path                                            |
+---------------------------------------------------------+
| volumes/2/22                                            |
| volumes/2/19/f93face9-6521-4184-b89a-cb07f86bbae8.qcow2 |
| volumes/2/23                                            |
| volumes/2/24                                            |
+---------------------------------------------------------+
4 rows in set (0.00 sec)
```

* server: disallow removing template/iso in NotUpload and UploadInProgress state
2021-03-06 14:53:40 +05:30
Wei Zhou
cdc3b08759
novnc: Accept new novnc client and disconnect old session (#4531)
* novnc: Reject new novnc client if novnc viewer object is still alive

* #4531 novnc: Accept new novnc client and disconnect old session
2021-03-06 14:43:21 +05:30
Rohit Yadav
0a401eb92d Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-05 10:48:26 +05:30
Rohit Yadav
370d3f2e8e
build: deprecate and remove md5 from releases (#4751)
This removes MD5 checksum created as part of release work due to ASF
infra policy to deprecate MD5:
https://infra.apache.org/release-distribution#sigs-and-sums

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-05 10:46:52 +05:30
Rohit Yadav
97c1b7278f Merge remote-tracking branch 'origin/4.14' into 4.15 2021-03-04 15:02:24 +05:30
Rohit Yadav
709ebf6062 Merge remote-tracking branch 'origin/4.13' into 4.14 2021-03-04 15:00:51 +05:30
Abhishek Kumar
f893c5c0e3
cks: fix token TTL, set it to never expire (#4747)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-03-04 15:00:41 +05:30
Gabriel Beims Bräscher
6e7516c48a
kvm: Fix get_bridge_physdev where it returns "device:" instead of just "device" (#4740)
When running get_bridge_physdev(brname) from security_group.py it is returned the bridge device as brname: instead of the expected brname.

We experienced this issue on CloudStack 4.13.1.0 with Security Groups enabled for Advanced Networking. Additionally, KVM nodes are running on Ubuntu 18.04.

PR #4303 (merged in 4.15) added support for Ubuntu 20.04 which turned out to fix get_bridge_physdev(brname); however, we faced the very same issue with the previous CloudStack and Ubuntu versions.

Even though we might not get a 4.13.2, and CloudStack 4.14.1.0 has just been released, this PR proposes merging the fix into branch 4.13 and then get it forwarded into 4.14. Thus, allowing users to have this fix referenced and also opening the possibility of addressing this in case of a potential 4.14.2.0.
2021-03-04 14:49:24 +05:30
Abhishek Kumar
f8ad3adaff
cks: use HttpsURLConnection for checking api server (#4639)
* cks: use HttpsURLConnection for checking api server

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ignore ssl certificate validation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-03-02 12:39:51 +05:30
Rohit Yadav
233785446d Merge remote-tracking branch 'origin/4.14' into 4.15 2021-03-02 12:32:54 +05:30
Rohit Yadav
fa067e02a7 Updating pom.xml version numbers for release 4.14.2.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-03-02 12:32:27 +05:30
Rohit Yadav
2e36e905d9 Tagging release 4.14.1.0 on branch 4.14.
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEXtHhEi3F6KSkURLCSEJIIQ7j2IQFAmAhGGIACgkQSEJIIQ7j
 2IQN7RAAtyIlEUYp0X0KABNJKaAFOCy/GkBxETTmkjYgMNMX1GOj16F8X3ZXAJ/D
 q0ZBBsvLu62WmTzG6hWTMllFz0w5HRl4X4lObDnBci1u35oa+f6KUh0tyWsAYgWM
 qsIAtePcudJA60MOPwffTZjkxwTUNl2fjwCOxCBOLLTiIa6F0DA4zjNQ4L1/frSS
 KefQt9UDM5GsGrZIGlZIwFD7aqL6PIilxh++3PfCseAUPY6FZiHDYNwN1gPWkziM
 vjQbJyT5cmZdcCs7vlm3fcKYIvXL5S3p64C80L5U/NMdcYQ1mHabwZGQTsTkhDt3
 a7aeXBKIm/V8Aeq3jIRCtNb2m0qiIjmxlLrYRQaUfyIhCU9jRO4gWMzneLVoeUtY
 UACzTpFJvAgSR8viYwtlKgBCbiRVvcJgaPR58pIroiUvc2TIaERw121u4wxDVT5Z
 JqX4zpJjuhXcClHPis6AV8AISjftAbSl24D+IoL0rUn2w3/oH1dMrfkIPPF0wbDw
 /0pwvGHCwXZ5sBqos+OSy1ChPt8KiKHfWDRrHQm5TLDk738uuUV4hf6v4OMKa3Hy
 Tyu/9ZDZbLGBdYit6R7DHWPJ0/y5J+zC7Opt9ufqP3dH6ZIV+j0yvqs9HQJMvZ7F
 LuPcjjWVq2tXnSeLBzjkBTvCrpejb6MJyPO8mcLKw1+BRM93XZY=
 =jGff
 -----END PGP SIGNATURE-----

Merge tag '4.14.1.0' into 4.14

Tagging release 4.14.1.0 on branch 4.14.
2021-03-02 12:30:08 +05:30
Rafael Weingärtner
569720fcc2 Merge forward PR#4668 of Branch 4.14 on branch 4.15 2021-02-25 21:15:21 -03:00
Rafael Weingärtner
7be87f1cf8 Merge forward PR#4668 of Branch 4.13 on branch 4.14 2021-02-25 21:01:18 -03:00
Daniel Augusto Veronezi Salvador
e1f3179446
Adjust tests to fix a problem with the container builders (https://github.com/khos2ow/cloudstack-deb-builder) (#4668)
* Changes to allow builders containers to build ACS

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-02-25 20:28:11 -03:00
Abhishek Kumar
88337bdea4
server: fix finding pools for volume migration (#4693)
While finding pools for volume migration list following compatible storages:
- all zone-wide storages of the same hypervisor.
- when the volume is attached to a VM, then all storages from the same cluster as that of VM.
- for detached volume, all storages that belong to clusters of the same hypervisor. 

Fixes #4692 
Fixes #4400
2021-02-25 22:13:50 +05:30
Greg Goodrich
af0f6422ec
kvm: Suspending the VM prior to deleting snapshots to avoid corruption, th… (#4032)
These changes are related to PR #3194, but include suspending/resuming the VM when doing a VM snapshot as well, when deleting a VM snapshot, as it is performing the same operations via Libvirt. Also, there was an issue with the UI/localization changes in the prior PR, as that PR was altering the Volume snapshot behavior, but was altering the VM snapshot wording. Both have been altered in this PR.

Issuing this in response to the work happening in PR #4029.
2021-02-25 22:11:00 +05:30
Rakesh
787491871a
server: Look for active templates for VR deployment (#4047)
If the template from which VR is created got deleted, the state
is set to inactive and removed to null.
Since the template is already deleted, the VR can't be created
using this template again.

If someone restarts network with cleanup then it will try to
deploy the vr from the old non existing template again.
So search only for active template which are not yet deleted.
2021-02-25 22:05:31 +05:30
Abhishek Kumar
fef0e66764
ui: fix add Vmware cluster (#4663)
When adding a cluster from existing host-datacenter UI shoudl list and autofill host, DC name while making addCluster API call.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-25 22:04:00 +05:30
Hoang Nguyen
a234501172
ui: Add guest IP ranges (#4716)
Fixes #4697
2021-02-25 22:01:40 +05:30
Hoang Nguyen
1545bf8dac
ui: add missing antd component use (#4728)
Fixes #4724
2021-02-25 21:58:42 +05:30
Abhishek Kumar
d0e71adbbf
packaging: update Requirements in README (#4725)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-24 17:46:01 +05:30
Pearl Dsilva
ccbcb08fbf
server: Increase leniency to list templates on secondary stores that have been marked deleted by updating the db (#4713)
This PR aims at restoring the previous level of leniency in listing templates on stores that have been marked as deleted / removed by updating the DB.
While Cloudstack doesn't allow deleting stores that have resources on them, it may so happen that users may mimic a deletion of a store by merely updating the DB. Under such a case, listing of templates is hampered due to an NPE that is caused. (as seen in #4606)

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-24 16:30:32 +05:30
Pearl Dsilva
a896760a40
storage: Secondary storage allow store deletion after successful data migration (#4615)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-24 16:27:39 +05:30
Wei Zhou
5a3ae159ca
upgrade: check systemvm template before db changes (#4582)
* Upgrade: check systemvm template before db changes

* Upgrade: move some codes to a separated method

* #4582 add txn.commit()
2021-02-24 16:26:31 +05:30
Hoang Nguyen
ba43825a9b
ui: Fix npm unit test failures in 4.15 (#4718)
* Edit the test to match the latest code

* fix travis build failures

* correct wrong comparison conditions

* fix travis build
2021-02-24 14:39:55 +05:30
Abhishek Kumar
e13363ed76
cks: fix CNI release url returning 404 (#4684)
CNI plugin release naming has changed, https://github.com/containernetworking/plugins/releases
Release are named for host OS from 0.8.0 onwards.
This change adds check for 404 response code and attempts retry

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-23 16:26:17 +05:30
dahn
6f361c3760
tools: format of checksum files convenient for automated checking (#4688)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-02-23 16:25:06 +05:30
Abhishek Kumar
06c16ac8f4
ui: fix systevmtype for create service offering form (#4683)
Fixes #4682

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-23 09:39:43 +00:00
Rohit Yadav
186ac69c49 Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-19 15:24:48 +05:30
Rohit Yadav
7a712e2c4c server: fix build failure
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-19 15:08:03 +05:30
Abhishek Kumar
f489439a3a
api: add zone, vm name params in listVmSnapshot response (#4604)
* api: add zone, vm name params in listVmSnaphots response

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-19 14:52:48 +05:30
Daniel Augusto Veronezi Salvador
3b5f99a7ca
kvm: Prevent KVM from performing volume migrations of running instances (#4562)
* Prevent KVM from performing volume migrations of running instances

KVM has a limitation to modify instances definitions while they are on running state. Therefore, it is not possible to change volumes backend location easily.

There is a problem in the `migrateVolume` API. This API command ignores that limitation and causes an inconsistence on the database. ACS processes the migrate command, copies the volume to the destination storage, modifies the database and finishes the process with success. However, the running backend is still using the "old volume file".

This PR intends to prevent KVM to perform volumes migrations while KVM instances are in the running state and inform the user of an alternative API command that enables such operation on running instances.

* Update VolumeApiServiceImpl.java

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
Co-authored-by: Rohit Yadav <rohit@apache.org>
2021-02-19 14:34:28 +05:30
Rakesh
06b5bd2966
ui: display account name only if its not null (#4667)
If account is null then "undefined" is displayed in ui
2021-02-19 14:07:38 +05:30
Wei Zhou
db1e0f49dc
kvm: set Backing Format of qcow2 images in vm start and migration (#4656)
In previous cloudstack versions, qcow2 image does not have a backing file format.
however, it is required in newer qemu versions, for example qemu 4.2 on ubuntu 20.04.

steps to reproduce the issue
(1) install cloudstack 4.14 or previous version, and ubuntu 19.04 or 18.04/16.04 LTS.
(2) create vms.
(3) upgrade to 4.15, upgrade os to ubuntu 20.04 , or install a new server with ubuntu 20.04.
(4) migrate vm from old ubuntu version to ubuntu 20.04, failed with exception below
```
2021-02-04 13:43:07,397 DEBUG [resource.wrapper.LibvirtMigrateCommandWrapper] (agentRequest-Handler-1:null) (logid:93da9385) ExecutionException : org.libvirt.LibvirtException: Requested operation is not valid: format of backing image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/66990fcc-fd98-4932-9649-989bf6583d59' of image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/a3dd1f0f-2557-4e07-951c-e4eb7b3f38b2' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting)
```
(5)stop vm, and start it on ubuntu 20.04 server. failed with exception below
```
2021-02-04 13:46:29,766 WARN  [resource.wrapper.LibvirtStartCommandWrapper] (agentRequest-Handler-5:null) (logid:b54745a7) LibvirtException
org.libvirt.LibvirtException: Requested operation is not valid: format of backing image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/66990fcc-fd98-4932-9649-989bf6583d59' of image '/mnt/03b6f487-9eaf-38bf-ad2d-d985423b832f/a3dd1f0f-2557-4e07-951c-e4eb7b3f38b2' was not specified in the image metadata (See https://libvirt.org/kbase/backing_chains.html for troubleshooting)
```

To make testing easier, step 1 and 2 can be replaced by
```
qemu-img create -f qcow2 -b <backing file> <qcow2 image>
```
so qcow2 image does not have a backing file format.
2021-02-19 14:06:47 +05:30
Rakesh
fd49efa9c1
server: Show network name in exception message (#4396)
* Show network name in exception message

* Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-02-19 14:04:39 +05:30
dahn
aab2447656
systemvm: loop optimisation in bash (#4451)
Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-02-18 18:18:16 +05:30
Pearl Dsilva
d6509f0a21
API discovery: Prevent overwrite of API parameters in case the API names are the same (#4609)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-18 14:37:20 +05:30
Rakesh
de7b131724
server: Cleanup domain details when domain is deleted (#4445)
When domain is deleted, all the settings configured under
the domain scope still exists in domain_details table.
All the entries for the domain should be deleted as well
2021-02-12 12:52:22 +05:30
Abhishek Kumar
0d04bd621f
ui: fix tags selection for add disk offering (#4665)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-02-12 12:50:32 +05:30
Rohit Yadav
66f0beda5f Updating pom.xml version numbers for release 4.14.1.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
4.14.1.0
2021-02-08 16:24:09 +05:30
Rohit Yadav
f5a44b3502 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-02-05 18:28:02 +05:30
Pearl Dsilva
c9c1d4a6e5
marvin: fix test failures when changing service offering of a VM (#4651)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-02-05 18:12:12 +05:30
Wei Zhou
d62d5c6cd2
VR: fix expunging vm will remove dhcp entries of another vm in VR (#4627)
Steps to reproduce the issue

(1) create two vm wei-001 and wei-002, start them

(2) check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They have entries for both of wei-001 and wei-002

(3) stop wei-002, and restart VR (or restart network with cleanup).
check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They have entries for wei-001 only (as wei-002 is stopped)

(4) expunge wei-002. when it is done,
check /etc/cloudstack/dhcpentry.json and /etc/dhcphosts.txt in VR
They do not have entries for wei-001.
VR health check fails at dhcp_check.py and dns_check.py
2021-02-05 18:10:53 +05:30
Rohit Yadav
6bde1384ff Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-02-05 16:01:01 +05:30