34057 Commits

Author SHA1 Message Date
Pearl Dsilva
4c16024a78
ui: fix login when username is email (#4910)
Fixes #4908

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-12 23:57:59 +05:30
Nicolas Vazquez
9cf1e0e869
vmware: Fix VMware OVF properties copy from template (#4738)
* Fix VMware OVF properties copy from template

* Fix vapp marvin test

* Remove unused code

* Fix check for deploy as is details

* Access class fields
2021-04-12 18:04:58 +05:30
Pearl Dsilva
a64ad9d9b7
server: Prevent vm snapshots being indefinitely stuck in Expunging state on deletion failure (#4898)
Fixes #4201

This PR addresses the issue of a vm snapshot being indefinitely stuck is Expunging state in case deletion fails. 

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-12 08:09:37 +05:30
Rohit Yadav
e824fdba20 Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
2021-04-10 13:41:50 +05:30
Abhishek Kumar
6b1c94ea3e
server: fix root disk size on vm reset (#4638)
If VM details contain rootdisksize, volume entry in DB should reflect correct size when VM reset is performed.

Fixes #3957

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-10 13:31:51 +05:30
Nicolas Vazquez
e47dc9c25e
ldap: Fix orphan entry on ldap trust map after account removal (#4899)
Fixes: #4673

Fix orphan entry on ldap trust map after account removal
2021-04-10 13:26:50 +05:30
Abhishek Kumar
fdefee75ff
vmware: fix inter-cluster stopped vm and volume migration (#4895)
Fixes #4838

For inter-cluster migration without shared storage, VMware needs a host to be specified. Fix is to specify an appropriate host in the target cluster during a stopped VM migration. Also, find target datastore using the host in the target cluster.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-10 13:22:28 +05:30
Spaceman1984
4bab06a74b
systemvm: Restricting http access on VR to internal network (#4847)
There is a potential security issue with having http access to the VR from anywhere.
This PR restricts http access to the VR to the internal network only.
2021-04-10 13:19:31 +05:30
Spaceman1984
99a9063cf4
server: Added recursive fetch of child domains for listUsageRecords API call (#4717)
When calling the listUasageRecords API records per domain are fetched recursively. This is not the case if you specify a domain id.

This PR adds a new parameter to enable fetching records recursively (isRecursive) when passing the domain id.
2021-04-10 13:15:29 +05:30
dahn
03c05bc38a
tests: skip livemigration for centos (#4801)
This PR skips live migration on centos as the default installations for qemu/libvirt don't support it.

Fixes #4757

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-04-10 13:11:59 +05:30
Rohit Yadav
ca8920dd36 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-09 13:17:39 +05:30
Abhishek Kumar
d8c6e00498
hypervisor: XCP-ng 8.2 support (#4672)
Adds new/missing guest os mappings for XCP-ng/Xenserver 8.1
Copy guest OS mappings from XCP-ng/Xenserver 8.1 for XCP-ng/Xenserver 8.2
Adds Ubuntu 20.04 guest os mapping for XCP-ng/Xenserver 8.2

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 13:12:06 +05:30
Abhishek Kumar
cd60b8d97d
host-allocator: check capacity for suitable hosts (#4884)
Fixes #4517

Adds capacity checks for RandomAllocator (host allocator)

Factors out host cpu capability and capacity check wrt serviceoffering code into CapacityManager.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 12:35:58 +05:30
Pearl Dsilva
cd3851946b
tests: Fix k8s test failures on VMware (#4896)
This PR fixes the k8s test failures noticed on vmware.

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 12:34:10 +05:30
Rakesh
262285639e
kvm: Disable shrinking QCOW2 volumes (#4679)
Since QCOW2 volumes cannot be resized to lower value in any case,
just disable it in backend.
2021-04-07 11:25:52 +05:30
Miklos Barabas
7208fac18b
mvn: using forked version of trilead-ssh2 (from org.jenkins-ci) (#4099)
* using forked version of trilead-ssh2 (from org.jenkins-ci)
- upgrade to support newer algorithms

* Update latest jar release

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
2021-04-07 11:24:07 +05:30
Hoang Nguyen
be781804f1
ui: fix physical network setup in zone wizard (#4892) 2021-04-06 17:35:58 +05:30
Nicolas Vazquez
35b20b2367
vmware: Fix worker VM hardware version format (#4851)
This PR fixes a small bug when explicitly setting VM hardware versions lower than version 10.

Vmware expects the hardware version in format: vmx-DD where DD is a two-digit representation of the virtual hardware version. For hardware version lower than 10, CloudStack was not using to digits for the hardware version number, which ended up on an error while creating worker VMs. (vmx-8 for example instead of vmx-08)
2021-04-06 16:28:52 +05:30
DK101010
9da8124fd9
maven: version 2.11 is incompatible with java 11 (#4802)
with 2.11 wire mock tests hanging and mvn can't execute in thread mode

Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
2021-04-06 15:04:07 +05:30
slavkap
52c36cadf0
kvm: Fix deploy VM from ISOs with UEFI (#4773)
This PR fixes #4244
deploying of VMs from ISOs and from templates with UEFI boot type
deploying of VMs from ISOs and from templates with UEFI boot type with
volumes in RAW format
2021-04-06 15:00:21 +05:30
Rohit Yadav
7270ca7e25 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-06 12:51:26 +05:30
Abhishek Kumar
1a335e880a
server: filter null details during volume to template creation (#4794)
Fixes #4628

mysql> describe user_vm_details;
+---------+-----------------+------+-----+---------+----------------+
| Field   | Type            | Null | Key | Default | Extra          |
+---------+-----------------+------+-----+---------+----------------+
| id      | bigint unsigned | NO   | PRI | NULL    | auto_increment |
| vm_id   | bigint unsigned | NO   | MUL | NULL    |                |
| name    | varchar(255)    | NO   |     | NULL    |                |
| value   | varchar(5120)   | YES  |     | NULL    |                |
| display | tinyint(1)      | NO   |     | 1       |                |
+---------+-----------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)

mysql> describe vm_template_details;
+-------------+-----------------+------+-----+---------+----------------+
| Field       | Type            | Null | Key | Default | Extra          |
+-------------+-----------------+------+-----+---------+----------------+
| id          | bigint unsigned | NO   | PRI | NULL    | auto_increment |
| template_id | bigint unsigned | NO   | MUL | NULL    |                |
| name        | varchar(255)    | NO   |     | NULL    |                |
| value       | varchar(1024)   | NO   |     | NULL    |                |
| display     | tinyint(1)      | NO   |     | 1       |                |
+-------------+-----------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)

While cloud.user_vm_details allows null values to be added for a detail, cloud.vm_template_details doesn't allow null values.
This change filters vm details with null values while creating template from a volume.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-06 12:48:56 +05:30
Rakesh
c2d51cb20e
ui: Fix bug in creating shared network (#4666)
* Fix bug in creating shared network

Sometimes creating shared network fails because the physical network
data is null and redereferencing it causes NPE.
Wait till the physical network data is available before accessing it

* Update CreateSharedNetworkForm.vue

Co-authored-by: Rohit Yadav <rohit@apache.org>
2021-04-06 12:47:58 +05:30
Hoang Nguyen
a92b294c5d
ui: Save and auto-expand list domain when reloading (#4769)
* save and auto-expand list domain when reloading

* fix for simplify the code

* clear variables, prop of common component

* fix lint errors found
2021-04-06 11:58:03 +05:30
Gabriel Beims Bräscher
cb91a769d3
Fix npe when migrating vm with volume (#4698) (#4775)
Cherry-pick commit 59fba4916b and fix conflict.

Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2021-04-06 11:54:29 +05:30
Pearl Dsilva
467a1e7ecf
travis: fix component test failure - persistent networks (#4894)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-05 15:24:25 +05:30
Rohit Yadav
6c3f4b43aa Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-05 14:58:41 +05:30
Abhishek Kumar
6048afb464
xenserver: retrieve correct name-label for presetup store (#4816)
Fixes #4729

As reported in the issue ACP 4.7 used a normal UUID in db for a presetup primary store on Xenserver.
Later the value has been changed to store's path with '/' removed.
Current changes try to retrieve SR's name-lable from store's path if UUID doesn't match path field for a pre-setup store.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-05 14:28:27 +05:30
Hoang Nguyen
4dcd9a2329
ui: Moves fetchdata() to the created() (#4811)
This PR to move fetchData() to created() insteadof mount()
2021-04-05 14:28:00 +05:30
Rohit Yadav
43257f8300 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-05 12:59:37 +05:30
Rakesh
c75c6baa3f
ui: Display public ip addresses for shared network (#4676)
If a vm belongs to shared network then display the list
of ip addresses available which can be used to assign for
secondary IP addresses.

Also display "Public IP addresses" tab for shared networks
2021-04-05 12:37:03 +05:30
aleskxyz
ca4669c4d4
systemvm: Add localized "data-server" records in /etc/hosts for VPC routers (#4873) 2021-04-05 12:34:10 +05:30
Rakesh
47b72883e2
server: Disable VR health check for VPC without tiers (#4888)
If the VPC has redundant VR's but doesnt have any tiers or there are no running VM's in
a VPC then disable the gateway check for VR
2021-04-05 12:33:45 +05:30
davidjumani
154734ceae
ui: Remove .env.local (#4893) 2021-04-05 12:33:30 +05:30
Wei Zhou
09428380f7
kvm: remove unnecessary new String (#4870)
Thanks @rubieHess to point it out.
see #4800 (comment)
2021-04-04 13:08:29 +05:30
Rohit Yadav
2cdde8774b Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-01 14:33:08 +05:30
Hoang Nguyen
575e18c00a
ui: Restored the Basic Networking (#4882)
* zone wizard: restored the basic networking

* set enabled Basic Networking
2021-04-01 14:31:57 +05:30
Wei Zhou
d4ba00434c
VR: fix rsyslog compresses log files but not release disk space in VR (#4869)
We had critical issue with VR recently. The VRs of shared network or vpc stops working after some days.
After investigation, I found that the disk space is full

```
root@r-10-VM:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/vda5        2086316 2069932         0 100% /
```

logrotate/ryslog has compresses the log files, but space is not released. see `lsof |grep deleted`

```
root@r-10-VM:~# lsof |grep deleted
rsyslogd    960                      root   12w      REG              254,5 493060096        137 /var/log/daemon.log.1 (deleted)
rsyslogd    960                      root   13w      REG              254,5  17715200        110 /var/log/messages.1 (deleted)
rsyslogd    960                      root   16w      REG              254,5 545968128        342 /var/log/auth.log.1 (deleted)
rsyslogd    960                      root   18w      REG              254,5  38313984        341 /var/log/cron.log.1 (deleted)
rsyslogd    960  962 in:imuxso       root   12w      REG              254,5 493060096        137 /var/log/daemon.log.1 (deleted)
rsyslogd    960  962 in:imuxso       root   13w      REG              254,5  17715200        110 /var/log/messages.1 (deleted)
rsyslogd    960  962 in:imuxso       root   16w      REG              254,5 545968128        342 /var/log/auth.log.1 (deleted)
rsyslogd    960  962 in:imuxso       root   18w      REG              254,5  38313984        341 /var/log/cron.log.1 (deleted)
rsyslogd    960  963 in:imklog       root   12w      REG              254,5 493060096        137 /var/log/daemon.log.1 (deleted)
rsyslogd    960  963 in:imklog       root   13w      REG              254,5  17715200        110 /var/log/messages.1 (deleted)
rsyslogd    960  963 in:imklog       root   16w      REG              254,5 545968128        342 /var/log/auth.log.1 (deleted)
rsyslogd    960  963 in:imklog       root   18w      REG              254,5  38313984        341 /var/log/cron.log.1 (deleted)
rsyslogd    960  964 in:imfile       root   12w      REG              254,5 493060096        137 /var/log/daemon.log.1 (deleted)
rsyslogd    960  964 in:imfile       root   13w      REG              254,5  17715200        110 /var/log/messages.1 (deleted)
rsyslogd    960  964 in:imfile       root   16w      REG              254,5 545968128        342 /var/log/auth.log.1 (deleted)
rsyslogd    960  964 in:imfile       root   18w      REG              254,5  38313984        341 /var/log/cron.log.1 (deleted)
rsyslogd    960  965 in:imudp        root   12w      REG              254,5 493060096        137 /var/log/daemon.log.1 (deleted)
rsyslogd    960  965 in:imudp        root   13w      REG              254,5  17715200        110 /var/log/messages.1 (deleted)
rsyslogd    960  965 in:imudp        root   16w      REG              254,5 545968128        342 /var/log/auth.log.1 (deleted)
rsyslogd    960  965 in:imudp        root   18w      REG              254,5  38313984        341 /var/log/cron.log.1 (deleted)
rsyslogd    960  966 rs:main         root   12w      REG              254,5 493060096        137 /var/log/daemon.log.1 (deleted)
rsyslogd    960  966 rs:main         root   13w      REG              254,5  17715200        110 /var/log/messages.1 (deleted)
rsyslogd    960  966 rs:main         root   16w      REG              254,5 545968128        342 /var/log/auth.log.1 (deleted)
rsyslogd    960  966 rs:main         root   18w      REG              254,5  38313984        341 /var/log/cron.log.1 (deleted)
```

workaround: restarting rsyslog to release the space.
```
systemctl restart rsyslog
```

The root cause is, the following command does not work in 4.15 template
```
root@r-10-VM:~# invoke-rc.d rsyslog rotate
[FAIL] Closing open files: rsyslogd failed!
```

Fix: use `/usr/lib/rsyslog/rsyslog-rotate` instead
```
root@r-10-VM:~# /usr/lib/rsyslog/rsyslog-rotate
root@r-10-VM:~# cat /usr/lib/rsyslog/rsyslog-rotate

if [ -d /run/systemd/system ]; then
    systemctl kill -s HUP rsyslog.service
else
    invoke-rc.d rsyslog rotate > /dev/null
fi

```
2021-04-01 14:30:58 +05:30
Abhishek Kumar
251368361e
ui: allow docHelp override using config.json (#4745)
config.json can have a property named 'docHelpMappings' which can be used to override docHelp suffixes.
In config.json admin can add mappings as follows:

"docHelpMappings": {
  "virtual_machine.html": "some.html",
  "some_string": "override_string"
}

UI will use these mappings and will make appropriate replacements in the documentation links for different sections and forms.

Addresses #4731

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-01 13:10:00 +05:30
Wei Zhou
dc5b9ec7c8
systemvm: remove logrotate config for wtmp and btmp (#4872)
logrotate in systemvms run every day. it exits with failure.
```
root@r-100-VM:~# systemctl status logrotate
● logrotate.service - Rotate log files
   Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2021-03-23 00:00:01 UTC; 2 days ago
     Docs: man:logrotate(8)
           man:logrotate.conf(5)
  Process: 25001 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
 Main PID: 25001 (code=exited, status=1/FAILURE)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
```

it is because the logrotate for wtmp and btmp already exist in 4.15 systemvm template.
```
root@r-100-VM:~# cat /etc/logrotate.d/wtmp
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    minsize 1M
    rotate 1
}
root@r-100-VM:~# cat /etc/logrotate.d/btmp
/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}
```

remove them from /etc/logrotate.conf fixes the issue.
```
root@r-100-VM:~# systemctl status logrotate
● logrotate.service - Rotate log files
   Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset: enabled)
   Active: inactive (dead) since Thu 2021-03-25 00:00:01 UTC; 9h ago
     Docs: man:logrotate(8)
           man:logrotate.conf(5)
  Process: 28211 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=0/SUCCESS)
 Main PID: 28211 (code=exited, status=0/SUCCESS)

Mar 25 00:00:01 r-100-VM systemd[1]: Starting Rotate log files...
Mar 25 00:00:01 r-100-VM systemd[1]: logrotate.service: Succeeded.
Mar 25 00:00:01 r-100-VM systemd[1]: Started Rotate log files.
```
2021-04-01 12:51:17 +05:30
Wei Zhou
63c91c1458
server: Fix network statistics for vpc (#3944)
This contains 3 main changes
(1) add NETWORK_STATS_ethX for all nics with public ips in VPC VRs (current: NETWORK_STATS_eth1)
(2) DO NOT create records in user_statistics for each VPC tier (only one record per public nic per VPC VR)
(3) send NetworkUsageCommand before unplugging a NIC with public IPs from VPC VR
2021-04-01 12:43:06 +05:30
Rakesh
76ba5c62d9
server: Fix displaying public IP address of shared networks (#4675)
Public IP addresses dedicated to one domain should not be accessed
by other domains. Also, root admin should be able to display all
public ip addresses in system.

Currently following issues exist

1. Public IP address assigned to one domain can be accessed by
other sibling domains

If use.system.public.ip is false then child domains should not
see public ip of ROOT domain

Before fix
```
(test1) mgt01 > list publicipaddresses listall=true fordisplay=true allocatedonly=false forvirtualnetwork=true filter=ipaddress,
{
  "count": 59,
  "publicipaddress": [
```

After fix

```
(test) mgt01 > list publicipaddresses listall=true fordisplay=true allocatedonly=false forvirtualnetwork=true filter=ipaddress,
{
  "count": 10,
```
2021-04-01 12:39:01 +05:30
Abhishek Kumar
95cd937d7b
server: fix hahost value in listHosts (#4789)
* api/server: fix hahost value in listHosts

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update server/src/main/java/com/cloud/api/query/dao/HostJoinDaoImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Update server/src/main/java/com/cloud/api/query/dao/HostJoinDaoImpl.java

Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-04-01 12:33:48 +05:30
Wei Zhou
02abb87f12
server: allow copy cross-zone templates to other zone (#4804) 2021-04-01 12:32:52 +05:30
Pearl Dsilva
69825f6cb6
ui: Display root disk size in Compute offering details (#4862)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-01 12:31:07 +05:30
davidjumani
fd4bc612d7
ui: assignVM: Set isrecursive to false when fetching accounts (#4867)
* Set isrecursive to false when fetching accounts

* Adding exception handling
2021-04-01 12:29:57 +05:30
Hoang Nguyen
5b986be5ba
ui: fix the style of zone wizard on mobile view (#4764)
This PR for fixes create zone form in mobile view
2021-03-30 14:09:56 +05:30
davidjumani
03ad702c1b
server: Set free memory to zero if greater than total memory (#4571)
Fixes https://github.com/apache/cloudstack/issues/4566

Sets `memoryintfreekbs` to zero if it is greater than `memorykbs`. Caused by KVM returning the RSS memory of the process running the VM rather than the free memory inside the VM.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-03-30 14:05:46 +05:30
Gabriel Beims Bräscher
d6a74272a4
plugins: Add 'break' at RedifshClient request re-try loop (fixed issue from 4846) (#4864)
* Break loop if no exception on http request

* Add new tests ensuring the correct execution flow of the RedfishClient retry

* Log retry as "retry attempt %d/%d"

* Fix string.format parameters order at RedfishClient.retryHttpRequest
2021-03-29 16:25:47 +05:30
sureshanaparti
89111110cb
server: Remove the rule(s) validation with api names while importing role, to be in sync with the create role permission behavior (#4840)
This PR removes the rule(s) validation with api names while importing a role. This will be in sync with the current create role permission behavior.
2021-03-29 16:23:43 +05:30