32596 Commits

Author SHA1 Message Date
Andrija Panic
cddf69c0f8
Usage-server update message improvement (#4278)
* Update cloud.spec

* Update cloud.spec

* Update cloudstack-usage.postinst
2020-08-23 16:04:38 +05:30
Rohit Yadav
bb73bedb55 Merge remote-tracking branch 'origin/4.14' 2020-08-21 15:39:00 +05:30
Rohit Yadav
05ae3f8d81 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-21 15:38:18 +05:30
Sid Kattoju
1da76d27f1
iscsi session cleanup now configurable, filters iscsi partitions (#4219)
Added property to agent.properties that enables or disables the iscsi session clean up feature. #4210
Added a condition to prevent disk partitions from being cleaned up. #4216
2020-08-21 14:38:36 +05:30
Rohit Yadav
961f3ece41 Merge remote-tracking branch 'origin/4.14' 2020-08-21 14:31:52 +05:30
davidjumani
3872bf1ff9
kvm: Enable PVLAN support on L2 networks (#4040)
This is an extention of #3732 for kvm.
This is restricted to ovs > 2.9.2
Since Xen uses ovs 2.6, pvlan is unsupported.
This also fixes issues of vms on the same pvlan unable to communicate if they're on the same host
2020-08-20 15:46:34 +05:30
Paul Angus
400641b1cf Revert "Revert "move power sync messages to debug""
This reverts commit e06cf1a642bd00cc9be5457c97a0fa2f9f9b4e0e.
2020-08-20 10:51:05 +01:00
Paul Angus
e06cf1a642 Revert "move power sync messages to debug"
This reverts commit 1954ff32b7405702013b51da45a7f2b19f8132a7.
2020-08-20 10:50:58 +01:00
Paul Angus
1954ff32b7 move power sync messages to debug 2020-08-20 10:50:35 +01:00
Paul Angus
9ae4362908 add info only log file. 2020-08-20 10:45:11 +01:00
Rakesh
9681a28c7f
api: Display hypervisor type for VM snapshot (#4275)
The "hypervisor" field in listvmsnapshot response will
be used in primate to enable/disable creating snapshot
from vm snapshot functionality.

Creating snpashot from vm snapshot will be enabled only if
hypervisor is KVM
2020-08-20 14:13:49 +05:30
Spaceman1984
d57aa83517
server: Added nfs minor version support (#4180)
This PR adds minor version support when mounting nfs on the SSVM as requested in #2861

The global setting "secstorage.nfs.version" has been changed to use the String data type which allows any minor version to be specified.
2020-08-19 14:53:38 +05:30
davidjumani
d949302d0f
packaging: Adding Centos8, Ubuntu 20.04, XCPNG8.1 Support (#4068)
* DB : Add support for MySQL 8

- Splits commands to create user and grant access on database, the old
statement is no longer supported by MySQL 8.x
- `NO_AUTO_CREATE_USER` is no longer supported by MySQL 8.x so remove
that from db.properties conn parameters

For mysql-server 8.x setup the following changes were added/tested to
make it work with CloudStack in /etc/mysql/mysql.conf.d/mysqld.cnf and
then restart the mysql-server process:

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    default-authentication-plugin=mysql_native_password

Notice the last line above, this is to reset the old password based
authentication used by MySQL 5.x.

Developers can set empty password as follows:

    > sudo mysql -u root
    ALTER USER 'root'@'localhost' IDENTIFIED BY '';

In libvirt repository, there are two related commits

2019-08-23 13:13 Daniel P. Berrangé            ● rpm: don't enable socket activation in upgrade if --listen present
2019-08-22 14:52 Daniel P. Berrangé            ● remote: forbid the --listen arg when systemd socket activation

In libvirt.spec.in

        /bin/systemctl mask libvirtd.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-ro.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-admin.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tls.socket >/dev/null 2>&1 || :
        /bin/systemctl mask libvirtd-tcp.socket >/dev/null 2>&1 || :

Co-authored-by: Wei Zhou <w.zhou@global.leaseweb.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-17 16:28:30 +05:30
Gabriel Beims Bräscher
3fe724bd32
db: Prevent NullPointerException on GenericDaoBase (#4268)
* Prevent NullPointerException on GenericDaoBase

* Fix checkstyle: remove unused import
2020-08-17 16:13:46 +05:30
Pearl Dsilva
c383269a20
testing: improve vmware ssvm test (#4262)
Add sleep before ssvm health check script is run on reboot

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-08-15 23:06:22 +05:30
Spaceman1984
b586eb22f1
Human readable sizes in logs (#4207)
This PR adds outputting human readable byte sizes in the management server logs, agent logs, and usage records. A non-dynamic global variable is added (display.human.readable.sizes) to control switching this feature on and off. This setting is sent to the agent on connection and is only read from the database when the management server is started up. The setting is kept in memory by the use of a static field on the NumbersUtil class and is available throughout the codebase.

Instead of seeing things like:
2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"106496","bytesReceived":"0","result":"true","details":"","wait":"0",}}] }

The KB MB and GB values will be printed out:

2020-07-23 15:31:58,593 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) (logid:) Seq 8-1863645820801253428: Processing: { Ans: , MgmtId: 52238089807, via: 8, Ver: v1, Flags: 10, [{"com.cloud.agent.api.NetworkUsageAnswer":{"routerName":"r-224-VM","bytesSent":"(104.00 KB) 106496","bytesReceived":"(0 bytes) 0","result":"true","details":"","wait":"0",}}] }

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Human+Readable+Byte+sizes
2020-08-13 15:55:16 +05:30
Rohit Yadav
55a5470da6 Merge remote-tracking branch 'origin/4.14' 2020-08-13 15:52:47 +05:30
Pearl Dsilva
a3d5b08334
vmware: Name public network appropriately to avoid conflicts (#4254)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2020-08-13 15:52:00 +05:30
Pearl Dsilva
c578004fe5
projects: Role based users in Projects (#4128)
Enabling Role Based users in projects
Primate PR related to the FR: apache/cloudstack-primate#382
Doc PR: https://github.com/apache/cloudstack-documentation/pull/145

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2020-08-13 15:45:39 +05:30
Pearl Dsilva
b68be664b7
server: Search vm snapshots using tags (#4213)
Search VM snapshots using tags
Currently, search of VM snapshots doesn't comply with tags passed as input
2020-08-13 15:39:08 +05:30
Rodrigo D. Lopez
3adee270c7
api: Prevent null pointer on listPublicIpAddress cmd (#4255)
* Prevent null pointer on listPublicIpAddress cmd

Insert an inner join between data_center table and user_ip_address where data_center.removed field is null

* Remove extra join and add a filter for VLAN removed
2020-08-13 15:36:19 +05:30
Rodrigo D. Lopez
6704ef2794
server: fix comparison using nullable objects (#4256)
This PR fix some wrongs comparison using nullable objects. Preventing null pointer exception
2020-08-13 15:35:23 +05:30
Abhishek Kumar
95c4ef167d
cks: fix for null hypervisor type (#4260)
Fixes #4241 

Failure to deploy VM when vm.instancename.flag is set to true

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-13 15:33:58 +05:30
Rohit Yadav
7bbb4d4bef Merge remote-tracking branch 'origin/4.14' 2020-08-12 13:46:13 +05:30
Rohit Yadav
6b6cc71be9 Merge remote-tracking branch 'origin/4.13' into 4.14 2020-08-12 13:44:41 +05:30
Spaceman1984
86939e7f9d
server: Fixed private gateway can't be deleted (#4016)
When the static route service is not available on the VPC and a static route is created, the static route is created in a revoked state.

Currently, the UI doesn't distinguish between active or revoked static routes.

This PR adds the missing state filter to the list routes command and only lists active routes in the UI.
It also ignores revoked routes when the private gateway is being removed but clears out the inactive routes before the gateway is removed.

Fixes #2908
2020-08-12 13:29:12 +05:30
Rohit Yadav
6a1ddf7d14 Merge remote-tracking branch 'origin/4.14' 2020-08-10 16:37:39 +05:30
Sina Kashipazha
ce7a32c51d
build: Fix sed command failure in Mac OS. (#4253)
apidoc build failed in Mac OS because of sed in-place command. It is a minor change that fixed this issue. More information here

Fixes: #3247
Fixes: #3312
2020-08-10 13:11:45 +05:30
Rohit Yadav
7707d3fd92 engine: fix NPE from fwd-merge 562a7db8dfd923c986f231a42c15669922574a7c
This fixes NPE caused due to merge conflict fix from the forward merge
commit 562a7db8dfd923c986f231a42c15669922574a7c and fixes travis test
regression:

    === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-07 15:31:43 +05:30
Wido den Hollander
29fb74538b
appliance: Host SSVM Debian ISO on download.cloudstack.org (#4249)
This way we do not depend on Debian keeping the ISO online or changing
URLs which can break our build system(s).
2020-08-07 13:03:43 +05:30
Gabriel Beims Bräscher
1a2cc40905
systemvmtemplate: Update debian 10 iso to 10.5.0 (#4243)
Debian 10.4.0 is not available, therefore SystemVM templates will fail to build with a missing iso.

Fixed updating with the Debian 10.5.0 iso located at https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/
2020-08-06 15:57:17 +05:30
Rohit Yadav
562a7db8df Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 23:59:16 +05:30
Wei Zhou
cd8e28b279
server: Move restoreVM to vm work job queue (#4019) 2020-08-05 09:46:55 +00:00
Rohit Yadav
db665fa8df Merge remote-tracking branch 'origin/4.14'
Conflicts:
	engine/orchestration/src/main/java/com/cloud/vm/VirtualMachineManagerImpl.java

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:46:13 +05:30
Gabriel Beims Bräscher
2237486079
server: Allow renaming cluster, host, and storage (#4165)
This PR adds implementation for changing host and storage name, additionally, it fixes a Bug on cluster updateCluster API command. This PRs also enhances the UI by allowing editing field name on Host and Storage pool. Due to the fact that there is no support to editing cluster via UI, it was not edited.

TODO: I will address Host, Cluster, and Storage Pool name edition on CloudStack Primate once the API implementation gets merged.

Details:
Prior to this PR the following API commands did not offer support for updating name:

updateHost (enhancement)
updateStoragePool (enhancement)
Additionally, updateCluster claims to support changing a cluster name (via clustername parameter); however, such operation did not work. (bug)
2020-08-05 14:08:29 +05:30
Rakesh
a529470b16
server: Fix cpuallocated value in findHostsForMIgration api (#4220)
The findHostsForMigration api displays 0% always for
cpuallocated field which is wrong.

Fixes #4221
2020-08-05 14:02:09 +05:30
Rohit Yadav
2fce8b733e
vmware: volume utilisation is always zero (#4225)
This fixes issues of virtual size to be twice in case the disk is a
linked-clone root disk. The virtual size of root disk (first in chain)
must be used.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-08-05 14:01:16 +05:30
Wei Zhou
4527424fce
vm: Reset deviceId to fix missing nic with vm (#4000)
Co-authored-by: Wei Zhou <57355700+weizhouapache@users.noreply.github.com>
2020-08-05 13:49:03 +05:30
Wido den Hollander
c3554ec31d
kvm: For ceph only if a port number has been specified define in the XML (#4231)
Ceph used to use port 6789 (no need to specify it), but with the messenger v2
from Ceph it switched to port 3300 while 6789 still works.

librados/librbd/libvirt will automatically figure out the ports to use if none is
specified.

Therefor there is no need for CloudStack to explicitely define the port in the XML
passed to Libvirt or Qemu.

Leave blank if no port number has been defined by the user.
2020-08-05 13:44:40 +05:30
Rohit Yadav
9ae1170b29 Merge remote-tracking branch 'origin/4.14' 2020-08-04 11:28:43 +05:30
Abhishek Kumar
a7f56d41c8
cks: assorted bug fixes (#4116)
Fixes #4056
Fixes #4107
Fixes #4113
Fixes #4133

Fixes deployment, template and network deletion.
Also allows filetering in listKubernetesSupportedVersions with keyword

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-08-04 11:27:32 +05:30
Wei Zhou
407e34d4e7
vrouter: remove a POSTROUTING rule for port forwarding in VPC router (#3952)
As discussed in #3937 (comment)
a rule for port forwarding in VPC router might not be needed.

This fixes the failed result of health check for network VRs.
2020-08-04 11:25:28 +05:30
Gabriel Beims Bräscher
ba6e2ac843
plugins: Redfish Client & Redfish OOBM Driver (#4175)
This PR adds support for the OOBM Redfish protocol, implementing a Java client to send HTTP requests to Redfish supported systems.

Implementation overview:
- Redfish Java client: a Java Client for Redfish that makes Redfish actions available to the HA workflow via an OOB driver.
- OOB Redfish driver: a new Out-of-band driver was created for Redfish, allowing to integrate the Redfish Client with the CloudStack Out-of-band management implementation.

Fixes: #3624
2020-07-30 10:51:16 +05:30
Rohit Yadav
36ef85012f Merge remote-tracking branch 'origin/4.14'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-07-29 14:08:02 +05:30
Gregor Riepl
c856614350
Document how to pass CIDRs lists API calls (#4035) 2020-07-29 10:33:30 +02:00
davidjumani
eec56025c8
api: Bug fixes for primate (#4214)
Adding the following fixes so primate can work without issues :
- Adding pagination for listNetworkAclLists
- Adding pagination for listRoles
- Returning mshost uuid rather than msid in list hosts response
- Allowing listVirtualMachinesMetrics to respect hostid
- Fixing return all details in template response
2020-07-29 12:26:39 +05:30
Spaceman1984
e225db46e4
ca: Removed check on client mode (#4226)
Fixes unit test failures due to behavior change in OpenJDK 11.0.8

JDK-8245077: Default SSLEngine Created in Server Role
=====================================================
In JDK 11 and later, `javax.net.ssl.SSLEngine` by default used client
mode when handshaking.  As a result, the set of default enabled
protocols may differ to what is expected. `SSLEngine` would usually be
used in server mode. From this JDK release onwards, `SSLEngine` will
default to server mode. The
`javax.net.ssl.SSLEngine.setUseClientMode(boolean mode)` method may be
used to configure the mode.
2020-07-29 12:25:00 +05:30
nvazquez
7e3b61b723 Merge branch '4.14' 2020-07-18 14:17:43 -03:00
nvazquez
5c6e79b1eb Merge branch '4.13' into 4.14 2020-07-18 14:15:46 -03:00
Nicolas Vazquez
f843c537f0
Fix snapshots garbage collection (#4188)
* Cleanup orphan entries from snapshot store ref for primary storage

* Add debug message
2020-07-18 14:12:53 -03:00