733 Commits

Author SHA1 Message Date
Abhishek Kumar
426f14b6ed Merge remote-tracking branch 'apache/4.15'
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-18 15:19:20 +05:30
Rohit Yadav
2286c8d2bf Merge remote-tracking branch 'origin/4.14' into 4.15 2021-05-14 23:19:06 +05:30
Abhishek Kumar
dc91a1fd4d
server: destroy ssvm, cpvm on last host maintenance (#4644)
* server: destroy ssvm, cpvm on last host maintenance

When a single or last UP host enters into maintenance just stopping SSVM and CPVM will leave behind VMs on hypervisor side. As these system vms will be recreated they can be destroyed.
Fixes #3719

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix methods

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* immediately destroy systemvms

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix destroy

Added bypassHostMaintenance flag in Comma.java class to allow command to be handled by host agent even when host is in maintenace.
Flag is set true only for delete commands for ssvm and cpvm.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit test fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix missing return statement

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

VM should be stopped with cleanup before calling expunge else it server may through error with host in PrepareForMaintenance state.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-14 23:16:15 +05:30
Abhishek Kumar
755791089d
api/server: cpu, memory values with overprovisioning in metrics response (#4795)
* metrics: operprovisioned host cpu allocated

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unused

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-14 23:15:35 +05:30
Wei Zhou
e2183ed666
forceha: fix two issues when (1)stop vm from inside (2) force remove host (#4647)
* forceha: fix vm is not started if it is poweroff from inside

steps to reproduce the issue
(1) make sure force.ha is true in global setting. if not, change it to true, and restart mgt server
(2) create a service offering , ha is not enabled
(3) create a vm
(4) log into the vm, and power off via cli.

expected result: vm is started again by cloudstack
actual result: vm is not started.

* forceha: fix vms are still running if host is force-removed

when host can be force removed, however vms are stopped in cloudstack, but not stopped on host
```
(localcloud) 🐱 > delete host id="a5625393-444d-4d0a-b31d-62baf88a8be1" forced=true
{
  "success": true
}```

after some minutes, vms are still runnning on host
```
root@mgt01:~# ssh node63 virsh list
 Id   Name        State
---------------------------
 1    i-2-19-VM   running
 2    i-2-11-VM   running
```

error message are
```
Cannot transmit host 2 to Enabled state
com.cloud.utils.fsm.NoTransitionException: No next resource state found for current state = Enabled event = DeleteHost
        at com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1216)
        at com.cloud.resource.ResourceManagerImpl$1.doInTransactionWithoutResult(ResourceManagerImpl.java:907)
```

* forceha: Make ForceHA dynamic
2021-05-14 23:14:39 +05:30
Wei Zhou
1b28ea1ebb
network: fix dhcp/password/metadata issues on shared networks with multiple subnets (#5013)
* #4943: apply iptables for password and metadata

* #4943: fix wrong ip alias

* #4943: revert previous change and add ip_aliases

Co-authored-by: Wei Zhou <weizhouapache@gmail.com>
2021-05-13 14:31:47 +05:30
Nicolas Vazquez
aa289542f0
vmware: Fix deploy-as-is not honoured on upload from local (#5015)
* Fix deploy-as-is not honoured on upload from local

* Missing param
2021-05-13 14:30:34 +05:30
Daniel Augusto Veronezi Salvador
cc2c6e91cb
Improve logs on HAManagerImpl (#4707)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-05-10 10:22:47 +02:00
Rohit Yadav
cb167072a1 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-07 16:37:42 +05:30
Harikrishna
32e3bbdcc5
VMware Datastore Cluster primary storage pool synchronisation (#4871)
Datastore cluster as a primary storage support is already there. But if any changes at vCenter to datastore cluster like addition/removal of datastore is not synchronised with CloudStack directly. It needs removal of primary storage from CloudStack and add it again to CloudStack.

Here synchronisation of datastore cluster is fixed without need to remove or add the datastore cluster.
1. A new API is introduced syncStoragePool which takes datastore cluster storage pool UUID as the parameter. This API checks if there any changes in the datastore cluster and updates management server accordingly.
2. During synchronisation if a new child datastore is found in datastore cluster, then management server will create a new child storage pool in database under the datastore cluster. If the new child storage pool is already added as an individual storage pool then the existing storage pool entry will be converted to child storage pool (instead of creating a new storage pool entry)
3. During synchronisaton if the existing child datastore in CloudStack is found to be removed on vCenter then management server removes that child datastore from datastore cluster and makes it an individual storage pool.
The above behaviour is on par with the vCenter behaviour when adding and removing child datastore.
2021-05-07 16:30:54 +05:30
Rohit Yadav
f5e665cd63
server: NPE may cause management server to not start (#5001)
This NPE may happen when a VM is marked removed in the DB but not its
nics on a shared network. This can usually happen due to a failed
expunged VM or when an admin manually marks a VM as removed in DB but
does not cleanup the nics/network resources.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-07 00:03:16 +05:30
Rohit Yadav
fce84e62ab Merge remote-tracking branch 'origin/4.15' 2021-05-06 00:27:43 +05:30
Pearl Dsilva
bc80815cf5
server: Adding VPN options for IKE version and IKE split connections (#4953)
IKE version allows selecting ike (autoselect), ikev1, or ikev2.
Split connections gives an option of separating the first right subnet from the rest, and kicking out individual statements for each right subnet for better cross-compatibility.

Backported from PR: #4137
update per PR suggestion

Fixes #3138

Co-authored-by: Greg Goodrich <ggoodrich@ippathways.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-05-05 12:54:23 +05:30
Rohit Yadav
1abd10199c Merge remote-tracking branch 'origin/4.15' 2021-05-04 19:37:45 +05:30
Gabriel Beims Bräscher
ab790c11d5
server: Allow to upgrade service offerings from local <> shared storage pools (#4915)
This PR addresses the issue raised at #4545 (Fail to change Service offering from local <> shared storage).

When upgrading a VM service offering it is validated if the new offering has the same storage scope (local or shared) as the current offering. I think that the validation makes sense in a way of preventing running Root disks with an offering that does not match the current storage pool. However, the validation only compares both offerings and does not consider that it is possible to migrate Volumes between local <> shared storage pools.

The idea behind this implementation is that CloudStack should check the scope of the current storage pool which the ROOT volume is allocated; this, it is possible to migrate the volume between storage pools and list/upgrade according to the offerings that are supported for such pool.

This PR also fixes an issue where the API command that lists offerings for a VM should follow the same idea and list based on the storage pool that the volume is allocated and not the previous offering.

Fixes: #4545
2021-04-30 11:59:50 +05:30
Nicolas Vazquez
9349d20dd3
vmware: Make deploy-as-is optional (#4901)
* [Vmware] Make deploy-as-is optional

* Do not use deployasis on create volume test

* Update api/src/main/java/org/apache/cloudstack/api/command/user/template/RegisterTemplateCmd.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/template/RegisterTemplateCmd.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Review comments

* Refactor condition to select suitable template

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-30 09:47:50 +05:30
Rohit Yadav
4742ac15f7 Merge remote-tracking branch 'origin/4.15' 2021-04-29 21:50:40 +05:30
dahn
be255e4203
server: protect against stray snapshot-details without snapshot (#4924)
This PR makes sure no orphaned snapshot details are considered in the cleanup at startup job.
a real solution would be to implement some kind of cascading delete, but as the parent record is "only" marked as removed this would be a bit com

Co-authored-by: Daan Hoogland <dahn@onecht.net>
2021-04-29 20:40:29 +05:30
Pearl Dsilva
98e2ed3c4f
vmware: Add force parameter to iso attach/detach operations (#4907)
Fixes: #4808, #4941

This PR adds a force flag to the attachIso / detachIso commands, especially for VMware where it is noticed that when trying to either detach an iso or attach an iso when there already exists another present it fails to do the necessary operation as from ACS end we either answer the question returned by Esxi for CDRom disconnect operation as No (for detach operation) or do not answer the question at all (for Attach operation).

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-28 13:46:03 +05:30
Rohit Yadav
1e859a0e2c Merge remote-tracking branch 'origin/4.15' 2021-04-27 22:20:30 +05:30
Abhishek Kumar
8e31d1e650
api: fix disk/service offering keys (#4946)
Volume can either have an associated disk offering (for DATA disks & ROOT disks for VMs created from ISO) or a compute/service offering (for ROOT disks of VMs created from templates).
This fix simplifies and fixes check to return the appropriate response keys in these cases.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-27 21:54:24 +05:30
Pearl Dsilva
d4db1015b8
marvin: fix test_scale_vm for xenserver/Xcp-ng (#4929)
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-27 10:21:48 +05:30
Abhishek Kumar
42c83b08f5 Merge remote-tracking branch 'apache/4.15' 2021-04-26 14:33:58 +05:30
lujiefsi
f8ba33d570
server: Some APIs should have access check (#4859)
This PR fixes the CLOUDSTACK-10434. I think some APIs lack access check and list them in below table. I also give the pattch to add the access check for the api in this table. Anyone chould change this table, If you think the APIs do not need access check and change their lable as "no".

API	Lack?
VolumeApiServiceImpl # updateVolume	yes
VolumeApiServiceImpl # detachVolumeViaDestroyVM	yes
VolumeApiServiceImpl # takeSnapshot	yes
VolumeApiServiceImpl # migrateVolume	yes
AccountManagerImpl#createApiKeyAndSecretKey	yes
LoadBalancingRulesManagerImpl#applyLBStickinessPolicy	yes
LoadBalancingRulesManagerImpl#applyLBHealthCheckPolicy	yes
TemplateManagerImpl#createPrivateTemplate	yes
SnapshotManagerImpl#updateSnapshotPolicy

Co-authored-by: lujie <lujie@foxmail.com>
2021-04-26 13:39:03 +05:30
Abhishek Kumar
a30d518e8a
vmware: fix stopped VM volume migration (#4758)
* prevent other vm disks getting deleted

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* vmware: fix inter-cluster stopped vm migration

Fixes #4838

For inter-cluster migration without shared storage, VMware needs a host to be specified. Fix is to specify an appropriate host in the target cluster.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix detached volume inter-cluster migration

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* cleanup unused method

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* review changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* vmware: allow attached volume migration using VmwareStorageMotionStrategy

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* find vm clusterid with multiple ROOT volumes

VM can have multiple ROOT volumes and some can be on zone-wide store therefore iterate over all of them till a cluster ID is found.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix successive storage migration

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix intercluster check

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor vm cluster, host method

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove inter-pod check

Added by mistake, VMware won't have pods

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* address review comment

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-24 18:55:25 +05:30
Rohit Yadav
e107f9aa93 Merge remote-tracking branch 'origin/4.15' 2021-04-21 13:07:44 +05:30
Rohit Yadav
5051fde952
server: Stat collector solidfire capacity fix (#4918)
Fixes regression introduced in 71c5dbcf492a023dbea5f8c34f8fd883c3ad653f
which would cause capacity bytes of certain pools to be update which
shouldn't get updated by StatsCollector such as solidfire.

Fixes #4911

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-04-21 12:48:11 +05:30
Gabriel Beims Bräscher
de557663ec
Migrate/Stop VMs with local storage when preparing host for maintenance (#4212) 2021-04-19 09:41:42 +02:00
Rohit Yadav
f42024714c Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-04-15 16:17:13 +05:30
Rohit Yadav
0302750aac
vmware: Add support for VMware 7 (#4300) 2021-04-15 16:10:14 +05:30
Harikrishna
f00b5fc7ac
server: Fix for the issue of recover VM not able to attach the data disks which are there before destroy in case of VMware (#4493)
This PR fixes: #4462

Problem Statement:
In case of VMware, when a VM having multiple data disk is destroyed (without expunge) and tried to recover the VM then the previous data disks are not attached to the VM like before destroy. Only root disk is attached to the VM.

Root cause:
All data disks were removed as part of VM destroy. Only the volumes which are selected to delete (while destroying VM) are supposed to be detached and destroyed.

Solution:
During VM destroy, detach and destroy only volumes which are selected during VM destroy. Detach the other volumes during expunge of VM.
2021-04-15 12:50:53 +05:30
Abhishek Kumar
cce736709e Merge remote-tracking branch 'apache/4.15' 2021-04-12 11:43:57 +05:30
Rohit Yadav
e824fdba20 Merge remote-tracking branch 'origin/4.14' into 4.15
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	server/src/main/java/com/cloud/vm/UserVmManagerImpl.java
2021-04-10 13:41:50 +05:30
Abhishek Kumar
6b1c94ea3e
server: fix root disk size on vm reset (#4638)
If VM details contain rootdisksize, volume entry in DB should reflect correct size when VM reset is performed.

Fixes #3957

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-10 13:31:51 +05:30
Spaceman1984
99a9063cf4
server: Added recursive fetch of child domains for listUsageRecords API call (#4717)
When calling the listUasageRecords API records per domain are fetched recursively. This is not the case if you specify a domain id.

This PR adds a new parameter to enable fetching records recursively (isRecursive) when passing the domain id.
2021-04-10 13:15:29 +05:30
Rohit Yadav
22f6c19248 Merge remote-tracking branch 'origin/4.15' 2021-04-09 13:21:07 +05:30
Abhishek Kumar
6d0d7012f1
api,engine/schema,server: vm readonly details rename (#4671)
* api,engine/schema,server: vm readonly details rename

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* ui: update api response key

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 13:20:11 +05:30
Rohit Yadav
ca8920dd36 Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-09 13:17:39 +05:30
Abhishek Kumar
cd60b8d97d
host-allocator: check capacity for suitable hosts (#4884)
Fixes #4517

Adds capacity checks for RandomAllocator (host allocator)

Factors out host cpu capability and capacity check wrt serviceoffering code into CapacityManager.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-09 12:35:58 +05:30
Rohit Yadav
ecc18163ca Merge remote-tracking branch 'origin/4.15' 2021-04-07 11:41:16 +05:30
Rakesh
262285639e
kvm: Disable shrinking QCOW2 volumes (#4679)
Since QCOW2 volumes cannot be resized to lower value in any case,
just disable it in backend.
2021-04-07 11:25:52 +05:30
Rohit Yadav
49acd1dec7 Merge remote-tracking branch 'origin/4.15' 2021-04-06 15:09:48 +05:30
slavkap
52c36cadf0
kvm: Fix deploy VM from ISOs with UEFI (#4773)
This PR fixes #4244
deploying of VMs from ISOs and from templates with UEFI boot type
deploying of VMs from ISOs and from templates with UEFI boot type with
volumes in RAW format
2021-04-06 15:00:21 +05:30
Rohit Yadav
ca5c11a29a Merge remote-tracking branch 'origin/4.15' 2021-04-06 12:55:08 +05:30
Abhishek Kumar
1a335e880a
server: filter null details during volume to template creation (#4794)
Fixes #4628

mysql> describe user_vm_details;
+---------+-----------------+------+-----+---------+----------------+
| Field   | Type            | Null | Key | Default | Extra          |
+---------+-----------------+------+-----+---------+----------------+
| id      | bigint unsigned | NO   | PRI | NULL    | auto_increment |
| vm_id   | bigint unsigned | NO   | MUL | NULL    |                |
| name    | varchar(255)    | NO   |     | NULL    |                |
| value   | varchar(5120)   | YES  |     | NULL    |                |
| display | tinyint(1)      | NO   |     | 1       |                |
+---------+-----------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)

mysql> describe vm_template_details;
+-------------+-----------------+------+-----+---------+----------------+
| Field       | Type            | Null | Key | Default | Extra          |
+-------------+-----------------+------+-----+---------+----------------+
| id          | bigint unsigned | NO   | PRI | NULL    | auto_increment |
| template_id | bigint unsigned | NO   | MUL | NULL    |                |
| name        | varchar(255)    | NO   |     | NULL    |                |
| value       | varchar(1024)   | NO   |     | NULL    |                |
| display     | tinyint(1)      | NO   |     | 1       |                |
+-------------+-----------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)

While cloud.user_vm_details allows null values to be added for a detail, cloud.vm_template_details doesn't allow null values.
This change filters vm details with null values while creating template from a volume.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-06 12:48:56 +05:30
Pearl Dsilva
0dbeb262e4
server: Support for persistence mode in L2 networks (#4561)
This PR aims at introducing persistence mode in L2 networks and enhancing the behavior in Isolated networks
Doc PR apache/cloudstack-documentation#183

Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-04-05 14:37:11 +05:30
Rohit Yadav
3783fd5cec Merge remote-tracking branch 'origin/4.15' 2021-04-05 13:00:18 +05:30
Rakesh
47b72883e2
server: Disable VR health check for VPC without tiers (#4888)
If the VPC has redundant VR's but doesnt have any tiers or there are no running VM's in
a VPC then disable the gateway check for VR
2021-04-05 12:33:45 +05:30
Rohit Yadav
d4635e3442 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-04-01 14:35:01 +05:30
Rohit Yadav
2cdde8774b Merge remote-tracking branch 'origin/4.14' into 4.15 2021-04-01 14:33:08 +05:30