11029 Commits

Author SHA1 Message Date
Pearl Dsilva
665142701d
server: Fix NPE when no recipients configured for sending alerts (#5154)
* Fix NPE when no recipients configured for sending alerts

* Adding logs

* Address comments
2021-07-05 11:50:10 +05:30
Pearl Dsilva
50bf2861c0
server: Fix NPE during destroy VM (#5142) 2021-07-05 11:48:43 +05:30
Pearl Dsilva
293dd4d8ef
volume: Fix deletion of Uploaded volumes (#5125)
Fixes issue with deletion of Uploaded volumes
2021-07-01 19:49:41 +05:30
Abhishek Kumar
5fd970d88f
db, server: refactor host_view to prevent duplicate entries (#4796)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-07-01 11:00:51 -03:00
Rakesh
9fd38c4da0
server: Display proper names in error message (#5140)
Display the proper account and other names rather than the
uuid representation of the object
2021-06-29 13:03:31 +05:30
Wei Zhou
9ac3925936
server: list routers by healthchecksfailed (#4886)
* server: list routers by healthchecksfailed

* #4886: fix build error

* #4886: ignore global setting

Co-authored-by: Wei Zhou <weizhou@apache.org>
2021-06-29 12:52:24 +05:30
davidjumani
29109b4332
db: Cleanup obsolete tables (#5002)
* db: Cleanup unused tables

* Removing volume_host_ref references

* Removing template_host_ref references

* fix space issue

* Fix fk constraint

* Removing certificate table

* Revert "Removing certificate table"

This reverts commit fa24e6483f339903ce895e26e3409a1751620a3f.

* Addressing comments
2021-06-24 16:50:31 -03:00
davidjumani
1a7bfa98d8
Follow consistent vo table name convention (#5004) 2021-06-24 16:46:50 -03:00
Daniel Augusto Veronezi Salvador
a3f39db62b
server: Remove meaningless password regeneration on resetSSHKeyForVirtualMachine (#4819)
On API `resetSSHKeyForVirtualMachine`, ACS also regenerates VM password when it uses a template with `Password Enabled` as true; there is already anAPI to reset VM password, therefore, the reset SSH keys API should not reset the VM SSH password as well.

Besides running a meaningless process, the VM's password regeneration slows down the main process and may cause a confusion in operations due to password change in the VM without being explicity requested.

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-06-17 11:57:52 +05:30
nicolas
d2ab350a40 Merge branch '4.15' into main 2021-06-16 15:38:18 -03:00
Wei Zhou
46a3032178
server: set correct gateway when update vm nic on shared networks (#5105)
This PR fixes the issue that nic has wrong gateway after updating vm nic.

Steps to reproduce the issue
(1) create shared network (in advanced zone or advanced zone with sg)
(2) create new shared network (with same startip/endip/netmask, but different gateway).
(3) create a vm in new network
(4) stop vm and update vm nic ip address

Expected result:
The vm has correct gateway and netmask (of second network)

Actual result:
The vm has wrong gateway and netmask (of first network)
2021-06-16 21:19:42 +05:30
sureshanaparti
07cabbe7ac
scaleio: Updated PowerFlex/ScaleIO gateway client with some improvements. (#5037)
- Added connection manager to the gateway client.
 - Renew the client session on '401 Unauthorized' response.
 - Refactored the gateway client calls, for GET and POST methods.
 - Consume the http entity content after login/(re)authentication and close the content stream if exists.
 - Updated storage pool client connection timeout configuration 'storage.pool.client.timeout' to non-dynamic.
 - Added storage pool client max connections configuration 'storage.pool.client.max.connections' (default: 100) to specify the maximum connections for the ScaleIO storage pool client.
 - Updated unit tests.
and blocked the attach volume operation for uploaded volume on ScaleIO/PowerFlex storage pool
2021-06-16 12:45:27 +05:30
Daniel Augusto Veronezi Salvador
e962f0f271
Unify SMTP mail sending (#4954)
* Add mail dependencies

* Create util to send SMTP mail

* Add unit tests to SMTP mail sender

* Use SMTP mail util on quota alert

* Use SMTP mail util on alert

* Use SMTP mail util on project

* Use SMTP mail util on usage alert

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

* Remove copyright line in license header

Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
Co-authored-by: Gabriel Beims Bräscher <gabrascher@gmail.com>
2021-06-15 23:18:26 +02:00
Gabriel Beims Bräscher
bc12833ccf
server: Failed to scale between Service Offerings with the same root disk size (#5095)
* Cover a case where resizing root disk failed; add isNotPossibleToResize method.

* remove format from resize validation

* Revert if-conditional changes that removed ImageFormat.ISO validation

* Add JUnit tests for VolumeApiServiceImpl.isNotPossibleToResize

* Fix checkstyle of test Class

* Use _templateDao.findByIdIncludingRemoved instead of _templateDao.findById

* Prevent null serviceOfferingView and Mock findByIdIncludingRemoved instead of findById
2021-06-14 12:49:55 +05:30
Harikrishna
12b2e80d82
vmware: Fix fetching chain_info of the volumes. It is used to assume datastore names are in the form of UUIDs but it can be any name. So fetch chain_info based on the datastore name. (#5097)
his PR fixes the problem of not updating the chain info or setting chain info to null after volume migrations.

Problem: While fetching the volume chain info, management server assumes datastore name to be a UUID (this is true only for NFS storages added by CloudStack) but datastore name can be with any name.
Solution: To fetch the volume chain info, use datastore name instead of UUID.

The fix is made in the flow of following API operations

migrateVirtualMachine
migrateVirtualMachineWithVolume
migrateVolume
2021-06-11 20:06:06 +05:30
Daniel Augusto Veronezi Salvador
bf6266188c
Improve logs on RemoteAccessVpnManagerImpl (#4960)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-06-10 16:51:45 +02:00
Rohit Yadav
a0788f5816 Merge remote-tracking branch 'origin/4.15' 2021-06-10 12:39:44 +05:30
slavkap
b0f2112dc2
kvm: Fix of some UEFI related issues (#5072)
* Fix of some UEFI related issues

1 - fix of attach/detach ISO of VM with UEFI boot type
2 - if OS type of an ISO is categorized as "Other" the bus type of the disk
will be set to "sata"

* Simplify the validation of OS types
2021-06-09 11:15:07 +05:30
Gabriel Beims Bräscher
1afcd823e0
server: Root disk size should be listed in GB at listServiceOffering (#5085)
* Root disk size should be listed in GB at listServiceOffering

* Update UI to handle rootdisk size in GB instead of Bytes
2021-06-09 11:14:18 +05:30
Harikrishna
083646b3c3
server: VM dynamic scaling option granularity (#4643)
This PR introduces new granularity levels to configure VM dynamic scalability. Previously VM is configured to be dynamically scalable based on the template and global setting. Now we bringing this option to configure at service offering and VM level also.

VM can dynamically scale only when all flags are ON at VM level, template, service offering and global setting. If any of the flags is set to false then VM cannot be scalable. This result will be persisted in DB for each VM and will be honoured for that VM till it is updated.

We are introducing 'dynamicscalingallowed' parameter with permitted values of true or false for deployVM API and createServiceOffering API.

Following are the API parameter changes:
createServiceOffering API: 
          dynamicscalingenabled: an optional parameter of type Boolean with default value “true”.
deployVirtualMachine API: 
          dynamicscalingenabled: an optional parameter of type Boolean with default value “true”.

Following are the UI changes:
Service offering creation has ON/OFF switch for dynamic scaling enabled with default value true
2021-06-08 17:10:00 +05:30
Suresh Kumar Anaparti
958182481e cloudstack: make code more inclusive
Inclusivity changes for CloudStack

- Change default git branch name from 'master' to 'main' (post renaming/changing default git branch to 'main' in git repo)
- Rename some offensive words/terms as appropriate for inclusiveness.

This PR updates the default git branch to 'main', as part of #4887.

Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-06-08 15:47:20 +05:30
Rohit Yadav
5aefd3e204 Merge remote-tracking branch 'origin/4.15' 2021-06-02 14:03:54 +05:30
davidjumani
704b24e68f
server: Fix string format error (#5061) 2021-06-01 17:34:19 +05:30
Rohit Yadav
d184e282b2 Merge remote-tracking branch 'origin/4.15' 2021-05-31 20:01:09 +05:30
Pearl Dsilva
d04fa0201d
server: usage generated for destroyed VMs with no backups (#5017)
Fixes: #4990
When a VM associated with a backup offering is destroyed/expunged, the backup offering isn't unassigned, and despite the VM having no backups present, backup usage is generated. This PR prevent usage record generation when there are no backups present for a VM with a backup offering associated to it. This is done by ensuring that usage event for backups is generated only when a the backup size > 0
2021-05-31 18:59:48 +05:30
Rohit Yadav
57e4e3b6b1 Merge remote-tracking branch 'origin/4.15' 2021-05-31 16:04:42 +05:30
Abhishek Kumar
b0cb087bd0
server: fixes NPE on empty vmware.root.disk.controller config (#5045)
* server: fixes NPE on empty vmware.root.disk.controller config

When global config - vmware.root.disk.controller is set to empty and template is registered with deployasis, server will throw NPE while deploying a VM. This change fixes the problem by using default value of the config in this case.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* use StringUtils utility

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>

* fix indentation

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2021-05-31 14:30:11 +05:30
Gabriel Beims Bräscher
a3cdd1f836
Allow deploy Admin VMs and VRs in disabled zones/pods/clusters (#3600) 2021-05-28 10:45:30 +02:00
Daniel Augusto Veronezi Salvador
6531ee5871
Externalize config to set min memory/cpu with division by overprovisi… (#4722)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2021-05-28 10:15:51 +02:00
Daniel Augusto Veronezi Salvador
06744c16ba
Externalize OOBM background task's interval (#4881)
* Externalize OOBM background task's interval

* Rename variable

Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-05-25 11:42:53 +02:00
Abhishek Kumar
4b7fa4e630 Merge remote-tracking branch 'apache/4.15'
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-24 16:43:53 +05:30
slavkap
d47e273329
server: Prevent NPE if hypervisor's capabilities are null (#5029)
If the hypervisor's capabilities are null, CloudRuntimeException will be thrown;
Format the error message.
2021-05-21 15:19:04 +05:30
Spaceman1984
5f734f718e
vmware: Disk controller vmware deploy as is (#5006)
Fixes #4344
2021-05-19 14:33:16 +05:30
Pearl Dsilva
2eae0f5385
SystemVM: Set agent state to disconnected on Stopping the systemVM (#5010)
Fixes: #4972
This PR sets systevms' agent state to disconnected when it is stopped. Currently, when a systemVM (Console Proxy VM / Secondary storage VM) is stopped, the agent state still appears to be 'Up'
2021-05-19 13:00:17 +05:30
Abhishek Kumar
426f14b6ed Merge remote-tracking branch 'apache/4.15'
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-18 15:19:20 +05:30
Rohit Yadav
2286c8d2bf Merge remote-tracking branch 'origin/4.14' into 4.15 2021-05-14 23:19:06 +05:30
Abhishek Kumar
dc91a1fd4d
server: destroy ssvm, cpvm on last host maintenance (#4644)
* server: destroy ssvm, cpvm on last host maintenance

When a single or last UP host enters into maintenance just stopping SSVM and CPVM will leave behind VMs on hypervisor side. As these system vms will be recreated they can be destroyed.
Fixes #3719

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix methods

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* immediately destroy systemvms

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix destroy

Added bypassHostMaintenance flag in Comma.java class to allow command to be handled by host agent even when host is in maintenace.
Flag is set true only for delete commands for ssvm and cpvm.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unit test fix

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix missing return statement

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* fix

VM should be stopped with cleanup before calling expunge else it server may through error with host in PrepareForMaintenance state.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* rename

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* refactor

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-14 23:16:15 +05:30
Abhishek Kumar
755791089d
api/server: cpu, memory values with overprovisioning in metrics response (#4795)
* metrics: operprovisioned host cpu allocated

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* changes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* remove unused

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-05-14 23:15:35 +05:30
Wei Zhou
e2183ed666
forceha: fix two issues when (1)stop vm from inside (2) force remove host (#4647)
* forceha: fix vm is not started if it is poweroff from inside

steps to reproduce the issue
(1) make sure force.ha is true in global setting. if not, change it to true, and restart mgt server
(2) create a service offering , ha is not enabled
(3) create a vm
(4) log into the vm, and power off via cli.

expected result: vm is started again by cloudstack
actual result: vm is not started.

* forceha: fix vms are still running if host is force-removed

when host can be force removed, however vms are stopped in cloudstack, but not stopped on host
```
(localcloud) 🐱 > delete host id="a5625393-444d-4d0a-b31d-62baf88a8be1" forced=true
{
  "success": true
}```

after some minutes, vms are still runnning on host
```
root@mgt01:~# ssh node63 virsh list
 Id   Name        State
---------------------------
 1    i-2-19-VM   running
 2    i-2-11-VM   running
```

error message are
```
Cannot transmit host 2 to Enabled state
com.cloud.utils.fsm.NoTransitionException: No next resource state found for current state = Enabled event = DeleteHost
        at com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1216)
        at com.cloud.resource.ResourceManagerImpl$1.doInTransactionWithoutResult(ResourceManagerImpl.java:907)
```

* forceha: Make ForceHA dynamic
2021-05-14 23:14:39 +05:30
Wei Zhou
1b28ea1ebb
network: fix dhcp/password/metadata issues on shared networks with multiple subnets (#5013)
* #4943: apply iptables for password and metadata

* #4943: fix wrong ip alias

* #4943: revert previous change and add ip_aliases

Co-authored-by: Wei Zhou <weizhouapache@gmail.com>
2021-05-13 14:31:47 +05:30
Nicolas Vazquez
aa289542f0
vmware: Fix deploy-as-is not honoured on upload from local (#5015)
* Fix deploy-as-is not honoured on upload from local

* Missing param
2021-05-13 14:30:34 +05:30
Daniel Augusto Veronezi Salvador
cc2c6e91cb
Improve logs on HAManagerImpl (#4707)
Co-authored-by: Daniel Augusto Veronezi Salvador <daniel@scclouds.com.br>
2021-05-10 10:22:47 +02:00
Rohit Yadav
cb167072a1 Merge remote-tracking branch 'origin/4.15'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-07 16:37:42 +05:30
Harikrishna
32e3bbdcc5
VMware Datastore Cluster primary storage pool synchronisation (#4871)
Datastore cluster as a primary storage support is already there. But if any changes at vCenter to datastore cluster like addition/removal of datastore is not synchronised with CloudStack directly. It needs removal of primary storage from CloudStack and add it again to CloudStack.

Here synchronisation of datastore cluster is fixed without need to remove or add the datastore cluster.
1. A new API is introduced syncStoragePool which takes datastore cluster storage pool UUID as the parameter. This API checks if there any changes in the datastore cluster and updates management server accordingly.
2. During synchronisation if a new child datastore is found in datastore cluster, then management server will create a new child storage pool in database under the datastore cluster. If the new child storage pool is already added as an individual storage pool then the existing storage pool entry will be converted to child storage pool (instead of creating a new storage pool entry)
3. During synchronisaton if the existing child datastore in CloudStack is found to be removed on vCenter then management server removes that child datastore from datastore cluster and makes it an individual storage pool.
The above behaviour is on par with the vCenter behaviour when adding and removing child datastore.
2021-05-07 16:30:54 +05:30
Rohit Yadav
f5e665cd63
server: NPE may cause management server to not start (#5001)
This NPE may happen when a VM is marked removed in the DB but not its
nics on a shared network. This can usually happen due to a failed
expunged VM or when an admin manually marks a VM as removed in DB but
does not cleanup the nics/network resources.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2021-05-07 00:03:16 +05:30
Rohit Yadav
fce84e62ab Merge remote-tracking branch 'origin/4.15' 2021-05-06 00:27:43 +05:30
Pearl Dsilva
bc80815cf5
server: Adding VPN options for IKE version and IKE split connections (#4953)
IKE version allows selecting ike (autoselect), ikev1, or ikev2.
Split connections gives an option of separating the first right subnet from the rest, and kicking out individual statements for each right subnet for better cross-compatibility.

Backported from PR: #4137
update per PR suggestion

Fixes #3138

Co-authored-by: Greg Goodrich <ggoodrich@ippathways.com>
Co-authored-by: Daan Hoogland <dahn@onecht.net>
Co-authored-by: Pearl Dsilva <pearl.dsilva@shapeblue.com>
2021-05-05 12:54:23 +05:30
Rohit Yadav
1abd10199c Merge remote-tracking branch 'origin/4.15' 2021-05-04 19:37:45 +05:30
Gabriel Beims Bräscher
ab790c11d5
server: Allow to upgrade service offerings from local <> shared storage pools (#4915)
This PR addresses the issue raised at #4545 (Fail to change Service offering from local <> shared storage).

When upgrading a VM service offering it is validated if the new offering has the same storage scope (local or shared) as the current offering. I think that the validation makes sense in a way of preventing running Root disks with an offering that does not match the current storage pool. However, the validation only compares both offerings and does not consider that it is possible to migrate Volumes between local <> shared storage pools.

The idea behind this implementation is that CloudStack should check the scope of the current storage pool which the ROOT volume is allocated; this, it is possible to migrate the volume between storage pools and list/upgrade according to the offerings that are supported for such pool.

This PR also fixes an issue where the API command that lists offerings for a VM should follow the same idea and list based on the storage pool that the volume is allocated and not the previous offering.

Fixes: #4545
2021-04-30 11:59:50 +05:30
Nicolas Vazquez
9349d20dd3
vmware: Make deploy-as-is optional (#4901)
* [Vmware] Make deploy-as-is optional

* Do not use deployasis on create volume test

* Update api/src/main/java/org/apache/cloudstack/api/command/user/template/RegisterTemplateCmd.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/template/RegisterTemplateCmd.java

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Review comments

* Refactor condition to select suitable template

Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2021-04-30 09:47:50 +05:30