10548 Commits

Author SHA1 Message Date
Gabriel Beims Bräscher
cd6f0cb1e1
Prevent overflow on StatsCollector + add a few enhancements on code (#3932) 2020-03-13 19:51:12 +01:00
Wei Zhou
19fb23781b
server: password is not displayed when reinstall a vm or reset… (#3948) 2020-03-12 11:14:34 +01:00
Rohit Yadav
0fab5e8d60
server: fix database exception while searching network offerings (#3947)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-03-06 10:34:09 +01:00
Nicolas Vazquez
bd7d41bf6d
server: fix VM with ISO attached migration issue (#3935)
As previously described by PR #3929:
If vm has attached ISO, the migration fails with error message "org.libvirt.LibvirtException: Cannot access storage file /mnt/b33e5a1d-e4ea-3465-b6ac-c98dc8ff8af0/207-2-cc5fd717-2d57-3bb3-bcf6-2c930268db6c.iso"
2020-03-06 13:32:19 +05:30
Rohit Yadav
b4fdf22397
kvm: fix/optimize propogating configs (#3911)
Make some changes based on @nvazquez 's comments in PR #3491
Fix a bug in #3491
2020-03-05 12:20:51 +05:30
Wei Zhou
313e21a0da
VR: Fix Redundant VRouter guest network on wrong interface (#3847) 2020-02-29 19:52:40 +01:00
Wei Zhou
79f7f0f007
server: fix issue while list ssh keypairs by keyword (#3916)
in 4.13, list sshkeypairs with keyword will ignore the search by name if name is specifed
Fixes an issue in #3098

for example,
(local) > list sshkeypairs name=wei keyword=wei filter=name
{
  "count": 3,
  "sshkeypair": [
    {
      "name": "wei3"
    },
    {
      "name": "wei2"
    },
    {
      "name": "wei"
    }
  ]
}

with this patch ,it gives correct result.

(local) > list sshkeypairs name=wei keyword=wei filter=name
{
  "count": 1,
  "sshkeypair": [
    {
      "name": "wei"
    }
  ]
}
2020-02-28 15:05:49 +05:30
Rakesh
7e30e3d141
router: Avoid duplicate alerts when router state changes (#3904)
When both routers of VPC is in MASTER state
then multiple alerts are sent equally to the number of tiers in the VPC.
If the VPC has 3 tiers then 6 alerts will be sent. This is not good
if VPC has more than 10 networks in it.

Instead of checking the router status for all the tiers in the VPC,
just check the status of the router for one tier in a VPC so that
multiple duplicate alerts can be avoided
2020-02-28 14:24:12 +05:30
Rakesh
abb39a25af
server: send VM password to all Running VRs in network/vpc (#3903)
Currently, the cloudstack sends VM password only to the first
router in the network even if its the backup and return the result.

In some cases the first router will be back up and the second will be master.
Since password server is not running in backup, when the user resets the password,
it is sent to the first router which can be backup.
In that case, the new password is not stored in the password server and users cant log in with a new password.

This change ensures that we send the password to both the routers instead
of the first router so that a new password is stored in the master router.
2020-02-28 12:00:16 +05:30
Pearl Dsilva
4d8a2da133
api: Fix count and item issues returned by list APIs (#3894) 2020-02-26 15:14:23 +00:00
Rakesh
e269b14095
Fix network rules issue if default egress policy is Allow (#3905) 2020-02-23 21:12:06 +00:00
Wei Zhou
ac7bcde45b
KVM: Propagating changes on host parameters to the agents (#3491) 2020-02-19 13:13:37 +00:00
Wei Zhou
37d2b8537c
kvm: Enable virtio drivers based on guest os display name (#3879)
When we add new guest os, sometimes we missed the records in guest_os_hypervisor.
However, the guest disk model (virtio/ide) is determined by record in the table.
It causes the issue that some new guest os(eg Debian 8/9) uses e1000 instead of virtio nic, and ide disk instead of virtio disk.

To fix the issue permanantly, pass the guest os name in guest_os if the record for kvm is not found in guest_os_hypervisor.

Related commit:7ac9f00eeeb4cd37ec39efeba066e799b581b1a0
2020-02-19 13:12:20 +05:30
Wei Zhou
649ed45965
kvm: fix exception in volume statts after storage migration (#3884)
On kvm, the 'path' of volume is the file name on primary storage. we should use 'path' instead of 'uuid' in volume statistics.

Fixes: #3878
2020-02-19 13:06:19 +05:30
Rakesh
bbe2bf1a6e
server: ignore site to site vpn status check on internallbvm (#3864)
When the state of the site to site vpn changes, the check
is done on all the virtual routers including the internal
load balancing vm as well. It is not needed to check the
state for internal load balancing vm
2020-02-18 14:03:30 +05:30
Rohit Yadav
78cc0a44c1
server: use host record related to a ssvm/cpvm (#3876)
This implements the systemvm list API response creator to find and use
the host record for a ssvm/cpvm to get the agent status and other
details like last disconnected date and agent version.

Fixes 3875

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-10 19:01:34 +05:30
Rohit Yadav
afcbbc4b3e
systemvm: list systemvm does not return agent state and version (#3870)
This makes the listSystemVms API to return the host status (agent state),
version and last pinged information. This makes it possible for UIs
to call a single API to get this information.
2020-02-07 13:19:35 +01:00
Wei Zhou
a9a1737dd9
vpc: set traffic type of private gateway IP to Public to fix ke… (#3851) 2020-02-06 20:22:08 +01:00
Pearl Dsilva
bfdb914693
usage: publish zone id while uploading template and volume (#3867)
After a local template is uploaded via browser, the generated usage event with type = "TEMPLATE.CREATE" is persisted with the data store ID instead of the zone ID on the zone_id column. The fix will refactor the upload monitor logic, as after the upload completes, it sets the datastore ID on the zone ID column for the created "TEMPLATE.CREATE" usage event. This refactor will query the DB for the data store and will set its associated zone ID in the usage field.
The fix produces the same behaviour as when registering a template from URL.
FIx is also for uploading VOLUME from local/via browser.
2020-02-06 11:31:24 +05:30
Abhishek Kumar
a71874682c
server: fix checking disk offering access for snapshot volume (#3791)
Fixes #3783
As reported in the issue, creating volumes from pure snapshot fails with NPE. This is due to order of calls where disk offering access is checked before checking disk offering value. This PR fixes the same.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2020-01-31 15:24:25 +05:30
Abhishek Kumar
9d105b6546
template: copy md5 mismatch (#3383)
Fixes #3191

When a template is registered, code stores md5sum of the downloaded file in the vm_template table. However, this downloaded file could be deleted after template installation if it is not an actual (.qcow2, .ova, etc.) file. When the user copies a template using copyTemplate API, the actual template file will be copied across the image stores. Matching checksum for the copied templated file and the stored value from the vm_template table will result in a mismatch.
Changes will set an empty checksum value for the copied template while passing to download service which allows skipping wrong checksum check for the copied while install.
However, this results in a change in checksum value for concerned template entry in vm_template table post template install.

Co-authored-by: dahn <daan.hoogland@gmail.com>
2020-01-31 14:16:37 +05:30
davidjumani
7a25e40d5a
api: allow listing management server by id and name (#3840)
The List Management Server api returns a list of all the management servers but fails when trying to list by id or name. This ensures that it fetches the details as per the parameters passed.
Fixes: #3833
2020-01-30 10:38:25 +05:30
Pearl Dsilva
1c130a5dd4
api: metrics API response is not super-set of resources response keys (#3834)
The metrics API has few properties missing that are present in the corresponding resource. 

Fixes #3831

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
2020-01-30 08:49:45 +05:30
Gregor Riepl
8792070f84 Rethrow takeVMSnapshot() exception instead of returning null in VMSnapshotManagerImpl (#3761)
Fixes: #3518
2020-01-28 11:05:15 +05:30
Wei Zhou
a77d74ba0d server: Fix NPE while update displayvm on vm with dynamic service offering (#3758)
Steps to reproduce the issue
(1) create a custom service offering
(2) create a vm with the offering
(3) update vm with displayvm=false, returns an error

(local) > update virtualmachine id=f33fd06a-7643-40d1-833f-272845d9ba09 displayvm=false
Error 530: {"updatevirtualmachineresponse":{"uuidList":[],"errorcode":530,"cserrorcode":9999}}
2020-01-28 11:04:26 +05:30
Wei Zhou
136505b22c server: double check host capacity when start/migrate a vm (#3728)
When start a vm or migrate a vm (away from a host in host maintenance), cloudstack will check capacity of all hosts and choose one. If there are hundreds of hosts on the platform, it will take some seconds. When cloudstack choose a host and start/migrate vm to it, the resource consumption of the host might have been changed. This normally happens when we start/migrate multiple vms.
It would be better to double check the host capacity when start vm on a host.

This PR includes the fix for cpucore capacity when start/migrate a vm.
2020-01-28 10:55:11 +05:30
Wei Zhou
71e53ab01d server: Capacity check should take vms in Migrating state into calculation (#3727)
When we calculate a resource consumption of a host, we need to take the vms in following states into calculation: Running, Starting, Stopping, Migrating (to the host), and vms are Migrating from the host. Because, when stop a vm, the resource on host will be released when vm is stopped. When migrate a vm, the resource on destination host will be increased before migration starts, and resource on source host will be decreased after migraiton succeeds.

In cloudstack, there is a task named CapacityChecked which run every 5 minutes (capacity.check.period =300000 ms by default). It recalculates capacity of all hosts. However, it takes only vms in Running and Starting into consideration. We have faced some issues in host maintenance due to it.

Steps to reproduce the issue
(1) migrate N vms from host A to host B, cpu/ram resource increases before the migration.
(2) capacity check recalculate the capacity of hosts. used capacity of Host B will be reset to original value (not including the vms in Migrating).
(3) migrate some more vms from other host to host B, the migrations are allowed by cloudstack (because used capacity is incorrect). If the actual used memory exceed the physical memory on the host, there might be some critical issues (for example, libvirt dies)
2020-01-28 10:54:32 +05:30
Pearl Dsilva
b581f85d4b fixed inconsistency of IP on VR when VR is destroyed and recrea… (#3825) 2020-01-23 16:59:00 +01:00
Wei Zhou
b79a539191 server: fix resource count error when upgrade a vm (#3759)
Steps to reproduce the issue
(1) create an account (test)
(2) create a vm with the account (test)
(3) login with admin, and upgrade the vm to another offering
(4) the resource count (cpu,memory) of admin increases, not the account (test).
2020-01-23 16:53:43 +01:00
dahn
6ebd02e7af Agent lb on svm (#3795)
* add host if needed

* allow for test with null-host

* full coverage of states

* EnumSet.contains instead of Arrays.binarySearch
2020-01-14 15:40:21 +01:00
mdominka
54cc73af08 Add missing HA config keys (#3776) 2020-01-14 09:35:34 +01:00
dahn
8844f59247
break session only on illegal origin (#3715) 2019-12-24 10:22:34 +01:00
Spaceman1984
100308536c Added zone check for attach iso (#3755) 2019-12-23 16:48:51 +01:00
Wei Zhou
2712decf50 config: add isdynamic flag in configuration response (#3729) 2019-12-20 15:17:21 +01:00
dahn
3ac03c8858
filter hosts to query on zone wide storage (#3733) 2019-12-20 14:12:10 +01:00
dahn
0b34971b6d
Once again allow a VM to be on multiple networks from VPCs (#3754)
to once again allow a VM to be on multiple networks from VPCs
2019-12-20 14:10:02 +01:00
Nicolas Vazquez
07d64cb65a
[KVM] Agent LB Fix: Connections from disabled KVM host agents are refused (#3617)
* Remove constraint for just enabled hosts receiving the ms list

* Allow sending SetupMSList command when host is in maintenance
2019-12-04 10:43:17 -03:00
Wei Zhou
94474af879 consoleproxy: Enable console for vms in Stopping/Migrating state (#3640) 2019-11-28 12:03:38 +01:00
Wei Zhou
d981edb4bc server: acquire IPv4 address when add secondary IP to nic if IP is not specified (#3635)
After commit fbf488497fb863c13fc0908281e3f4f86906df43, admin need to specify an ipv4 or ipv6 addresses when add IP to nic which breaks backward compatibity. If IP is not specified, a IPv4 address should be returned.
2019-11-21 11:44:41 +05:30
Wei Zhou
cc27095554 server: Do NOT cleanup dhcp and dns when stop a vm (#3627)
* server: Do NOT cleanup dhcp and dns when stop a vm

According comment in PR #3608, dhcp and dns entries are cleaned up only when a VM is expunged.
Revert part of commit 8fb388e9312b917a8f36c7d7e3f45985a95ce773.

* server: cleanup dns/dhcp entries in removeNic instead of finalizeExpunge
2019-10-18 01:20:55 +05:30
Rohit Yadav
b853ebd8f7
server: Cleanup dhcp and dns entries only on expunging VM (#3608)
This fixes a behaviour to not cleanup DHCP and DNS rules for NICs of a
VM in the VR when it is stopped, but instead when VM is expunged because
stopped VMs in CloudStack still retain the IPs and records.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-09-26 18:16:37 +05:30
Paul Angus
61b8b77913 Updating pom.xml version numbers for release 4.13.1.0-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-09-01 13:36:50 +01:00
Paul Angus
80c740fd4e Merge branch '4.13.0.0-RC20190820T1535' into 4.13 2019-09-01 13:29:59 +01:00
Gabriel Beims Bräscher
87c4350160 Table "user_ipv6_address" (UserIpv6AddressVO) is empty and seems that it (#3571)
is not used; probably it is a legacy code/table.

Therefore, remove the verification that counts the IPs from
UserIpv6AddressVO in order to check if it can use the network for
deploying new VMs in UI [1].

[1] com.cloud.network.NetworkModelImpl.canUseForDeploy(Network).
2019-08-28 19:25:29 +01:00
Rohit Yadav
b7470bc51c
server: fix NPE for the case where volume is not attached to a VM (#3566)
Fixes NPE when trying to find suitable storage pools for a volume
when the volume is not attached to a VM.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-08-22 16:07:25 +05:30
Paul Angus
8e08b47cc9 Updating pom.xml version numbers for release 4.13.0.0
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2019-08-20 15:35:49 +01:00
Nicolas Vazquez
bfc08715cc Display VM snapshot tags on usage records (#3560)
* Refactor usage helper tables to include VM snapshot id

* Fix resource type and resource id while listing usage records

* Add defensive checks
2019-08-20 14:20:23 +01:00
Gabriel Beims Bräscher
d7aa0a2718 server: Prevent NullPointer on a network with removed IP ranges/"VLANs" (#3551)
When a network IP range is removed, the "vlan" stays mapped on pod_vlan_map; therefore, the method that lists the VLANs by pod id will return null VLANS.

This PR adds proper verifications to avoid null pointer exception when deploying VRs on a pod with removed VLANs. The exception was caused on getPlaceholderNicForRouter.
2019-08-15 13:35:58 +05:30
Rohit Yadav
b576972f71
test: stabilize 4.13/master (#3547)
Fix failing smoketests, fix NPEs. 

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-08-13 11:51:10 +05:30
Nicolas Vazquez
3c2af55d81 vmware: allow configuring appliances on the VM instance wizard when OVF properties are available (#3271)
Problem: In Vmware, appliances that have options that are required to be answered before deployments are configurable through vSphere vCenter user interface but it is not possible from the CloudStack user interface.

Root cause: CloudStack does not handle vApp configuration options during deployments if the appliance contains configurable options. These configurations are mandatory for VM deployment from the appliance on Vmware vSphere vCenter. As shown in the image below, Vmware detects there are mandatory configurations that the administrator must set before deploy the VM from the appliance (in red on the image below):

Solution:
On template registration, after it is downloaded to secondary storage, the OVF file is examined and OVF properties are extracted from the file when available.
OVF properties extracted from templates after being downloaded to secondary storage are stored on the new table 'template_ovf_properties'.
A new optional section is added to the VM deployment wizard in the UI:
If the selected template does not contain OVF properties, then the optional section is not displayed on the wizard.
If the selected template contains OVF properties, then the optional new section is displayed. Each OVF property is displayed and the user must complete every property before proceeding to the next section.
If any configuration property is empty, then a dialog is displayed indicating that there are empty properties which must be set before proceeding
image
The specific OVF properties set on deployment are stored on the 'user_vm_details' table with the prefix: 'ovfproperties-'.
The VM is configured with the vApp configuration section containing the values that the user provided on the wizard.
2019-08-09 16:14:46 +05:30