- do not keep passwords in databag (/etc/cloudstack/vmpasswd.json)
- process only the password we get in (vm_password.json) from mgt server
- lookup the correct passwd server instead of adding passwd to all of them
Example:
- 4 tiers and 199 VMs running
- Start vm 200 would cause new passwd from vm_password.json (1) to be merged with /etc/cloudstack/vmpasswd.json (199)
- A curl command was exected foreach password (200) foreach tier (4) resulting in 800 calls
- In fact, since passwds are never cleaned it could very well be even more as the ip address was the key in the json file so until the ip address was reused the original password would remain and be sent to passwd server every time another vm starts.
- This took ~40 seconds
Now we just figure out the right tier and only process the new password resulting in a single curl call.
- takes 0,03 seconds!
Ldap auto creation of accounts is broken due to the security fix for
CLOUDSTACK-9369.
There was an explicit check to not allow login incase the
user doesnt exist. removed the same.
MySQL 5.7 has a more strict SQL mode by default with which CloudStack
is not compatible.
By setting the SQL Mode to a more relaxed mode on run-time we can
run without changing any SQL server settings.
Admins could also apply this to the [mysqld] section of their my.cnf:
sql_mode = 'STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'
Signed-off-by: Wido den Hollander <wido@widodh.nl>
[dvswitch blocker] CLOUDSTACK-9591: Fix systemvmtemplate to not include network detailsThis removes nic/network specific details while exporting the systemvmtemplate for vmware (ova file). Having this causes the ssvms to not deploy in dvswitch-based vmware environments that have no vswitch portgroups (dummy etc). Tested this on a local Trillian env.
* pr/2022:
CLOUDSTACK-9591: Fix guest VM ovf xml to remove network nodes
CLOUDSTACK-9591: Fix systemvmtemplate to not include network details
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
Test scenarios:
- Enable cluster HA after VR is created. Now stop and start VR and check its restart priority, should be High.
- Enable cluster HA before VR is created. Now create some VM and verify that VR created must have High restart priority.
CLOUDSTACK-9783: Improve metrics view performanceThis improves the metrics view feature by improving the rendering performance
of metrics view tables, by re-implementing the logic at the backend and data
served via APIs. In large environments, the older implementation would
make several API calls that increases both network and database load.
List of APIs introduced for improving the performance that re-implement the frontend logic at backend:
listClustersMetrics
listHostsMetrics
listInfrastructure
listStoragePoolsMetrics
listVMsMetrics
listVolumesMetrics
listZonesMetrics
Pinging for review - @abhinandanprateek @DaanHoogland @borisstoyanov @karuturi @rashmidixit
Marvin test results:
=== TestName: test_list_clusters_metrics | Status : SUCCESS ===
=== TestName: test_list_hosts_metrics | Status : SUCCESS ===
=== TestName: test_list_infrastructure_metrics | Status : SUCCESS ===
=== TestName: test_list_pstorage_metrics | Status : SUCCESS ===
=== TestName: test_list_vms_metrics | Status : SUCCESS ===
=== TestName: test_list_volumes_metrics | Status : SUCCESS ===
=== TestName: test_list_zones_metrics | Status : SUCCESS ===
* pr/1944:
CLOUDSTACK-9783: Improve metrics view performance
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
This removes network details from the guest VM template OVF xml before deploying
a VM which would fail in case of dvswitch-based vmware environment with no
dummy/existing vswitch.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This removes nic/network specific details while exporting the systemvmtemplate
for vmware (ova file). Having this causes the ssvms to not deploy in
dvswitch-based vmware environments that have no vswitch portgroups (dummy etc).
Tested this on a local Trillian env.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Displays a VR tab that lists VRs for the network in the detail views for
isolated networks, shared networks and for VPCs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9811: fix duplicated nics on VR caused by nic name p<slot_number>p<port_number>
* pr/2011:
CLOUDSTACK-9811: fix duplicated nics on VR caused by nic name p<slot_number>p<port_number>
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
[4.9] CLOUDSTACK-9569: propagate global configuration router.aggregation.command.each.timeout to KVM agentThe router.aggregation.command.each.timeout in global configuration is only applied on new created KVM host.
For existing KVM host, changing the value will not be effective.
We need to propagate the configuration to existing host when cloudstack-agent is connected.
* pr/1856:
CLOUDSTACK-9569: propagate global configuration router.aggregation.command.each.timeout to KVM agent
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
This improves the metrics view feature by improving the rendering performance
of metrics view tables, by reimplementing the logic at the backend and data
served via APIs. In large environments, the older implementation would
make several API calls that increases both network and database load.
List of APIs introduced for improving the performance:
listClustersMetrics
listHostsMetrics
listInfrastructure
listStoragePoolsMetrics
listVMsMetrics
listVolumesMetrics
listZonesMetrics
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-9746 system-vm: logrotate config causes critical failures* rotate both daily and by size by using maxsize in stead of size
* decrease the max size to 10M for rsyslog files
* remove delaycompress for rsyslog files
* increase rotate to 10 for cloud.log
* pr/1915:
CLOUDSTACK-9746 system-vm: logrotate config causes critical failures
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9363: Fix HVM VM restart bug in XenServerHere is the longer description of the problem:
By default XenServer limits HVM guests to only 4 disks. Two of those are reserved for the ROOT disk (deviceId=0) and CD ROM (device ID=3) which means that we can only attach 2 data disks. This limit however is removed when Xentools is installed on the guest. The information that a guest has Xentools installed and can handle more than 4 disks is stored in the VM metadata on XenServer. When a VM is shut down, Cloudstack removes the VM and all the metadata associated with the VM from XenServer. Now, when you start the VM again, even if it has Xentools installed, it will default to only 4 attachable disks.
Now this problem manifests itself when you have a HVM VM and you stop and start it with more than 2 data disks attached. The VM fails to start and the only way to start the VM is to detach the extra disks and then reattach them after the VM start.
In this fix, I am removing the check which is done before creating a `VBD` which enforces this limit. This will not affect current workflow and will fix the HVM issue.
@koushik-das this is related to the "autodetect" feature that you introduced a while back (https://issues.apache.org/jira/browse/CLOUDSTACK-8826). I would love your review on this fix.
* pr/1829:
Fix HVM VM restart bug in XenServer
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-8663: Fixed various issues to allow VM snapshots and volumesnapshots to exist together
Reverting VM to disk only snapshot in Xenserver corrupts VM
Stale NFS secondary storage on XS leads to volume creation failure from snapshot
Fixed various concerns raised in #672
* pr/1941:
CLOUDSTACK-8663: Fixed various issues to allow VM snapshots and volume snapshots to exist together
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9789: Fix releasing secondary guest IP fails with associated static nat which is actually not used
* pr/1947:
CLOUDSTACK-9789: Fix releasing secondary guest IP fails with associated static nat which is actually not used
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9628: Fix Template Size in Swift as Secondary StorageCloudstack incorrectly uses the physical size as the size of the
template. Ideally, the size should refelct the virtual size. This
PR fixes that issue.
* pr/1770:
CLOUDSTACK-9628: Use correct virtualsize with Swift as secondary storage
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
* rotate both daily and by size by using maxsize in stead of size
* decrease the max size to 10M for rsyslog files
* remove delaycompress for rsyslog files
* increase rotate to 10 for cloud.log
CLOUDSTACK-9691: Fixed unhandeled excetion in list snapshot command when a primary store is deleted related to it@mike-tutkowski After support for snapshots on solidifire there are many places which are prone to these NullPointer exceptions resulting in various issues. Root cause for these issues is that we get the primary storage associated with snapshot and then figure out how to handle but if that store is deleted then it results in NullPointer exceptions. Without solidfire we don't need to access primary storage.
Should we handle that as issues are found or could there be other way to fix all of these issues?
* pr/1847:
CLOUDSTACK-9691: Added test list_snapshots_with_removed_data_store
CLOUDSTACK-9691: Fixed unhandeled excetion in list snapshot command when a primary store is deleted related to it
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
CLOUDSTACK-9655 The template which is registered in all zones will be deleted by deleting 1 template on any zoneadded extra warning message if it's a cross-zone template ("This is a
cross zone template and will be deleted from all the zones. Are you sure
you want to proceed?").
* pr/1818:
CLOUDSTACK-9655 The template which is registered in all zones will be deleted by deleting 1 template on any zone
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
snapshots to exist together
Reverting VM to disk only snapshot in Xenserver corrupts VM
Stale NFS secondary storage on XS leads to volume creation failure from snapshot
CLOUDSTACK-8896: allocated percentage of storage pool going beyond 100%This issue occurs when a volume in Ready state is moved across storage
pools.
While finding if the storage pool has enough space, it has a check to
consider the size of non Ready volumes only. This is true if the volume
to be attached to a vm is in the same storage pool. But, if the volume
is in another storage pool and has to be moved to a vm's storage pool,
the size of the volume should be considered in doing the space check.
computing the asking size when volume is not in ready state or when the
volume is on a different storage pool.
Testing:
I couldnt write unittests for it. This class is not in a unittestable state.
manually tested in the below environment
1. xenserver 6.5 setup with 2 clusters and a host each in each of them.
2. added storage tags for the primary storage.
3. created two service offerings with the storage tags.
4. deployed two vms using newly created offerings in step 3.
5. at this stage, there are two vms one on each host with root disks on the corresponding primary.
6. create a data disk and attach it to vm1
7. detach the data disk. now the data disk is in the primary storage of the cluster of vm1 (let us say primary1)
8. attach this data disk to vm2(running on a host in different cluster)
9. the volume should be moved to the primary storage of another cluster and op_host_capacity should be accordingly updated.
* pr/873:
CLOUDSTACK-8896: allocated percentage of storage pool going beyond 100%
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
deleted by deleting 1 template on any zone
added extra warning message if it's a cross-zone template ("This is a
cross zone template and will be deleted from all the zones. Are you sure
you want to proceed?").
This issue occurs when a volume in Ready state is moved across storage
pools.
While finding if the storage pool has enough space, it has a check to
consider the size of non Ready volumes only. This is true if the volume
to be attached to a vm is in the same storage pool. But, if the volume
is in another storage pool and has to be moved to a vm's storage pool,
the size of the volume should be considered in doing the space check.
computing the asking size when volume is not in ready state or when the
volume is on a different storage pool.
[4.9] CLOUDSTACK-9770: fix missing ip routes in VRIn network VR, the routes to current subnets are missing in corresponding ip route Table.
It is a typo in commit 6749785caba78a9379e94bf3aaf0c1fbc44c5445
In VPC VR, it works fine.
* pr/1929:
CLOUDSTACK-9770: fix missing ip routes in VR
Signed-off-by: Rajani Karuturi <rajani.karuturi@accelerite.com>
