With d8c7e34b38 options were added to the host.allocators.order config. Currently, it allows adding only FirstFitRouting as the value. This PR fixes the behaviour and allows other host allocators to be added.
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Got an exception when delete a zone
```
com.cloud.utils.exception.CloudRuntimeException: The zone cannot be deleted because there are Secondary storages in this zone
```
- Changes behaviour of details param handling via global setting:
- listVirtualMachines API: when the details param is not provided, it returns whether stats are returned controlled by a new global setting `list.vm.default.details.stats`
- listVirtualMachinesMetrics API: when the details param is not provided, it uses `all` details including `stats`
- Users who are affected slow performance of the listVirtualMachines API response time can set `list.vm.default.details.stats` to `false`
- Remove ConfigKey vm.stats.increment.metrics.in.memory which was renamed to `vm.stats.increment.metrics` in #5984 and also remove unused/unnecessary global settings via upgrade path
- Changes default value of VM stats accumulation setting `vm.stats.increment.metrics` to false until a better solution emerges. Since #5984, this is true and during the execution of listVM APIs the stats are clubbed/calculated which can immensely slow down list VM API calls. Any costly operations such as summing of stats shouldn't be done during the course of a synchronous API, such as the list VM API.
- Fix UI that uses listVirtualMachinesMetrics to not call `stats` detail when in list view without metrics selected.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The user_vm_view can end up not picking the right index to join against
the user_ip_address table causing full table scan on the user_ip_address
table. This could be related to a MySQL bug
https://bugs.mysql.com/bug.php?id=41220
In a test environment with 20k shared networks and over 20M IPs, the
listVirtualMachines API was found to take over 17s to return list of
just 10 VMs. However, with this fix it would now take under 200ms to
return the list.
MySQL slow query logging showed ~nearly 20M table scans of the IP
address table:
```
# User@Host: cloud[cloud] @ localhost [127.0.0.1] Id: 39
# Query_time: 8.227541 Lock_time: 0.000014 Rows_sent: 12 Rows_examined: 19,667,235
SET timestamp=1715410270;
SELECT user_vm_view.id, user_vm_view.name /*snipped*/ FROM user_vm_view
WHERE user_vm_view.id IN (4,6,7,8,9,10,11,12,13,14,15,16);
```
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* kvm: replace ISO path in vm XML configuration during vm migration
* Update 9212: address comments
* kvm: fix vm migration if there are multiple image stores
* list by isEncrypted
* use filter on VO and cleanup
* add encryption type to volume response
* Update api/src/main/java/org/apache/cloudstack/api/command/user/volume/ListVolumesCmd.java
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
Co-authored-by: Henrique Sato <henriquesato2003@gmail.com>
This fixes https://github.com/apache/cloudstack/issues/8595
```
2024-02-01 16:23:52,473 INFO [c.c.n.s.SecurityGroupManagerImpl] (AgentManager-Handler-16:null) (logid:) Network Group full sync for agent 1 found 3 vms out of sync
2024-02-01 16:23:52,473 DEBUG [c.c.n.s.SecurityGroupManagerImpl] (AgentManager-Handler-16:null) (logid:) Security Group Mgr v2: scheduling ruleset updates for 3 vms (unique=3), current queue size=0
2024-02-01 16:23:52,473 DEBUG [c.c.n.s.SecurityGroupManagerImpl] (AgentManager-Handler-16:null) (logid:) Security Group Mgr v2: done scheduling ruleset updates for 3 vms: num new jobs=3 num rows insert or updated=0 time taken=0
2024-02-01 16:23:52,478 ERROR [c.c.n.s.SecurityGroupManagerImpl] (SecGrp-Worker-20:ctx-0aa3885d) (logid:472b30d2) Problem during SG work com.cloud.network.security.LocalSecurityGroupWorkQueue$LocalSecurityGroupWork@5
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: SELECT SQL_CACHE security_group_vm_map.id, security_group_vm_map.security_group_id, security_group_vm_map.instance_id, nics.ip4_address, vm_instance.state, security_group.name FROM security_group_vm_map INNER JOIN nics ON security_group_vm_map.instance_id=nics.instance_id INNER JOIN vm_instance ON security_group_vm_map.instance_id=vm_instance.id INNER JOIN security_group ON security_group_vm_map.security_group_id=security_group.id WHERE security_group_vm_map.security_group_id = 3 AND vm_instance.state='Running'
at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:424)
at com.cloud.utils.db.GenericDaoBase.listIncludingRemovedBy(GenericDaoBase.java:938)
at com.cloud.utils.db.GenericDaoBase.listBy(GenericDaoBase.java:928)
at com.cloud.network.security.dao.SecurityGroupVMMapDaoImpl.listBySecurityGroup(SecurityGroupVMMapDaoImpl.java:134)
at jdk.internal.reflect.GeneratedMethodAccessor555.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy245.listBySecurityGroup(Unknown Source)
at com.cloud.network.security.SecurityGroupManagerImpl2.generateRulesForVM(SecurityGroupManagerImpl2.java:246)
at com.cloud.network.security.SecurityGroupManagerImpl2.sendRulesetUpdates(SecurityGroupManagerImpl2.java:177)
at com.cloud.network.security.SecurityGroupManagerImpl2.work(SecurityGroupManagerImpl2.java:157)
at com.cloud.network.security.SecurityGroupManagerImpl2$WorkerThread$1.run(SecurityGroupManagerImpl2.java:75)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at com.cloud.network.security.SecurityGroupManagerImpl2$WorkerThread.run(SecurityGroupManagerImpl2.java:72)
Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.id, security_group_vm_map.security_group_id, security_group_vm_map.instance_id,' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
... 28 more
```
This PR fixes several issues in the testing of Veeam 11 and Veeam12
- Import Veeam.Backup.PowerShell and silently ignore the warning messages
- Fix issue when assign vm to backup offerings, which caused by separator (\r\n)
- Fix authorization failure in veeam 12a, which is because v1_4 is not supported in veeam 12a any more
- Fix exception if backup name has space
- Fix backup metrics in veeam12, which is because powershell command does not return the values needed
- Fix Incorrect datetime value, which is because powershell command returns a datetime which is not supported in Java
- Fix issue during backup restoration if VM has both ROOT and DATA disks.
This PR also has the following update
- Add integration test test/integration/smoke/test_backup_recovery_veeam.py
- Make some UI changes
- Add zone setting backup.plugin.veeam.version. If it is not set, CloudStack will get veeam version via powershell commands.
- Add zone setting backup.plugin.veeam.task.poll.interval and backup.plugin.veeam.task.poll.max.retry
This PR fixes reorder/list pools when cluster details are not set, while deploying vm / attaching volume.
Problem:
Attach volume to a VM fails, on infra with zone-wide pools & vm.allocation.algorithm=userdispersing as the cluster details are not set (passed as null) while reordering / listing pools by volumes.
Solution:
Ignore cluster details when not set, while reordering / listing pools by volumes.
Fixes#8412
Add support for 8.0.0.2 explicitly to prevent falling over to the parent version
Adds log when hypervisor capabilities fail over to the parent version
---------
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
GuestOS mappings are retrieved from the parent hypervisor version when a minor, patch hypervisor version doesn't exist.
Fixes#8412
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
This PR updates the conserve mode of default vpc tier offering to conserve_mode=1
so we can create both port forwarding and load balancing rules on a public IP in vpc tiers.
This fixes#8313
When a public IP gets removed from quarantine, the removal reason gets saved to the database; however, it may also be useful for operators to know who removed the public IP from quarantine. For that reason, this PR extends the public IP quarantine feature so that the account that deliberately removed an IP from quarantine also gets saved to the database.
This PR adds missing indexes on `alerts` & `events` tables.
For alerts table, some of the queries are part of a couple of APIs and some operations. I have added the index for the same. Ref:
8f39087377/engine/schema/src/main/java/com/cloud/alert/dao/AlertDaoImpl.java (L40-L45)
For Events table, we query for `resource_id` & `resource_type` in the UI for a resource's events. Indexes were missing, so I have added those.
Sometimes users have the need to move resources between domains, for example, in a big company, a department may be moved from one part of the company to another, changing the company's department hierarchy, the easiest way of reflecting this change on the company's cloud environment would be to move subdomains between domains, but currently ACS offers no option to do that.
This PR adds the moveDomain API, which will move domains between subdomains. Furthermore, if the domain that is being moved has any subdomains, those will also be moved, maintaining the current subdomain tree.
