Add a global setting to control whether redirection is allowed while
downloading templates and volumes
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Add a global setting to control whether redirection is allowed while
downloading templates and volumes
core: some changes on SimpleHttpMultiFileDownloader
similar as HttpTemplateDownloader
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit b1642bc3bf58ccde9f56f632b5a9fe46a3eb5356)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This introduces new global settings to handle how client address checks
are handled by the API layer:
proxy.header.verify: enables/disables checking of ipaddresses from a
proxy set header
proxy.header.names: a list of names to check for allowed ipaddresses
from a proxy set header.
proxy.cidr: a list of cidrs for which \"proxy.header.names\" are
honoured if the \"Remote_Addr\" is in this list.
(cherry picked from commit b65546636d84a5790e0297b1b0ca8e5a67a48dbc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b1e0bf9dbd464f8fb7c22f36505dee0148e2d6f4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Move allow.additional.vm.configuration.list.kvm from Global to Account setting
- Disallow VM details start with "extraconfig" when deploy VMs
- Skip changes on VM details start with "extraconfig" when update VM settings
- Allow only extraconfig for DPDK in service offering details
- Check if extraconfig values in vm details are supported when start VMs
- Check if extraconfig values in service offering details are supported when start VMs
- Disallow add/edit/update VM setting for extraconfig on UI
(cherry picked from commit e6e4fe16fb1ee428c3664b6b57384514e5a9252e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 7aea9db1c8d8ef1febb723913dc01c3040641425)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This introduces new global settings to handle how client address checks
are handled by the API layer:
proxy.header.verify: enables/disables checking of ipaddresses from a
proxy set header
proxy.header.names: a list of names to check for allowed ipaddresses
from a proxy set header.
proxy.cidr: a list of cidrs for which \"proxy.header.names\" are
honoured if the \"Remote_Addr\" is in this list.
(cherry picked from commit b65546636d84a5790e0297b1b0ca8e5a67a48dbc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Move allow.additional.vm.configuration.list.kvm from Global to Account setting
- Disallow VM details start with "extraconfig" when deploy VMs
- Skip changes on VM details start with "extraconfig" when update VM settings
- Allow only extraconfig for DPDK in service offering details
- Check if extraconfig values in vm details are supported when start VMs
- Check if extraconfig values in service offering details are supported when start VMs
- Disallow add/edit/update VM setting for extraconfig on UI
(cherry picked from commit e6e4fe16fb1ee428c3664b6b57384514e5a9252e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* snapshot: don't schedule next snapshot job for a removed volume
When management server starts, it starts the snapshot scheduler. In case
there is a volume snapshot policy which exists for a volume which does
not exist, it can cause SQL constraint issue and cause the management
server to break from starting its various components and cause HTTP 503
error.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* remove schedule on missing volume
---------
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* storage,plugins: delegate allow zone-wide volume migration check and access grant to storage drivers
Following checks have been delegated to storage drivers,
- For volumes on zone-wide storage, whether they need storage migration when VM is migrated
- Whther volume required grant access
Apply fixes in resolving PrimaryDataStore
* add tests
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* unused import
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
* Update engine/orchestration/src/test/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestratorTest.java
---------
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
This fixes https://github.com/apache/cloudstack/issues/8595
```
2024-02-01 16:23:52,473 INFO [c.c.n.s.SecurityGroupManagerImpl] (AgentManager-Handler-16:null) (logid:) Network Group full sync for agent 1 found 3 vms out of sync
2024-02-01 16:23:52,473 DEBUG [c.c.n.s.SecurityGroupManagerImpl] (AgentManager-Handler-16:null) (logid:) Security Group Mgr v2: scheduling ruleset updates for 3 vms (unique=3), current queue size=0
2024-02-01 16:23:52,473 DEBUG [c.c.n.s.SecurityGroupManagerImpl] (AgentManager-Handler-16:null) (logid:) Security Group Mgr v2: done scheduling ruleset updates for 3 vms: num new jobs=3 num rows insert or updated=0 time taken=0
2024-02-01 16:23:52,478 ERROR [c.c.n.s.SecurityGroupManagerImpl] (SecGrp-Worker-20:ctx-0aa3885d) (logid:472b30d2) Problem during SG work com.cloud.network.security.LocalSecurityGroupWorkQueue$LocalSecurityGroupWork@5
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: SELECT SQL_CACHE security_group_vm_map.id, security_group_vm_map.security_group_id, security_group_vm_map.instance_id, nics.ip4_address, vm_instance.state, security_group.name FROM security_group_vm_map INNER JOIN nics ON security_group_vm_map.instance_id=nics.instance_id INNER JOIN vm_instance ON security_group_vm_map.instance_id=vm_instance.id INNER JOIN security_group ON security_group_vm_map.security_group_id=security_group.id WHERE security_group_vm_map.security_group_id = 3 AND vm_instance.state='Running'
at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:424)
at com.cloud.utils.db.GenericDaoBase.listIncludingRemovedBy(GenericDaoBase.java:938)
at com.cloud.utils.db.GenericDaoBase.listBy(GenericDaoBase.java:928)
at com.cloud.network.security.dao.SecurityGroupVMMapDaoImpl.listBySecurityGroup(SecurityGroupVMMapDaoImpl.java:134)
at jdk.internal.reflect.GeneratedMethodAccessor555.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy245.listBySecurityGroup(Unknown Source)
at com.cloud.network.security.SecurityGroupManagerImpl2.generateRulesForVM(SecurityGroupManagerImpl2.java:246)
at com.cloud.network.security.SecurityGroupManagerImpl2.sendRulesetUpdates(SecurityGroupManagerImpl2.java:177)
at com.cloud.network.security.SecurityGroupManagerImpl2.work(SecurityGroupManagerImpl2.java:157)
at com.cloud.network.security.SecurityGroupManagerImpl2$WorkerThread$1.run(SecurityGroupManagerImpl2.java:75)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at com.cloud.network.security.SecurityGroupManagerImpl2$WorkerThread.run(SecurityGroupManagerImpl2.java:72)
Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.id, security_group_vm_map.security_group_id, security_group_vm_map.instance_id,' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
... 28 more
```
PR #8549 replaced RSA with ed25519. unfornately, ed25519 is unsupported in FIPS mode
```
$ ssh-keygen -t ed25519 -m PEM -N '' -f key1
ED25519 keys are not allowed in FIPS mode
$ ssh-keygen -t ecdsa -m PEM -N '' -f key1
Generating public/private ecdsa key pair.
Your identification has been saved in key1
Your public key has been saved in key1.pub
The key fingerprint is:
.........
```
* Introduced a new API checkVolumeAndRepair that allows users or admins to check and repair if any leaks observed.
Currently this is supported only for KVM
* some fixes
* Added unit tests
* addressed review comments
* add repair volume while granting access
* Changed repair parameter to accept both leaks/all
* Introduced new global setting volume.check.and.repair.before.use to do volume check and repair before VM start or volume attach operations
* Added volume check and repair changes only during VM start and volume attach operations
* Refactored the names to look similar across the code
* Some code fixes
* remove unused code
* Renamed repair values
* Fixed unit tests
* changed version
* Address review comments
* Code refactored
* used volume name in logs
* Changed the API to Async and the setting scope to storage pool
* Fixed exit value handling with check volume command
* Fixed storage scope to the setting
* Fix volume format issues
* Refactored the log messages
* Fix formatting
