11800 Commits

Author SHA1 Message Date
Wei Zhou
0b857def68
New feature: Import/Unamange DATA volume from storage pool (#8808) 2024-04-23 16:05:59 +02:00
Wei Zhou
f6802e9198
server: fix duplicated records for templates if there are multiple zones (#8933) 2024-04-23 12:06:45 +02:00
Rohit Yadav
0fa71f5696 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-04-23 15:21:44 +05:30
Vishesh
c081f60427
server: Fix null pointer exception in restore VM (#8930) 2024-04-22 19:21:16 +02:00
dahn
0577b0ac8e
server: add logs to public ip allocation attempt (#8239) 2024-04-22 10:31:48 +02:00
SadiJr
49d244f251
[Usage] Create VPC billing (#7235)
Co-authored-by: SadiJr <sadi@scclouds.com.br>
Co-authored-by: Bryan Lima <bryan.lima@hotmail.com>
2024-04-22 09:42:24 +02:00
João Jandre
8a101fbbc1 Updating pom.xml version numbers for release 4.18.3.0-SNAPSHOT
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-04-17 11:11:57 -03:00
Vishesh
8511014707
Fix error message for checkVolume command (#8842) 2024-04-17 17:27:27 +05:30
Henrique Sato
075b413041
Fix VR inheriting boot settings from user's VM (#8894)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-04-16 16:54:16 +02:00
Vishesh
b998e7dbb6
Allow overriding root disk offering & size, and expunge old root disk while restoring a VM (#8800)
* Allow overriding root diskoffering id & size while restoring VM

* UI changes

* Allow expunging of old disk while restoring a VM

* Resolve comments

* Address comments

* Duplicate volume's details while duplicating volume

* Allow setting IOPS for the new volume

* minor cleanup

* fixup

* Add checks for template size

* Replace strings for IOPS with constants

* Fix saveVolumeDetails method

* Fixup

* Fixup UI styling
2024-04-12 17:47:52 +05:30
João Jandre
154566f914 Updating pom.xml version numbers for release 4.18.2.0
Signed-off-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2024-04-12 08:25:04 -03:00
Vishesh
730cc5d5b8
Change iops on offering change (#8872)
* Change IOPS on disk offering change

* Remove iops & bandwidth limits before copying template

* minor refactor

* Handle diskOfferingDetails

* Fixup
2024-04-11 17:01:55 +05:30
Vishesh
c24c1a5c00
Fix VM import & VM delete with custom offering (#8813) 2024-04-10 11:29:29 +02:00
Wei Zhou
c7626ebfd6
Merge remote-tracking branch 'apache/4.18' into 4.19 2024-04-08 14:14:04 +02:00
Wei Zhou
a5508acc54
server: fix haproxy misconfiguration after VPC VR start (#8881) 2024-04-05 15:01:08 -03:00
Vishesh
c25264bada
UI Fixes: allow filtering on alertype in UI & add events tab (#8630) 2024-04-04 13:51:56 +02:00
Vishesh
b8f8321f0f
Allow listing of inactive offerings (#8821) 2024-04-04 13:46:44 +02:00
Abhishek Kumar
ff3e9bd821 engine-storage: control download redirection
Add a global setting to control whether redirection is allowed while
downloading templates and volumes

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-04-04 14:11:05 +05:30
Wei Zhou
939d0b9011 engine-storage: control download redirection
Add a global setting to control whether redirection is allowed while
downloading templates and volumes

core: some changes on SimpleHttpMultiFileDownloader
similar as HttpTemplateDownloader

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
(cherry picked from commit b1642bc3bf58ccde9f56f632b5a9fe46a3eb5356)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:19:20 +05:30
dahn
67e2061f4b api: client verification in servlet
This introduces new global settings to handle how client address checks
are handled by the API layer:

proxy.header.verify: enables/disables checking of ipaddresses from a
                     proxy set header
proxy.header.names: a list of names to check for allowed ipaddresses
                    from a proxy set header.
proxy.cidr: a list of cidrs for which \"proxy.header.names\" are
            honoured if the \"Remote_Addr\" is in this list.

(cherry picked from commit b65546636d84a5790e0297b1b0ca8e5a67a48dbc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit b1e0bf9dbd464f8fb7c22f36505dee0148e2d6f4)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:19:15 +05:30
Wei Zhou
fc3c625beb server: fix security issues caused by extraconfig on KVM
- Move allow.additional.vm.configuration.list.kvm from Global to Account setting
- Disallow VM details start with "extraconfig" when deploy VMs
- Skip changes on VM details start with "extraconfig" when update VM settings
- Allow only extraconfig for DPDK in service offering details
- Check if extraconfig values in vm details are supported when start VMs
- Check if extraconfig values in service offering details are supported when start VMs
- Disallow add/edit/update VM setting for extraconfig on UI

(cherry picked from commit e6e4fe16fb1ee428c3664b6b57384514e5a9252e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 7aea9db1c8d8ef1febb723913dc01c3040641425)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:19:08 +05:30
dahn
00f687db1b api: client verification in servlet
This introduces new global settings to handle how client address checks
are handled by the API layer:

proxy.header.verify: enables/disables checking of ipaddresses from a
                     proxy set header
proxy.header.names: a list of names to check for allowed ipaddresses
                    from a proxy set header.
proxy.cidr: a list of cidrs for which \"proxy.header.names\" are
            honoured if the \"Remote_Addr\" is in this list.

(cherry picked from commit b65546636d84a5790e0297b1b0ca8e5a67a48dbc)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:00:33 +05:30
Wei Zhou
72b2eb0087 server: fix security issues caused by extraconfig on KVM
- Move allow.additional.vm.configuration.list.kvm from Global to Account setting
- Disallow VM details start with "extraconfig" when deploy VMs
- Skip changes on VM details start with "extraconfig" when update VM settings
- Allow only extraconfig for DPDK in service offering details
- Check if extraconfig values in vm details are supported when start VMs
- Check if extraconfig values in service offering details are supported when start VMs
- Disallow add/edit/update VM setting for extraconfig on UI

(cherry picked from commit e6e4fe16fb1ee428c3664b6b57384514e5a9252e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-04-04 11:00:10 +05:30
Wei Zhou
08d9d06d45
api,server,ui: add project ID and name to UserDataResponse (#8656)
* api,server,ui: add project ID and name to UserDataResponse

* Update: add since
2024-03-25 19:15:34 +05:30
Marcus Sorensen
2e88eb45a3
Update mysql-connector version (#8753)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2024-03-21 18:09:06 +05:30
Rohit Yadav
308ed1399e Merge remote-tracking branch 'origin/4.18' into 4.19 2024-03-20 15:49:21 +05:30
Rohit Yadav
720407ba73
snapshot: don't schedule next snapshot job for a removed volume (#8735)
* snapshot: don't schedule next snapshot job for a removed volume

When management server starts, it starts the snapshot scheduler. In case
there is a volume snapshot policy which exists for a volume which does
not exist, it can cause SQL constraint issue and cause the management
server to break from starting its various components and cause HTTP 503
error.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* remove schedule on missing volume

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-03-19 09:01:58 -03:00
Vishesh
fe5d741236
Merge branch '4.18' into '4.19' 2024-03-19 13:59:29 +05:30
Vishesh
0043540fa3
Use join instead of views (#8321) 2024-03-18 18:08:19 +01:00
Abhishek Kumar
ffd59720dd
storage,plugins: delegate allow zone-wide volume migration check and access grant check to storage drivers (#8762)
* storage,plugins: delegate allow zone-wide volume migration check and access grant to storage drivers

Following checks have been delegated to storage drivers,
- For volumes on zone-wide storage, whether they need storage migration when VM is migrated
- Whther volume required grant access

Apply fixes in resolving PrimaryDataStore

* add tests

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* unused import

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Update engine/orchestration/src/test/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestratorTest.java

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-03-18 17:28:14 +05:30
Wei Zhou
9d748f0837
server: update default value of setting 'expose.dns.externally' to 'false' (#8776) 2024-03-15 10:43:16 -03:00
Bernardo De Marco Gonçalves
e0731c5098
Fix the removal of network offering tags (#8758) 2024-03-15 13:59:19 +01:00
Nicolas Vazquez
f25d35c5d6
VM Import: Use datastore name when datastore path is not set to search for pools on DB (#8759) 2024-03-14 22:07:23 -03:00
Wei Zhou
e86363be8e Merge remote-tracking branch 'apache/4.18' into 4.19 2024-03-04 09:06:53 +01:00
Wei Zhou
10f72a17f6
server: use ecdsa instead of ed25519 when generate public/private keys (#8719)
PR #8549 replaced RSA with ed25519. unfornately, ed25519 is unsupported in FIPS mode

```
$  ssh-keygen -t ed25519 -m PEM -N '' -f key1
ED25519 keys are not allowed in FIPS mode

$  ssh-keygen -t ecdsa -m PEM -N '' -f key1
Generating public/private ecdsa key pair.
Your identification has been saved in key1
Your public key has been saved in key1.pub
The key fingerprint is:
.........

```
2024-03-03 15:56:13 +01:00
Suresh Kumar Anaparti
813d53d031
Sync the pool stats in DB with the actual stats from stats collector (#8686) 2024-02-29 15:26:32 +05:30
Harikrishna
c462be1412
New API "checkVolume" to check and repair any leaks or issues reported by qemu-img check (#8577)
* Introduced a new API checkVolumeAndRepair that allows users or admins to check and repair if any leaks observed.
Currently this is supported only for KVM

* some fixes

* Added unit tests

* addressed review comments

* add repair volume while granting access

* Changed repair parameter to accept both leaks/all

* Introduced new global setting volume.check.and.repair.before.use to do volume check and repair before VM start or volume attach operations

* Added volume check and repair changes only during VM start and volume attach operations

* Refactored the names to look similar across the code

* Some code fixes

* remove unused code

* Renamed repair values

* Fixed unit tests

* changed version

* Address review comments

* Code refactored

* used volume name in logs

* Changed the API to Async and the setting scope to storage pool

* Fixed exit value handling with check volume command

* Fixed storage scope to the setting

* Fix volume format issues

* Refactored the log messages

* Fix formatting
2024-02-29 14:41:49 +05:30
Abhishek Kumar
2a56c61ade Merge remote-tracking branch 'apache/4.18' into 4.19 2024-02-26 12:01:26 +05:30
Vishesh
e09fd2e26b
Fixup: Optimize getRouterHealthChecksConfig method (#8677) 2024-02-21 09:24:11 +01:00
Phsm Qwerty
af942e2260
vpc: optimize createMonitorServiceCommand() execution (#8385)
* Optimize createMonitorServiceCommand() execution.
Avoid an expensive getRouterHealthChecksConfig() execution multiple times during VPC restart.

Fixes #8055

* Move getRouterHealthChecksConfig() outside of loop
2024-02-19 11:07:48 +01:00
Henrique Sato
275abaff6b
Refactor updateDiskOffering API (#8446)
Co-authored-by: Henrique Sato <henrique.sato@scclouds.com.br>
2024-02-19 10:29:45 +01:00
dahn
a0e592e945
prevent nic removal on out of bounds router stop (#8371)
Co-authored-by: Vishesh <vishesh92@gmail.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-02-16 14:33:22 +01:00
GaOrtiga
6f3e4e6302
fix_filter_and_pagination (#8306)
Co-authored-by: Gabriel <gabriel.fernandes@scclouds.com.br>
2024-02-16 11:15:55 +01:00
Rohit Yadav
bda49ab08f Merge remote-tracking branch 'shapeblue/merged-4-18' into 4.19 2024-02-13 12:54:24 +05:30
Vishesh
a8028eecbd
Merge remote-tracking branch 'origin/4.18' into 4.19 2024-02-13 11:44:20 +05:30
Vishesh
1955d8f3db
Add advance settings to fine tune DRS imbalance calculation (#8521)
* Use free/total instead of free metric to calculate imbalance

* Filter out hosts for condensed while checking imbalance

* Make DRS more configurable

* code refactor

* Add unit tests

* fixup

* Fix validation for drs.imbalance.condensed.skip.threshold

* Add logging and other minor changes for drs

* Add some logging for drs

* Change format for drs imbalance to string

* Show drs imbalance as percentage

* Fixup label for memorytotal in en.json
2024-02-13 11:18:53 +05:30
Wei Zhou
e47a910019
VR: fix issue between VPC VMs and other Public IPs in the same subnet as additional Public IPs (#8599)
* VR: fix issue between VPC VMs and other Public IPs in the same subnet as additional Public IPs

* Update PR8599: move to VpcVirtualNetworkApplianceManagerImpl
2024-02-13 11:03:40 +05:30
Wei Zhou
a924a110e1
Veeam: revert unnecessary state change if restore backup and attach to another VM (#8614) 2024-02-09 16:29:59 +01:00
Wei Zhou
c795547152 server: fix build error with BackupManagerTest.tryRestoreVMTestRestoreSucceeded
```
[ERROR] Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.025 s <<< FAILURE! - in org.apache.cloudstack.backup.BackupManagerTest
[ERROR] tryRestoreVMTestRestoreSucceeded(org.apache.cloudstack.backup.BackupManagerTest)  Time elapsed: 0.469 s  <<< ERROR!
com.cloud.utils.exception.CloudRuntimeException: Unable to change state of volume [Mock for VolumeVO, hashCode: 220689785] to [Ready].
```
2024-02-05 13:24:04 +01:00
Wei Zhou
b8904f75dd Merge remote-tracking branch 'apache/4.18' into 4.19 2024-02-05 10:08:31 +01:00