37049 Commits

Author SHA1 Message Date
Pearl Dsilva
fe5c9d72de add APIs for management of backup repositories and backing up from local stores and stopped VMs 2024-08-12 12:59:10 -04:00
Pearl Dsilva
67821d7e4f Merge branch 'main' of https://github.com/apache/cloudstack into kvm-backup-plugin-shared-storage 2024-08-06 17:41:20 -04:00
nvazquez
47a6b7011d
Fix build 2024-08-06 13:03:31 -03:00
nvazquez
b73f634ea6
Merge branch '4.19' 2024-08-06 12:39:13 -03:00
Abhishek Kumar
6fbdda951a
[4.19] server, api, ui: access improvements and assorted fixes (#22)
* server, api, ui: access improvements and assorted fixes

Fixes domain-admin access check to prevent unauthorized access.

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* Revert "server: refactor listNetworks api database retrievals (#9184)"

This reverts commit c7f1ba5b8eec03d51bfe0f1432e46c9101b0f940.

* Fix snapshot chain being deleted on XenServer (#9447)

Using XenServer as the hypervisor, when deleting a snapshot that has a parent, that parent will also get erased on storage, causing data loss. This behavior was introduced with #7873, where the list of snapshot states that can be deleted was changed to add BackedUp snapshots.

This PR changes the states list back to the original list, and swaps the while loop for a do while loop to account for the changes in #7873.

Fixes #9446

* UI: Display Firewall, LB and Port Forwading rules tab for CKS clusters deployed on isolated networks (#9458)

---------

Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
Co-authored-by: Pearl Dsilva <pearl1594@gmail.com>
2024-08-06 12:36:29 -03:00
nvazquez
d541e9014d
Merge branch '4.18' into 4.19 2024-08-06 12:29:18 -03:00
Abhishek Kumar
2e0024e216
server, api: account and api entity access improvements
Fixes domain-admin access check to prevent unauthorized access.
Introduces a new non-dynamic global setting - api.allow.internal.db.ids
to control whether to allow using internal DB IDs as API parameters or
not. Default value for the global setting is false.

Co-authored-by: Fabricio Duarte <fabricio.duarte.jr@gmail.com>
Co-authored-by: nvazquez <nicovazquez90@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-08-06 12:24:19 -03:00
Rohit Yadav
452bc109f0 fix listbackups to return their original offering
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-02 17:46:54 +05:30
Rohit Yadav
7db5178e66 ui tweaks 2024-08-02 15:31:06 +05:30
Rohit Yadav
00b22d462b fix backup script
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-02 14:27:40 +05:30
Rohit Yadav
0b9db75c4f complete poc
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-02 13:38:54 +05:30
Rohit Yadav
97ce34ce2b introduce backup repository concept for NAS & other backups
Each backup repository is for a specific zone & provider.
Eack NAS bkcp offering is tied to a back repo & each bkup offer assinged
to VM -> defined its backup repository

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-02 12:58:40 +05:30
Rohit Yadav
18e05894c7 add todos
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 18:15:11 +05:30
Rohit Yadav
11cbea2b15 fix op typo
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 18:05:21 +05:30
Rohit Yadav
9b0be25a5c fix nasbackup script
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 17:55:49 +05:30
Rohit Yadav
50e407c1fa implement backup delete
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 17:55:19 +05:30
João Jandre
9033ab709e
Fix snapshot chain being deleted on XenServer (#9447)
Using XenServer as the hypervisor, when deleting a snapshot that has a parent, that parent will also get erased on storage, causing data loss. This behavior was introduced with #7873, where the list of snapshot states that can be deleted was changed to add BackedUp snapshots.

This PR changes the states list back to the original list, and swaps the while loop for a do while loop to account for the changes in #7873.

Fixes #9446
2024-08-01 17:33:04 +05:30
Rohit Yadav
e066c5a919 fix command execution handling
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 16:40:35 +05:30
Rohit Yadav
1933f9e3ee fix backup script
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 16:03:21 +05:30
Rohit Yadav
f13769d33d prototype backup e2e
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 14:31:13 +05:30
Rohit Yadav
f3bc03955a barebone take backup implementation
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 13:17:11 +05:30
Rohit Yadav
d5503e7735 add cmd-answer classes
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-08-01 13:16:52 +05:30
Rohit Yadav
aa90b6de14 Merge remote-tracking branch 'origin/main' into kvm-backup-plugin-shared-storage 2024-07-31 10:24:27 +05:30
Vishesh
5baac44139
ui: add UI too to view and download usage records (#8615)
This PR adds a new UI tool for admins for viewing and downloading usage records.

This PR also makes startdate and enddate as non required params for generateUsageRecords. (Fixes: #7133)
2024-07-30 14:38:17 +05:30
Rohit Yadav
7214c13e10
packaging: bundle latest cmk x86 build with deb and rpm packages (#9450)
This uses the latest tag to find and use the latest Github tag from
cloudstack-cloudmonkey repo, and use that to bundle the binary CLI in
packages.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Wei Zhou <weizhou@apache.org>
2024-07-30 10:01:39 +05:30
Abhishek Kumar
cd8442a42b
test: improve purge expunged resources b/g task testcase (#9426)
* test: improve purge expunged resources b/g task testcase

Failures were seen for during purging of expunged resources via
bacground task during different test runs. This PR tries to make sure
b/g task execution is not skipped after MS restrat in a multi-MS
environment. It also updates expunged.resources.purge.interval to allow
running task again in 60s.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

* new string formatting

---------

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-29 15:49:45 +05:30
Vishesh
f0ba90568d
api: Fix missing keys in listZonesMetrics response (#9419)
This PR fixes #9379
Fix missing keys in listZonesMetrics.

We use BeanUtils.copyProperties to copy the properties of ZoneResponse to ZoneMetricsResponse. Fields with missing getters are skipped from the response because of this.
2024-07-29 10:54:54 +05:30
Abhishek Kumar
d606ee7a07
ui: vm metrics note about behaviour across hypervisors (#9399)
The metrics for VMs and VM disks in CloudStack depend on the hypervisor plugin used for each hypervisor. The behavior can vary across different hypervisors. For VMware, metrics are retrieved as average value for the time interval defined in the global configuration - vmware.stats.time.window.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-29 10:52:31 +05:30
Vishesh
0be87ac654
Fixup CKS UI for external managed clusters (#9434) 2024-07-29 10:48:58 +05:30
Rohit Yadav
7f4a6861b3 Merge remote-tracking branch 'origin/4.19'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-29 09:46:43 +05:30
Rohit Yadav
3d32ab5a49 Merge remote-tracking branch 'origin/4.18' into 4.19 2024-07-29 09:42:38 +05:30
Pearl Dsilva
cf0e44dd7e
UI: Display Firewall, LB and Port Forwading rules tab for CKS clusters deployed on isolated networks (#9458) 2024-07-29 09:35:55 +05:30
Rohit Yadav
f358813e1d Merge remote-tracking branch 'origin/4.19' into main
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-26 12:32:40 +00:00
Vishesh
33be0ae745
Fix removal of usage records (#9442) 2024-07-26 11:40:02 +03:00
Rohit Yadav
f8385a48da
Add systemvmtemplate arm64 build support (#9437)
* Move and copy files to new filenames

* Adjust bash scripts to support arm64

* change preseed for arm64 to create EFI partition

* change packer template files to support arm64

* Correct preseed file name in x86_64 packer template

* fixes

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* try different machine cpu

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* don't accelerate KVM for different arch (arm64 vm on amd64 host)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix building arm64 on x86

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update README.md

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: StepBee <stephan@bienek.org>
2024-07-26 13:38:03 +05:30
Rohit Yadav
9a8bbbbc08 WIP B&R tool
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-26 13:37:37 +05:30
Rohit Yadav
b961a4d70b cleanup, refactor; add plugin config keys
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-26 13:25:58 +05:30
Rohit Yadav
2604f81264 cleanup and leave TODOs
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-26 13:09:27 +05:30
Rohit Yadav
f2baa68802 backup: Simple NAS backup plugin for KVM
This is an experimental simple NAS backup plugin for KVM which may be
later expanded for other hypervisors. This backup plugin aims to use
shared NAS storage on KVM hosts such as NFS or CephFS, which is used
to backup fully cloned VMs for backup & restore operations. This
may not be as efficient and performant as some of the other B&R
providers, but maybe useful for some KVM environments.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2024-07-26 12:04:40 +05:30
Wei Zhou
1662af131e
[4.20] VR: fix issue if userdata is binary data (#8739) 2024-07-25 10:04:58 +02:00
Lucas Martins
817251f1f8
Enhancement in the accuracy of the logs regarding the capacity, usage, and threshold of secondary storages (#9043)
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-07-24 15:30:40 +02:00
Lucas Martins
dc1556ca38
Change exception when orchestrating VM start (#9062)
* Remove info about the environment from exception when starting VM

* Change to log4j2 syntax

---------

Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br>
2024-07-24 09:18:25 -03:00
Fabricio Duarte
49cd5ba64a
Fix link to removed volumes being shown in info card and list view (#8833)
* Framework for validating links in the front-end

* Rename valid links map in the list view
2024-07-24 09:09:07 -03:00
Abhishek Kumar
d41dc17b2c Merge remote-tracking branch 'apache/4.19' 2024-07-22 11:36:05 +05:30
Abhishek Kumar
54c8b71fb5 Merge remote-tracking branch 'apache/4.18' into 4.19 2024-07-22 11:33:34 +05:30
Rohit Yadav
f24fb20e6b
ui: add new API docs tab (#9409)
* ui: add new API docs tab

This introduces a new API docs table which is enabled by default but
the admin can disable it via config.json. This uses the discovered
APIs for logged in user/account to show them the APIs accessible to them
and generates dynamic API docs based on them which are searchable. Also
introduces some common auto-completed API groups that are available to
most roles.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update ui/src/views/plugins/ApiDocsPlugin.vue

* Update ui/src/views/plugins/ApiDocsPlugin.vue

* Update ui/src/views/plugins/ApiDocsPlugin.vue

* Update ui/src/views/plugins/ApiDocsPlugin.vue

* Update ui/src/views/plugins/ApiDocsPlugin.vue

* fix performance issues

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update ui/src/views/plugins/ApiDocsPlugin.vue

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Update ui/public/locales/en.json

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* address Suresh's feedback

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* filter example/options as we type

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Address Joao's comments

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
2024-07-22 10:46:40 +05:30
Abhishek Kumar
e41add31e7 saml: signature check improvements
Adminstrators should ensure that IDP configuration has a signing certificate for the actual signature check to be performed. In addition to this, this change introduces a new global setting saml2.check.signature, with the default value of true, which can deliberately fail a SAML login attempt when the SAML response has a missing signature.
Purges the SAML token upon handling the first SAML response.

Authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-07-19 14:24:43 +05:30
Suresh Kumar Anaparti
3faf7cd2f1
Updating pom.xml version numbers for release 4.19.2.0-SNAPSHOT
Signed-off-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
2024-07-19 10:29:26 +05:30
Layon
56c661c1df
Icon changed for control-outlined (#9402) 2024-07-17 15:17:35 +02:00
Vishesh
35fd17c62e
Merge branch '4.19' 2024-07-17 17:43:40 +05:30