11762 Commits

Author SHA1 Message Date
Wei Zhou
1605b2f0db
server: remove registered userdata when cleanup an account (#7777) 2023-08-10 14:33:59 +02:00
João Jandre
fdb23dae40
server: Use max secondary storage defined on the account during upload (#7441) 2023-08-10 11:39:40 +02:00
Wei Zhou
a1af03f413
noVNC: fix JP keyboard on vmware7+ which uses websocket URL (#7694)
* noVNC: fix JP keyboard on vmware7+ which uses websocket URL

* noVNC: cleanup rfb.js

* noVNC: fix < and > on JP keyboard

* noVNC: fix Caps lock on JP keyboard
2023-08-09 11:43:16 +02:00
Daan Hoogland
32448e1ac7 Merge branch '4.18' 2023-08-07 11:13:17 +02:00
dahn
dea1373480
server: clean network offerings for domain on remove (#7775) 2023-08-07 09:56:58 +02:00
mprokopchuk
ab0297ea9b
VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted (#7760)
VM.CREATE/VOLUME.DELETE/VOLUME.DESTROY not being emitted

* Update server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

* Update api/src/main/java/org/apache/cloudstack/api/command/user/vm/DeployVMCmd.java

Co-authored-by: dahn <daan.hoogland@gmail.com>

---------

Co-authored-by: Maxim Prokopchuk <mprokopchuk@apple.com>
Co-authored-by: dahn <daan.hoogland@gmail.com>
2023-08-07 10:18:17 +05:30
Daan Hoogland
e3bd2c0a8c Merge branch '4.18' 2023-08-04 09:24:16 +02:00
dahn
b4032d9984
include all VMs when projects selected not just for admins (#7667) 2023-08-03 16:06:11 +02:00
John Bampton
6f4503488b
pre-commit: apply end-of-file-fixer to all files (#7551) 2023-08-02 13:47:21 +02:00
Daan Hoogland
d51d8a4a13 Merge release branch 4.18 to main
* 4.18:
  UI: Filter templates by zone and hypervisor type when reinstall a VM (#7739)
  KVM: fix SSVM starting when overprovisioning memory (#7663)
  pom.xml: add property project.systemvm.template.location (#7706)
  cloudutils: fix adding rocky9 host failure due to missing /etc/sysconfig/libvirtd (#7779)
  server: get id from persisted object ReservationVO (#7785)
  search in (too) large result sets (#7766)
  ui: fix 404 error when list volumes of system vms (#7772)
  packaging: install tzdata-java on centos7/centos8 (#7768)
2023-07-31 09:04:44 +02:00
GaOrtiga
0bab0db881
Introducing concept of domain VPCs (#7153)
Co-authored-by: Gabriel Ortiga Fernandes <gabriel.fernandes@scclouds.com.br>
Co-authored-by: Lopez <rodrigo@scclouds.com.br>
2023-07-29 20:06:49 +02:00
dahn
d127d7939d
KVM: fix SSVM starting when overprovisioning memory (#7663) 2023-07-28 11:23:30 +02:00
dahn
d958882282
server: get id from persisted object ReservationVO (#7785) 2023-07-28 05:16:33 +02:00
Daan Hoogland
8db8aa4163 Merge branch '4.18' 2023-07-25 11:29:04 +02:00
dahn
f9b7bcfd10
add remove tag to remove acl item method (#7750) 2023-07-25 10:55:44 +02:00
Rohit Yadav
bde80f14aa
Fix NPE in management server logs due to /proc/cpuinfo output (#7765)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-07-25 04:43:33 +02:00
Rohit Yadav
62a8f4ef72 Merge remote-tracking branch 'origin/4.18' 2023-07-24 15:57:37 +05:30
Harikrishna
80ca3acf15
Allow encrypted volume migration for PowerFlex volumes (#7757) 2023-07-21 10:08:21 +03:00
Abhishek Kumar
cee7a713aa
server: clear resource reservation and increment resource count in a transaction (#7724)
This PR addresses rare case of potential overlap of resource reservation and resource count.
For different resource types there could be some delay between incrementing of the resource count and clearing of the earlier done reservation. This may result in failures when there are parallel deployments happening.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-21 10:55:51 +05:30
Abhishek Kumar
729e6d1446
userdata: fix append scenarios (#7741)
Fixes case of appending userdata when both template and vm data are either shellscript or cloudconfig
Fixes error when appending gzip userdata
Fixes case when userdata manual text from VM is not getting decoded-encoded correctly.
Fixes case of appending multipart data when both template and vm data contain same format types.
Refactor - moved validateUserData method to UserDataManager class
Refactor userdata test to check resultant multipart userdata thoroughly

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-19 15:18:53 +05:30
Daan Hoogland
6bb95c0200 Merge release branch 4.18 to main
* 4.18:
  Storage and volumes statistics tasks for StorPool primary storage (#7404)
  proper storage construction (#6797)
  guarantee MAC uniqueness (#7634)
  server: allow migration of all VMs with local storage on KVM (#7656)
  Add L2 networks to Zones with SG (#7719)
2023-07-19 10:59:19 +02:00
dahn
73a269e3b3
guarantee MAC uniqueness (#7634)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-19 10:25:01 +02:00
dahn
d575cae3e3
server: allow migration of all VMs with local storage on KVM (#7656)
This PR enables systemvm and VR migration on KVM with local storage.

Fixes: #7651
2023-07-18 12:49:17 +05:30
Ben
981d74825a
Add L2 networks to Zones with SG (#7719) 2023-07-11 09:11:23 +02:00
Nicolas Vazquez
b1fc279872
Generate cloud-init multipart user data for template append policy (#7643)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-10 09:47:03 +02:00
Vishesh
594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Daan Hoogland
2132f46fcb Merge branch '4.18' 2023-07-06 11:24:08 +02:00
Abhishek Kumar
939ee9e153
server,engine-orchestration: allocate vm without transaction (#7695)
When deploying a VM is failed during the allocation process it may leave the resources that have been already allocated before the failure. They will get removed from the database as the whole code block is wrapped inside a transaction twice but the server would not inform the network or storage plugins to clean up the allocated resources.
This PR removes Transactions during VM allocation which results in the allocated VM and its resource records being persisted in DB even during failures. When failure is encountered VM is moved to Error state. This helps VM and its resources to be properly deallocated when it is expunged either by a server task such as ExpungeTask or during manual expunge.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-06 14:04:38 +05:30
Nicolas Vazquez
c733a23c90
Fix direct download URL checks (#7693)
This PR fixes the URL check for direct downloads, in the case of HTTPS URLs the certificates were not loaded into the SSL context
2023-07-06 13:47:13 +05:30
Daan Hoogland
02be2cdf67 Merge release branch 4.18 to main
* 4.18:
  cleanup removal of physical network (#7630)
  console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237)
2023-07-05 11:52:23 +02:00
dahn
8e171785d1
cleanup removal of physical network (#7630)
Co-authored-by: Bryan Lima <42067040+BryanMLima@users.noreply.github.com>
2023-07-05 11:12:46 +02:00
Wei Zhou
3e04779f60
console proxy: use AeadBase64Encryptor instead of AES/CBC/PKCS5Padding (#7237) 2023-07-05 11:01:32 +02:00
Daan Hoogland
fbb9dec6e5 Merge release branch 4.18 to main
* 4.18:
  server: do not check zone imagestores for directdownload template delete (#7607)
  UI: Zone wizard fix (#7588)
2023-07-04 10:17:43 +02:00
Abhishek Kumar
5e5d194d77
server: do not check zone imagestores for directdownload template delete (#7607)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-04 09:37:06 +02:00
mprokopchuk
70d5470f48
If ResourceCountCheckTask throws an exception the scheduled task is not going to run again until the management servers are restarted. (#7670)
Co-authored-by: Maxim Prokopchuk <mprokopchuk@apple.com>
2023-07-04 08:45:15 +02:00
dahn
1275db4081
UI: Zone wizard fix (#7588)
Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 11:56:52 +02:00
Rohit Yadav
3b054b2665 Merge remote-tracking branch 'origin/4.18' 2023-07-03 13:17:25 +05:30
Abhishek Kumar
de6ce503dc
api: correct error on resize volume resource allocation failure (#7687)
This PR resource throws exception with the correct error code and logs the error message when a resource allocation failure is encountered during resize volume operation.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-07-03 12:57:14 +05:30
Daniel Augusto Veronezi Salvador
c6237c48ac
Retrieve only resource count on user dashboard (#7617)
* Allow retrieving only the count of resources on APIs listPublicIpAddresses, listNetworks, listVirtualMachines and listVolumes

* Use parameter to retrieve only the count of resources in the dashboard

* Create abstract class
2023-06-30 11:09:55 -03:00
Wei Zhou
41403c9cf9 Merge remote-tracking branch 'apache/4.18' 2023-06-27 11:52:37 +02:00
dahn
06caf32bc8
Guest os mappings improvements (#6979)
* Guest OS mapping improvements
- Checks the OS mapping name in hypervisor (VMware, XenServer)
- Displays guest OS mappings in UI

* Added API getHypervisorGuestOsNames to list the guest OS names in the hypervisor, and code improvements

* Some static analysis fixes

* Removed commented code in listview

* Guest OS list

* UI changes for adding guest os and mappings

* Added guest os mappings in guest os form

* Added new filter to guest os mapping

* Name and description changes

* VMWare Host and cluster MO unit tests

* CheckGuestOsMapping command and answer unit tests

* GetHypervisorGuestOsNames command and answer unit tests

* VmwareResource unitests

* GuestOsMapper unittests

* icon changes

* Addressed review comments

* Renaming fixes

* Removed comments

* marvin tests for guest os operations

* Added marvin tests for OS mappings

* Document links and UI improvements

* Added deduplication for the list guest OS API

* Fixed linter failure

* Few bug fixes and UI changes

* Few improvements

* Addressed code smells

* Fixed UI issues after rebase

---------

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>
Co-authored-by: Harikrishna Patnala <harikrishna.patnala@gmail.com>
2023-06-27 13:49:20 +05:30
slavkap
faaf72b1a4
Volume encryption support for StorPool plug-in (#7539)
Supported Virtual machine operations:
- live migration of VM to another host
- virtual machine snapshots (group snapshot without memory)
- revert VM snapshot
- delete VM snapshot
Supported Volume operations:
- attach/detach volume
- live migrate volume between two StorPool primary storages
- volume snapshot
- delete snapshot
- revert snapshot
2023-06-26 11:24:51 +02:00
Vishesh
fa3f2a75eb
feature: VM Scheduler (#7397)
This PR adds a feature to setup schedules to stop/start/restart your VMs.

Documentation PR: apache/cloudstack-documentation#313
Related issue: #3387
2023-06-26 13:36:46 +05:30
Nicolas Vazquez
c809201247
Fix: Volumes on lost local storage cannot be removed (#7594) 2023-06-23 12:22:15 +02:00
Vishesh
0acc66f51d
server: Add check on host's status while deleting config drive on host cache (#7584)
This PR adds a check on host's status. Without this if the agent is not in Up or Connecting state, expunging of a VM fails.

Steps to reproduce:
- Enable vm.configdrive.force.host.cache.use in Global Configuration.
- Create a L2 network with config drive
- Deploy a vm with the L2 network created in previous step
- Stop the vm and destroy vm (not expunge it)
- Stop the cloudstack-agent on the VM's host
- Expunge the vm

Fixes: #7428
2023-06-23 13:46:22 +05:30
Abhishek Kumar
0466bf9d47
server,api,ui: host auto-select for migrateVirtualMachineWithVolume (#7554)
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2023-06-22 14:53:51 +02:00
Wei Zhou
09a4a252d7 Merge remote-tracking branch 'apache/4.18' into HEAD 2023-06-21 15:08:56 +02:00
Harikrishna
40cc10a73d
Allow volume migrations in ScaleIO within and across ScaleIO storage clusters (#7408)
* Live storage migration of volume in scaleIO within same storage scaleio cluster

* Added migrate command

* Recent changes of migration across clusters

* Fixed uuid

* recent changes

* Pivot changes

* working blockcopy api in libvirt

* Checking block copy status

* Formatting code

* Fixed failures

* code refactoring and some changes

* Removed unused methods

* removed unused imports

* Unit tests to check if volume belongs to same or different storage scaleio cluster

* Unit tests for volume livemigration in ScaleIOPrimaryDataStoreDriver

* Fixed offline volume migration case and allowed encrypted volume migration

* Added more integration tests

* Support for migration of encrypted volumes across different scaleio clusters

* Fix UI notifications for migrate volume

* Data volume offline migration: save encryption details to destination volume entry

* Offline storage migration for scaleio encrypted volumes

* Allow multiple Volumes to be migrated with migrateVirtualMachineWithVolume API

* Removed unused unittests

* Removed duplicate keys in migrate volume vue file

* Fix Unit tests

* Add volume secrets if does not exists during volume migrations. secrets are getting cleared on package upgrades.

* Fix secret UUID for encrypted volume migration

* Added a null check for secret before removing

* Added more unit tests

* Fixed passphrase check

* Add image options to the encypted volume conversion
2023-06-21 11:57:05 +05:30
dahn
1aa4f80741
accept first word from host os string for backwards compatibility (#7620) 2023-06-15 10:09:31 +03:00
dahn
ae10263b3b
Possibility to choose the source NAT IP address on a isolated network or VPC (#6442)
Co-authored-by: NuxRo <nux@li.nux.ro>
Co-authored-by: Daniel Augusto Veronezi Salvador <38945620+GutoVeronezi@users.noreply.github.com>
2023-06-09 14:51:53 +02:00