mirror of
https://github.com/apache/cloudstack.git
synced 2025-10-26 08:42:29 +01:00
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Abhisar Sinha
|
c5afee2101
|
UI improvements (#9773)
* Show Usage Server configuration in a separate pane * UI: Option to attach volume to an instance during create volume * Show service ip in management server details tab * change Schedule Snapshots to Recurring Snapshots * Change the hypervisor order so that kvm, vmware, xenserver show up first * Remove extra space in hypervisor names in config.java * Fix `updateTemplatePermission` when the UI is set to a language other than English (#9766) * Fix updateTemplatePermission UI in non-english language * Improve fix --------- Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br> * Autofill vcenter details in add cluster form * UI: condition to display create vm-vol-snapshots to same as create vol-snapshots * Fix alignment on wrapping in global settings tabs * rename Autofill vCenter credentials to Autofill vCenter credentials from Zone * Rename Service Ip to Ip Address in management server response * Change description of kvm.snapshot.enabled to say that it applies to volume snapshots * Return error when kvm vm snapshot is taken withoutsnapshot memory * Minor naming changes and grammar * Fix tooltip for attach volume to instance button * Show Usage Server configuration in a separate pane * UI: Option to attach volume to an instance during create volume * Show service ip in management server details tab * change Schedule Snapshots to Recurring Snapshots * Change the hypervisor order so that kvm, vmware, xenserver show up first * Remove extra space in hypervisor names in config.java * Autofill vcenter details in add cluster form * UI: condition to display create vm-vol-snapshots to same as create vol-snapshots * Fix alignment on wrapping in global settings tabs * rename Autofill vCenter credentials to Autofill vCenter credentials from Zone * Rename Service Ip to Ip Address in management server response * Change description of kvm.snapshot.enabled to say that it applies to volume snapshots * Return error when kvm vm snapshot is taken withoutsnapshot memory * Minor naming changes and grammar * Fix tooltip for attach volume to instance button * Show Usage Server configuration in a separate pane * UI: Option to attach volume to an instance during create volume * Show service ip in management server details tab * change Schedule Snapshots to Recurring Snapshots * Change the hypervisor order so that kvm, vmware, xenserver show up first * Remove extra space in hypervisor names in config.java * Autofill vcenter details in add cluster form * UI: condition to display create vm-vol-snapshots to same as create vol-snapshots * Fix alignment on wrapping in global settings tabs * rename Autofill vCenter credentials to Autofill vCenter credentials from Zone * Rename Service Ip to Ip Address in management server response * Change description of kvm.snapshot.enabled to say that it applies to volume snapshots * Return error when kvm vm snapshot is taken withoutsnapshot memory * Minor naming changes and grammar * Fix tooltip for attach volume to instance button * UI: Option to attach volume to an instance during create volume * UI: condition to display create vm-vol-snapshots to same as create vol-snapshots * moved db changes from 41900to42000 to 42000to42010 * Update group_id in already present usage configuration settings * remove "schedule" from message in create Recurring Snapshots form * Update server/src/main/java/com/cloud/vm/snapshot/VMSnapshotManagerImpl.java --------- Co-authored-by: Daan Hoogland <daan@onecht.net> Co-authored-by: Lucas Martins <56271185+lucas-a-martins@users.noreply.github.com> Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br> Co-authored-by: Boris Stoyanov - a.k.a Bobby <bss.stoyanov@gmail.com> Co-authored-by: Andrija Panic <45762285+andrijapanicsb@users.noreply.github.com> |
||
Vishesh
|
a4224e58cc
|
Improve logging to include more identifiable information (#9873)
* Improve logging to include more identifiable information for kvm plugin * Update logging for scaleio plugin * Improve logging to include more identifiable information for default volume storage plugin * Improve logging to include more identifiable information for agent managers * Improve logging to include more identifiable information for Listeners * Replace ids with objects or uuids * Improve logging to include more identifiable information for engine * Improve logging to include more identifiable information for server * Fixups in engine * Improve logging to include more identifiable information for plugins * Improve logging to include more identifiable information for Cmd classes * Fix toString method for StorageFilterTO.java |
||
Lucas Martins
|
5886780240
|
Change vmsnapshot.max config to be dynamic (#9883)
Co-authored-by: Lucas Martins <lucas.martins@scclouds.com.br> |
||
|
Abhisar Sinha
|
a88967bf61
|
Minor naming changes in Shared FileSystems 4.20 Feature (#9675) | ||
|
Abhisar Sinha
|
605534b417
|
feature: Shared Storage Filesystem as a First Class Feature (#9208)
This PR implements Storage filesystem as a first class feature. https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Filesystem+as+a+First+Class+Feature Documentation PR: apache/cloudstack-documentation#420 Co-authored-by: Wei Zhou <weizhou@apache.org> |
||
João Jandre
|
49cecaed06
|
Normalize loggers and upgrade log4j 1.2 to log4j 2.19 (#7131)
* Normalize logs All classes that could have their loggers inherited from their fathers had their own loggers deleted; Most loggers didn't have to be static, so most of them were normalized so that they wouldn't be; All loggers are protected now; Static logger's name are now 'LOGGER'; Non-static logger's name are now 'logger'; New class DbUpgradeAbstractImpl created so that all Upgraders extend it and inherit its logger * Upgrade log4j * fix errors caused by the merge * Refactor cglibThrowableRenderer functionality to log4j2 and upgrade the last configuration files * fix sonarcloud bug * Fix errors caused by merge, remove some unused loggers, and rename a variable that was mistakenly renamed on the normalization commit * Readd snmpTrapAppender, remove TestAppender * Regenerate changes * regenerate changes * refactor last custom appender * fix systemvm configuration xml * Regenerate changes * Regenerate changes * regenerate changes * Regenerate changes * regenerate changes * regenerate changes * regenerate changes * Fix utils pom * fix some tests * regenerate changes * Fix jar being printed on exception * fix logging in system VMs, fix commands not having log4j2 classpath. * regenerate changes * Fix some unwanted renomeations * fix end of file * regenerate changes * regenerate changes * fix merge error * regenerate changes * fix tests * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * readd reload4j to tungsten as juniper depends on it * Regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * re-add reload4j dependency to network-contrail, as juniper depends on it * regenerate changes * regenerate changes * regenerate changes * fix typo * regenerate changes * regenerate changes * Fix end of files * regenerate changes * add logj42 to cloud-utils-SHADED.jar * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * regenerate changes * Regenerate changes * Regenerate changes * Regenerate changes * regenerate changes * Regenerate changes * regenerate changes * Regenerate changes * Regenerate changes * Regenerate changes * regenerate changes * Regenerate changes * Regenerate changes * fix some tests * Regenerate changes * Regenerate changes * fix test * Regenerate changes * Regenerate changes |
||
Marcus Sorensen
|
697e12f8f7
|
kvm: volume encryption feature (#6522)
This PR introduces a feature designed to allow CloudStack to manage a generic volume encryption setting. The encryption is handled transparently to the guest OS, and is intended to handle VM guest data encryption at rest and possibly over the wire, though the actual encryption implementation is up to the primary storage driver. In some cases cloud customers may still prefer to maintain their own guest-level volume encryption, if they don't trust the cloud provider. However, for private cloud cases this greatly simplifies the guest OS experience in terms of running volume encryption for guests without the user having to manage keys, deal with key servers and guest booting being dependent on network connectivity to them (i.e. Tang), etc, especially in cases where users are attaching/detaching data disks and moving them between VMs occasionally. The feature can be thought of as having two parts - the API/control plane (which includes scheduling aspects), and the storage driver implementation. This initial PR adds the encryption setting to disk offerings and service offerings (for root volume), and implements encryption support for KVM SharedMountPoint, NFS, Local, and ScaleIO storage pools. NOTE: While not required, operations can be significantly sped up by ensuring that hosts have the `rng-tools` package and service installed and running on the management server and hypervisors. For EL hosts the service is `rngd` and for Debian it is `rng-tools`. In particular, the use of SecureRandom for generating volume passphrases can be slow if there isn't a good source of entropy. This could affect testing and build environments, and otherwise would only affect users who actually use the encryption feature. If you find tests or volume creates blocking on encryption, check this first. ### Management Server ##### API * createDiskOffering now has an 'encrypt' Boolean * createServiceOffering now has an 'encryptroot' Boolean. The 'root' suffix is added here in case there is ever any other need to encrypt something related to the guest configuration, like the RAM of a VM. This has been refactored to deal with the new separation of service offering from disk offering internally. * listDiskOfferings shows encryption support on each offering, and has an encrypt boolean to choose to list only offerings that do or do not support encryption * listServiceOfferings shows encryption support on each offering, and has an encrypt boolean to choose to list only offerings that do or do not support encryption * listHosts now shows encryption support of each hypervisor host via `encryptionsupported` * Volumes themselves don't show encryption on/off, rather the offering should be referenced. This follows the same pattern as other disk offering based settings such as the IOPS of the volume. ##### Volume functions A decent effort has been made to ensure that the most common volume functions have either been cleanly supported or blocked. However, for the first release it is advised to mark this feature as *experimental*, as the code base is complex and there are certainly edge cases to be found. Many of these features could eventually be supported over time, such as creating templates from encrypted volumes, but the effort and size of the change is already overwhelming. Supported functions: * Data Volume create * VM root volume create * VM root volume reinstall * Offline volume snapshot/restore * Migration of VM with storage (e.g. local storage VM migration) * Resize volume * Detach/attach volume Blocked functions: * Online volume snapshot * VM snapshot w/memory * Scheduled snapshots (would fail when VM is running) * Disk offering migration to offerings that don't have matching encryption * Creating template from encrypted volume * Creating volume from encrypted volume * Volume extraction (would we decrypt it first, or expose the key? Probably the former). ##### Primary Storage Support For storage developers, adding encryption support involves: 1. Updating the `StoragePoolType` for your primary storage to advertise encryption support. This is used during allocation of storage to match storage types that support encryption to storage that supports it. 2. Implementing encryption feature when your `PrimaryDataStoreDriver` is called to perform volume lifecycle functions on volumes that are requesting encryption. You are free to do what your storage supports - this could be as simple as calling a storage API with the right flag when creating a volume. Or (as is the case with the KVM storage types), as complex as managing volume details directly at the hypervisor host. The data objects passed to the storage driver will contain volume passphrases, if encryption is requested. ##### Scheduling For the KVM implementations specified above, we are dependent on the KVM hosts having support for volume encryption tools. As such, the hosts `StartupRoutingCommand` has been modified to advertise whether the host supports encryption. This is done via a probe during agent startup to look for functioning `cryptsetup` and support in `qemu-img`. This is also visible via the listHosts API and the host details in the UI. This was patterned after other features that require hypervisor support such as UEFI. The `EndPointSelector` interface and `DefaultEndpointSelector` have had new methods added, which allow the caller to ask for endpoints that support encryption. This can be used by storage drivers to find the proper hosts to send storage commands that involve encryption. Not all volume activities will require a host to support encryption (for example a snapshot backup is a simple file copy), and this is the reason why the interface has been modified to allow for the storage driver to decide, rather than just passing the data objects to the EndpointSelector and letting the implementation decide. VM scheduling has also been modified. When a VM start is requested, if any volume that requires encryption is attached, it will filter out hosts that don't support encryption. ##### DB Changes A volume whose disk offering enables encryption will get a passphrase generated for it before its first use. This is stored in the new 'passphrase' table, and is encrypted using the CloudStack installation's standard configured DB encryption. A field has been added to the volumes table, referencing this passphrase, and a foreign key added to ensure passphrases that are referenced can't be removed from the database. The volumes table now also contains an encryption format field, which is set by the implementer of the encryption and used as it sees fit. #### KVM Agent For the KVM storage pool types supported, the encryption has been implemented at Qemu itself, using the built-in LUKS storage support. This means that the storage remains encrypted all the way to the VM process, and decrypted before the block device is visible to the guest. This may not be necessary in order to implement encryption for /your/ storage pool type, maybe you have a kernel driver that decrypts before the block device on the system, or something like that. However, it seemed like the simplest, common place to terminate the encryption, and provides the lowest surface area for decrypted guest data. For qcow2 based storage, `qemu-img` is used to set up a qcow2 file with LUKS encryption. For block based (currently just ScaleIO storage), the `cryptsetup` utility is used to format the block device as LUKS for data disks, but `qemu-img` and its LUKS support is used for template copy. Any volume that requires encryption will contain a passphrase ID as a byte array when handed down to the KVM agent. Care has been taken to ensure this doesn't get logged, and it is cleared after use in attempt to avoid exposing it before garbage collection occurs. On the agent side, this passphrase is used in two ways: 1. In cases where the volume experiences some libvirt interaction it is loaded into libvirt as an ephemeral, private secret and then referenced by secret UUID in any libvirt XML. This applies to things like VM startup, migration preparation, etc. 2. In cases where `qemu-img` needs to use this passphrase for volume operations, it is written to a `KeyFile` on the cloudstack agent's configured tmpfs and passed along. The `KeyFile` is a `Closeable` and when it is closed, it is deleted. This allows us to try-with-resources any volume operations and get the KeyFile removed regardless. In order to support the advanced syntax required to handle encryption and passphrases with `qemu-img`, the `QemuImg` utility has been modified to support the new `--object` and `--image-opts` flags. These are modeled as `QemuObject` and `QemuImageOptions`. These `qemu-img` flags have been designed to supersede some of the existing, older flags being used today (such as choosing file formats and paths), and an effort could be made to switch over to these wholesale. However, for now we have instead opted to keep existing functions and do some wrapping to ensure backward compatibility, so callers of `QemuImg` can choose to use either way. It should be noted that there are also a few different Enums that represent the encryption format for various purposes. While these are analogous in principle, they represent different things and should not be confused. For example, the supported encryption format strings for the `cryptsetup` utility has `LuksType.LUKS` while `QemuImg` has a `QemuImg.PhysicalDiskFormat.LUKS`. Some additional effort could potentially be made to support advanced encryption configurations, such as choosing between LUKS1 and LUKS2 or changing cipher details. These may require changes all the way up through the control plane. However, in practice Libvirt and Qemu currently only support LUKS1 today. Additionally, the cipher details aren't required in order to use an encrypted volume, as they're stored in the LUKS header on the volume there is no need to store these elsewhere. As such, we need only set the one encryption format upon volume creation, which is persisted in the volumes table and then available later as needed. In the future when LUKS2 is standard and fully supported, we could move to it as the default and old volumes will still reference LUKS1 and have the headers on-disk to ensure they remain usable. We could also possibly support an automatic upgrade of the headers down the road, or a volume migration mechanism. Every version of cryptsetup and qemu-img tested on variants of EL7 and Ubuntu that support encryption use the XTS-AES 256 cipher, which is the leading industry standard and widely used cipher today (e.g. BitLocker and FileVault). Signed-off-by: Marcus Sorensen <mls@apple.com> Co-authored-by: Marcus Sorensen <mls@apple.com> |
||
Nicolas Vazquez
|
ea37d757d3
|
Reword KVM snapshot without memory error message (#6387) | ||
Abhishek Kumar
|
523805c8bc
|
schema,server,api: events improvement (#5997)
* schema,server,api: events improvement Add resource ID and resource type to event. Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wip Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * refactor resourcetype association with API class Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add resource anme to the response Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * more tests Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * new line Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add resource test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * smoke test for events resource Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui improvements Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * refactor Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * api,ui: add support for listing events for a resource Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * since key Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * tests and permission changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * missing test Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * events for domain Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * improvements Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add missing license Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * ui: fix js console errors Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * sort enumeration Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix event resource for vpc Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * feedback changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix order Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * events with parent resource Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * missing UI labels Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * donot call cmd resource methods before dispatch Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add restore vm to procedure Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add missing imports Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * resource details for more events Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * add test for changes Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * more test, license fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * wrong merge fix Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> * fix for more event types Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> |
||
slavkap
|
2b075ed39e
|
Storage-based Snapshots for KVM VMs (#3724)
* VM snapshots of running KVM instance using storage providers plugins for disk snapshots Added new virtual machine snapshot strategy which is using storage providers plugins to take/revert/delete snapshots. You can take VM snapshot without VM memory on KVM instance, using storage providers implementations for disk snapshots. Also revert and delete is added as functionality. Added Thaw/Freeze command for KVM instance. The snapshots will be consistent, because we freeze the VM during the snapshotting. Backup to secondary storage is executed after thaw of the VM and if it is enabled in global settings. * Removed duplicated functionality Set few methods in DefaultVMSnapshotStrategy to protected to reuse them without duplicating the code. Remove code that is actualy not needed * Added requirements in global setting kvm.vmstoragesnapshot.enabled Added more information in kvm.vmstoragesnapshot.enabled global setting, that it needs installation of: - qemu version 1.6+ - qemu-guest-agent installed on guest virtual machine when the option is enabled * Added Apache license header * Removed commented code * If "kvm.vmstoragesnapshot.enabled" is null should be considered as false * removed unused imports, replaced default template Removed unused imports which causing failures and replaced template to CentOS8 * "kvm.vmstoragesnapshot.enabled" set to dynamic * Getting status of freeze/thaw commands not the return code Will chacke the status if freeze/thaw of Guest VM succeded, rather than looking for return code. Code refactoring * removed "CreatingKVM" VMsnapshot state and events related to it * renamed AllocatedKVM to AllocatedVM the states should not be associated to a hypervisor type * loggin the result of "drive-backup" command * Check which VM snapshot strategy could handle the vm snapshots gets the best match of VM snapshot strategy which could handle the vm snapshots on KVM. Other storage plugins could integrate with this functionality to support group snapshots * Added poolId in canHandle for KVM hypervisors Added poolId into canHandle method used to check if all volumes are on the same PowerFlex's storage pool * skip smoke tests if the hypervisor's OS type is CentOS This PR works with functionality included in qemu-kvm-ev which does not come by default on CentOS. The smoke tests will be skipped if the hypervisor OS is CentOS * Added missed import in smoke test * Suggested change to use ` org.apache.commons.lang.StringUtils.isNotBlank` * Fix getting device on Ubuntu On Ubuntu the device isn't provided and we have to get it from node-name parameter. For drive-backup command (for Ubuntu) is needed and job-id which is the value of node-name (this extra param works on Ubuntu and CentOS as well). * Removed new snapshot states and functionality for NFS * throw CloudRuntimeException provide a properer error message when delete VM snapshot fails * exclude GROUP snapshots when listing snapshots * Skip tests if there is pool with NFS/Local * address comments |
||
|
Nicolas Vazquez
|
3fc31ca1e5
|
Revert "Honour isrecursive above listall (#6045)" (#6126)
This reverts commit 71056191f2bdad4be1a7eaf9bb73a7dcee3516f2. |
||
David Jumani
|
71056191f2
|
Honour isrecursive above listall (#6045)
* Honour isrecursive above listall * Revert "Honour isrecursive above listall" This reverts commit 12bc26ae1c58ff5a51937026c5be8d7cfa8bf141. * Being smart about it * Cleanup * fixing npe |
||
|
Nicolas Vazquez
|
413d10dd81
|
server: Extend the Annotations framework (#5103)
* Extend addAnnotation and listAnnotations APIs * Allow users to add, list and remove comments * Add adminsonly UI and allow admins or owners to remove comments * New annotations tab * In progress: new comments section * Address review comments * Fix * Fix annotationfilter and comments section * Add keyword and delete action * Fix and rename annotations tab * Update annotation visibility API and update comments table accordingly * Allow users seeing all the comments for their owned resources * Extend comments for volumes and snapshots * Extend comments to multiple entities * Add uuid to ssh keypairs * SSH keypair UI refactor * Extend comments to the infrastructure entities * Add missing entities * Fix upgrade version for ssh keypairs * Fix typo on DB upgrade schema * Fix annotations table columns when there is no data * Extend the list view of items showing they if they have comments * Remove extra test * Add annotation permissions * Address review comments * Extend marvin tests for annotations * updating ui stuff * addition to toggle visibility * Fix pagination on comments section * Extend to kubernetes clusters * Fixes after last review * Change default value for adminsonly column * Remove the required field for the annotationfilter parameter * Small fixes on visibility and other fixes * Cleanup to reduce files changed * Rollback extra line * Address review comments * Fix cleanup error on smoke test * Fix sending incorrect parameter to checkPermissions method * Add check domain access for the calling account for domain networks * Fix only display annotations icon if there are comments the user can see * Simply change the Save button label to Submit * Change order of the Tools menu to provent users getting 404 error on clicking the text instead of expanding * Remove comments when removing entities * Address review comments on marvin tests * Allow users to list annotations for an entity ID * Allow users to see all comments for allowed entities * Fix search filters * Remove username from search filter * Add pagination to the annotations tab * Display username for user comments * Fix add permissions for domain and resource admins * Fix for domain admins * Trivial but important UI fix * Replace pagination for annotations tab * Add confirmation for delete comment * Lint warnings * Fix reduced list as domain admin * Fix display remove comment button for non admins * Improve display remove action button * Remove unused parameter on groupShow * Include a clock icon to the all comments filter except for root admin * Move cleanup SQL to the correct file after rebasing main Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com> |
||
sureshanaparti
|
eba186aa40
|
storage: New Dell EMC PowerFlex Plugin (formerly ScaleIO, VxFlexOS) (#4304)
Added support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack (for KVM hypervisor) and enabled VM/Volume operations on that pool (using pool tag). Please find more details in the FS here: https://cwiki.apache.org/confluence/x/cDl4CQ Documentation PR: apache/cloudstack-documentation#169 This enables support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack Other improvements addressed in addition to PowerFlex/ScaleIO support: - Added support for config drives in host cache for KVM => Changed configuration "vm.configdrive.primarypool.enabled" scope from Global to Zone level => Introduced new zone level configuration "vm.configdrive.force.host.cache.use" (default: false) to force host cache for config drives => Introduced new zone level configuration "vm.configdrive.use.host.cache.on.unsupported.pool" (default: true) to use host cache for config drives when storage pool doesn't support config drive => Added new parameter "host.cache.location" (default: /var/cache/cloud) in KVM agent.properties for specifying the host cache path and create config drives on the "/config" directory on the host cache path => Maintain the config drive location and use it when required on any config drive operation (migrate, delete) - Detect virtual size from the template URL while registering direct download qcow2 (of KVM hypervisor) templates - Updated full deployment destination for preparing the network(s) on VM start - Propagate the direct download certificates uploaded to the newly added KVM hosts - Discover the template size for direct download templates using any available host from the zones specified on template registration => When zones are not specified while registering template, template size discovery is performed using any available host, which is picked up randomly from one of the available zones - Release the VM resources when VM is sync-ed to Stopped state on PowerReportMissing (after graceful period) - Retry VM deployment/start when the host cannot grant access to volume/template - Mark never-used or downloaded templates as Destroyed on deletion, without sending any DeleteCommand => Do not trigger any DeleteCommand for never-used or downloaded templates as these doesn't exist and cannot be deleted from the datastore - Check the router filesystem is writable or not, before performing health checks => Introduce a new test "filesystem.writable.test" to check the filesystem is writable or not => The router health checks keeps the config info at "/var/cache/cloud" and updates the monitor results at "/root" for health checks, both are different partitions. So, test at both the locations. => Added new script: "filesystem_writable_check.py" at /opt/cloud/bin/ to check the filesystem is writable or not - Fixed NPE issue, template is null for DATA disks. Copy template to target storage for ROOT disk (with template id), skip DATA disk(s) * Addressed some issues for few operations on PowerFlex storage pool. - Updated migration volume operation to sync the status and wait for migration to complete. - Updated VM Snapshot naming, for uniqueness in ScaleIO volume name when more than one volume exists in the VM. - Added sync lock while spooling managed storage template before volume creation from the template (non-direct download). - Updated resize volume error message string. - Blocked the below operations on PowerFlex storage pool: -> Extract Volume -> Create Snapshot for VMSnapshot * Added the PowerFlex/ScaleIO client connection pool to manage the ScaleIO gateway clients, which uses a single gateway client per Powerflex/ScaleIO storage pool and renews it when the session token expires. - The token is valid for 8 hours from the time it was created, unless there has been no activity for 10 minutes. Reference: https://cpsdocs.dellemc.com/bundle/PF_REST_API_RG/page/GUID-92430F19-9F44-42B6-B898-87D5307AE59B.html Other fixes included: - Fail the VM deployment when the host specified in the deployVirtualMachine cmd is not in the right state (i.e. either Resource State is not Enabled or Status is not Up) - Use the physical file size of the template to check the free space availability on the host, while downloading the direct download templates. - Perform basic tests (for connectivity and file system) on router before updating the health check config data => Validate the basic tests (connectivity and file system check) on router => Cleanup the health check results when router is destroyed * Updated PowerFlex/ScaleIO storage plugin version to 4.16.0.0 * UI Changes to support storage plugin for PowerFlex/ScaleIO storage pool. - PowerFlex pool URL generated from the UI inputs(Gateway, Username, Password, Storage Pool) when adding "PowerFlex" Primary Storage - Updated protocol to "custom" for PowerFlex provider - Allow VM Snapshot for stopped VM on KVM hypervisor and PowerFlex/ScaleIO storage pool and Minor improvements in PowerFlex/ScaleIO storage plugin code * Added support for PowerFlex/ScaleIO volume migration across different PowerFlex storage instances. - findStoragePoolsForMigration API returns PowerFlex pool(s) of different instance as suitable pool(s), for volume(s) on PowerFlex storage pool. - Volume(s) with snapshots are not allowed to migrate to different PowerFlex instance. - Volume(s) of running VM are not allowed to migrate to other PowerFlex storage pools. - Volume migration from PowerFlex pool to Non-PowerFlex pool, and vice versa are not supported. * Fixed change service offering smoke tests in test_service_offerings.py, test_vm_snapshots.py * Added the PowerFlex/ScaleIO volume/snapshot name to the paths of respective CloudStack resources (Templates, Volumes, Snapshots and VM Snapshots) * Added new response parameter “supportsStorageSnapshot” (true/false) to volume response, and Updated UI to hide the async backup option while taking snapshot for volume(s) with storage snapshot support. * Fix to remove the duplicate zone wide pools listed while finding storage pools for migration * Updated PowerFlex/ScaleIO volume migration checks and rollback migration on failure * Fixed the PowerFlex/ScaleIO volume name inconsistency issue in the volume path after migration, due to rename failure |
||
Rohit Yadav
|
55a5470da6 | Merge remote-tracking branch 'origin/4.14' | ||
Pearl Dsilva
|
b68be664b7
|
server: Search vm snapshots using tags (#4213)
Search VM snapshots using tags Currently, search of VM snapshots doesn't comply with tags passed as input |
||
|
Nicolas Vazquez
|
8c1d749360
|
[VMware] Enable unmanaging guest VMs (#4103)
* Enable unmanaging guest VMs * Minor fixes * Fix stop usage event only if VM is not stopped when unmanaging * Rename unmanaged VMs manager * Generate netofferingremove usage event if VM is not stopped * Generate usage event VM snapshot primary off when unmanaging |
||
Gregor Riepl
|
29e1bbc22d |
server: rethrow takeVMSnapshot() exception instead of returning null (#3546)
Fixes NPE, and throws actual exception with the error stacktrace Fixes: #3518 |
||
|
Abhishek Kumar
|
9b14e17a58 |
server: fix for vm snapshot search (#3208) (#3257)
Using db column instead of VO variable name was causing issue with SQL select statement. This change fixes the problem by using VO variable for adding conditional. Fixes #3208 Using db column instead of VO variable name was causing issue with SQL select statement. This PR fixes the problem by using VO variable for adding conditional. Additionally in UI listAll parameter was being sent twice in the listVMSnapshot API call. It is fixed with this PR. Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com> |
||
Marc-Aurèle Brothier
|
893a88d225 |
CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure. - moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources - grep scan to search for src/com and src/org left over - grep for <project>/scripts to fix pom.xml configuration - remove custom <build> configuration in pom.xml Signed-off-by: Marc-Aurèle Brothier <m@brothier.org> |