cloudstack/engine/components-api/src/main/java/com/cloud/vm/VirtualMachineProfileImpl.java
sureshanaparti eba186aa40
storage: New Dell EMC PowerFlex Plugin (formerly ScaleIO, VxFlexOS) (#4304)
Added support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack (for KVM hypervisor) and enabled VM/Volume operations on that pool (using pool tag).
Please find more details in the FS here:
https://cwiki.apache.org/confluence/x/cDl4CQ

Documentation PR: apache/cloudstack-documentation#169

This enables support for PowerFlex/ScaleIO (v3.5 onwards) storage pool as a primary storage in CloudStack

Other improvements addressed in addition to PowerFlex/ScaleIO support:

- Added support for config drives in host cache for KVM
	=> Changed configuration "vm.configdrive.primarypool.enabled" scope from Global to Zone level
	=> Introduced new zone level configuration "vm.configdrive.force.host.cache.use" (default: false) to force host cache for config drives
	=> Introduced new zone level configuration "vm.configdrive.use.host.cache.on.unsupported.pool" (default: true) to use host cache for config drives when storage pool doesn't support config drive
	=> Added new parameter "host.cache.location" (default: /var/cache/cloud) in KVM agent.properties for specifying the host cache path and create config drives on the "/config" directory on the host cache path
	=> Maintain the config drive location and use it when required on any config drive operation (migrate, delete)

- Detect virtual size from the template URL while registering direct download qcow2 (of KVM hypervisor) templates

- Updated full deployment destination for preparing the network(s) on VM start

- Propagate the direct download certificates uploaded to the newly added KVM hosts

- Discover the template size for direct download templates using any available host from the zones specified on template registration
	=> When zones are not specified while registering template, template size discovery is performed using any available host, which is picked up randomly from one of the available zones

- Release the VM resources when VM is sync-ed to Stopped state on PowerReportMissing (after graceful period)

- Retry VM deployment/start when the host cannot grant access to volume/template

- Mark never-used or downloaded templates as Destroyed on deletion, without sending any DeleteCommand
	=> Do not trigger any DeleteCommand for never-used or downloaded templates as these doesn't exist and cannot be deleted from the datastore

- Check the router filesystem is writable or not, before performing health checks
	=> Introduce a new test "filesystem.writable.test" to check the filesystem is writable or not
	=> The router health checks keeps the config info at "/var/cache/cloud" and updates the monitor results at "/root" for health checks, both are different partitions. So, test at both the locations.
	=> Added new script: "filesystem_writable_check.py" at /opt/cloud/bin/ to check the filesystem is writable or not

- Fixed NPE issue, template is null for DATA disks. Copy template to target storage for ROOT disk (with template id), skip DATA disk(s)

* Addressed some issues for few operations on PowerFlex storage pool.

- Updated migration volume operation to sync the status and wait for migration to complete.

- Updated VM Snapshot naming, for uniqueness in ScaleIO volume name when more than one volume exists in the VM.

- Added sync lock while spooling managed storage template before volume creation from the template (non-direct download).

- Updated resize volume error message string.

- Blocked the below operations on PowerFlex storage pool:
  -> Extract Volume
  -> Create Snapshot for VMSnapshot

* Added the PowerFlex/ScaleIO client connection pool to manage the ScaleIO gateway clients, which uses a single gateway client per Powerflex/ScaleIO storage pool and renews it when the session token expires.

- The token is valid for 8 hours from the time it was created, unless there has been no activity for 10 minutes.
  Reference: https://cpsdocs.dellemc.com/bundle/PF_REST_API_RG/page/GUID-92430F19-9F44-42B6-B898-87D5307AE59B.html

Other fixes included:

- Fail the VM deployment when the host specified in the deployVirtualMachine cmd is not in the right state (i.e. either Resource State is not Enabled or Status is not Up)

- Use the physical file size of the template to check the free space availability on the host, while downloading the direct download templates.

- Perform basic tests (for connectivity and file system) on router before updating the health check config data
	=> Validate the basic tests (connectivity and file system check) on router
	=> Cleanup the health check results when router is destroyed

* Updated PowerFlex/ScaleIO storage plugin version to 4.16.0.0

* UI Changes to support storage plugin for PowerFlex/ScaleIO storage pool.
- PowerFlex pool URL generated from the UI inputs(Gateway, Username, Password, Storage Pool) when adding "PowerFlex" Primary Storage
- Updated protocol to "custom" for PowerFlex provider
- Allow VM Snapshot for stopped VM on KVM hypervisor and PowerFlex/ScaleIO storage pool

and Minor improvements in PowerFlex/ScaleIO storage plugin code

* Added support for PowerFlex/ScaleIO volume migration across different PowerFlex storage instances.

- findStoragePoolsForMigration API returns PowerFlex pool(s) of different instance as suitable pool(s), for volume(s) on PowerFlex storage pool.
- Volume(s) with snapshots are not allowed to migrate to different PowerFlex instance.
- Volume(s) of running VM are not allowed to migrate to other PowerFlex storage pools.
- Volume migration from PowerFlex pool to Non-PowerFlex pool, and vice versa are not supported.

* Fixed change service offering smoke tests in test_service_offerings.py, test_vm_snapshots.py

* Added the PowerFlex/ScaleIO volume/snapshot name to the paths of respective CloudStack resources (Templates, Volumes, Snapshots and VM Snapshots)

* Added new response parameter “supportsStorageSnapshot” (true/false) to volume response, and Updated UI to hide the async backup option while taking snapshot for volume(s) with storage snapshot support.

* Fix to remove the duplicate zone wide pools listed while finding storage pools for migration

* Updated PowerFlex/ScaleIO volume migration checks and rollback migration on failure

* Fixed the PowerFlex/ScaleIO volume name inconsistency issue in the volume path after migration, due to rename failure
2021-02-24 14:58:33 +05:30

343 lines
8.3 KiB
Java

// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package com.cloud.vm;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.cloud.agent.api.to.DiskTO;
import com.cloud.host.Host;
import com.cloud.hypervisor.Hypervisor.HypervisorType;
import com.cloud.network.element.NetworkElement;
import com.cloud.offering.ServiceOffering;
import com.cloud.service.ServiceOfferingVO;
import com.cloud.template.VirtualMachineTemplate;
import com.cloud.template.VirtualMachineTemplate.BootloaderType;
import com.cloud.user.Account;
import com.cloud.utils.db.EntityManager;
/**
* Implementation of VirtualMachineProfile.
*
*/
public class VirtualMachineProfileImpl implements VirtualMachineProfile {
VirtualMachine _vm;
ServiceOffering _offering;
VirtualMachineTemplate _template;
UserVmDetailVO _userVmDetails;
Map<Param, Object> _params;
List<NicProfile> _nics = new ArrayList<NicProfile>();
List<DiskTO> _disks = new ArrayList<DiskTO>();
StringBuilder _bootArgs = new StringBuilder();
Account _owner;
BootloaderType _bootloader;
Float cpuOvercommitRatio = 1.0f;
Float memoryOvercommitRatio = 1.0f;
Host _host = null;
VirtualMachine.Type _type;
List<String[]> vmData = null;
String configDriveLabel = null;
String configDriveIsoBaseLocation = "/tmp/";
String configDriveIsoRootFolder = null;
String configDriveIsoFile = null;
NetworkElement.Location configDriveLocation = NetworkElement.Location.SECONDARY;
public VirtualMachineProfileImpl(VirtualMachine vm, VirtualMachineTemplate template, ServiceOffering offering, Account owner, Map<Param, Object> params) {
_vm = vm;
_template = template;
_offering = offering;
_params = params;
_owner = owner;
if (_params == null) {
_params = new HashMap<Param, Object>();
}
if (vm != null)
_type = vm.getType();
}
public VirtualMachineProfileImpl(VirtualMachine vm) {
this(vm, null, null, null, null);
}
public VirtualMachineProfileImpl(VirtualMachine.Type type) {
_type = type;
}
@Override
public String toString() {
return _vm.toString();
}
@Override
public VirtualMachine getVirtualMachine() {
return _vm;
}
@Override
public ServiceOffering getServiceOffering() {
if (_offering == null) {
_offering = s_entityMgr.findById(ServiceOffering.class, _vm.getServiceOfferingId());
}
return _offering;
}
@Override
public void setParameter(Param name, Object value) {
_params.put(name, value);
}
@Override
public void setBootLoaderType(BootloaderType bootLoader) {
_bootloader = bootLoader;
}
@Override
public VirtualMachineTemplate getTemplate() {
if (_template == null && _vm != null) {
_template = s_entityMgr.findByIdIncludingRemoved(VirtualMachineTemplate.class, _vm.getTemplateId());
}
return _template;
}
@Override
public HypervisorType getHypervisorType() {
return _vm.getHypervisorType();
}
@Override
public long getTemplateId() {
return _vm.getTemplateId();
}
@Override
public long getServiceOfferingId() {
return _vm.getServiceOfferingId();
}
@Override
public long getId() {
return _vm.getId();
}
@Override
public String getUuid() {
return _vm.getUuid();
}
public void setNics(List<NicProfile> nics) {
_nics = nics;
}
public void setDisks(List<DiskTO> disks) {
_disks = disks;
}
@Override
public List<NicProfile> getNics() {
return _nics;
}
@Override
public List<DiskTO> getDisks() {
return _disks;
}
@Override
public void addNic(int index, NicProfile nic) {
_nics.add(index, nic);
}
@Override
public void addDisk(int index, DiskTO disk) {
_disks.add(index, disk);
}
@Override
public StringBuilder getBootArgsBuilder() {
return _bootArgs;
}
@Override
public void addBootArgs(String... args) {
for (String arg : args) {
_bootArgs.append(arg).append(" ");
}
}
@Override
public VirtualMachine.Type getType() {
return _type;
}
@Override
public Account getOwner() {
if (_owner == null) {
_owner = s_entityMgr.findById(Account.class, _vm.getAccountId());
}
return _owner;
}
@Override
public String getBootArgs() {
return _bootArgs.toString();
}
static EntityManager s_entityMgr;
static void init(EntityManager entityMgr) {
s_entityMgr = entityMgr;
}
@Override
public void addNic(NicProfile nic) {
_nics.add(nic);
}
@Override
public void addDisk(DiskTO disk) {
_disks.add(disk);
}
@Override
public Object getParameter(Param name) {
return _params.get(name);
}
@Override
public Long getHostId() {
if (_host != null) {
return _host.getId();
}
return _vm.getHostId();
}
@Override
public void setHost(Host host) {
this._host = host;
}
@Override
public String getHostName() {
return _vm.getHostName();
}
@Override
public String getInstanceName() {
return _vm.getInstanceName();
}
@Override
public BootloaderType getBootLoaderType() {
return _bootloader;
}
@Override
public Map<Param, Object> getParameters() {
return _params;
}
public void setServiceOffering(ServiceOfferingVO offering) {
_offering = offering;
}
public void setCpuOvercommitRatio(Float cpuOvercommitRatio) {
this.cpuOvercommitRatio = cpuOvercommitRatio;
}
public void setMemoryOvercommitRatio(Float memoryOvercommitRatio) {
this.memoryOvercommitRatio = memoryOvercommitRatio;
}
@Override
public Float getCpuOvercommitRatio() {
return cpuOvercommitRatio;
}
@Override
public Float getMemoryOvercommitRatio() {
return memoryOvercommitRatio;
}
@Override
public boolean isRollingRestart() {
return Boolean.TRUE.equals(getParameter(VirtualMachineProfile.Param.RollingRestart));
}
@Override
public List<String[]> getVmData() {
return vmData;
}
@Override
public void setVmData(List<String[]> vmData) {
this.vmData = vmData;
}
@Override
public String getConfigDriveLabel() {
return configDriveLabel;
}
@Override
public void setConfigDriveLabel(String configDriveLabel) {
this.configDriveLabel = configDriveLabel;
}
@Override
public String getConfigDriveIsoRootFolder() {
return configDriveIsoRootFolder;
}
@Override
public void setConfigDriveIsoRootFolder(String configDriveIsoRootFolder) {
this.configDriveIsoRootFolder = configDriveIsoRootFolder;
}
public String getConfigDriveIsoBaseLocation() {
return configDriveIsoBaseLocation;
}
@Override
public String getConfigDriveIsoFile() {
return configDriveIsoFile;
}
@Override
public void setConfigDriveIsoFile(String isoFile) {
this.configDriveIsoFile = isoFile;
}
@Override
public NetworkElement.Location getConfigDriveLocation() {
return configDriveLocation;
}
@Override
public void setConfigDriveLocation(NetworkElement.Location location) {
this.configDriveLocation = location;
}
}