442 Commits

Author SHA1 Message Date
Suresh Kumar Anaparti
8ea9fc911d
StoragePoolType as class (#8544)
* StoragePoolType as a class

* Fix agent side StoragePoolType enum to class

* Handle StoragePoolType for StoragePoolJoinVO

* Since StoragePoolType is a class, it cannot be converted by @Enumerated annotation.
Implemented conveter class and logic to utilize @Convert annotation.

* Fix UserVMJoinVO for StoragePoolType

* fixed missing imports

* Since StoragePoolType is a class, it cannot be converted by @Enumerated annotation.
Implemented conveter class and logic to utilize @Convert annotation.

* Fixed equals for the enum.

* removed not needed try/catch for prepareAttribute

* Added license to the file.

* Implemented "supportsPhysicalDiskCopy" for storage adaptor.

Co-authored-by: mprokopchuk <mprokopchuk@apple.com>

* Add javadoc to StoragePoolType class

* Add unit test for StoragePoolType comparisons

* StoragePoolType "==" and ".equals()" fix.

* Fix StoragePoolType for FiberChannelAdapter

* Fix for abstract storage adaptor set up issue

* review comments

* Pass StoragePoolType object for poolType dao attribute

---------

Co-authored-by: Marcus Sorensen <mls@apple.com>
Co-authored-by: mprokopchuk <mprokopchuk@apple.com>
Co-authored-by: mprokopchuk <mprokopchuk@gmail.com>
2024-02-05 13:27:15 +05:30
Abhishek Kumar
7dffbc6e47 Updating pom.xml version numbers for release 4.20.0.0-SNAPSHOT
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-02-02 18:16:37 +05:30
Abhishek Kumar
a7b97ff3b0 Updating pom.xml version numbers for release 4.19.1.0-SNAPSHOT
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-02-02 18:06:04 +05:30
Abhishek Kumar
2746225b99 Updating pom.xml version numbers for release 4.19.0.0
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2024-01-29 10:21:52 +05:30
Abhishek Kumar
d6ac91f2df
minio: fix store user creation (#8425)
To prevent errors during multi-user access, use account UUID to create/access user on the provider side. Also, update the existing secret key for a user that already exists.
2024-01-09 17:44:11 +05:30
Rene Glover
1031c31e6a
FiberChannel Multipath for KVM + Pure Flash Array and HPE-Primera Support (#7889)
This PR provides a new primary storage volume type called "FiberChannel" that allows access to volumes connected to hosts over fiber channel connections. It requires Multipath to provide path discovery and failover. Second, the PR adds an AdaptivePrimaryDatastoreProvider that abstracts how volumes are managed/orchestrated from the connector to communicate with the primary storage provider, using a ProviderAdapter interface, allowing the code interacting with the primary storage provider API's to be simpler and have no direct dependencies on Cloudstack code. Lastly, the PR provides an implementation of the ProviderAdapter classes for the HP Enterprise Primera line of storage solutions and the Pure Flash Array line of storage solutions.
2023-12-09 11:31:33 +05:30
Abhishek Kumar
c599011ef5 Merge remote-tracking branch 'apache/4.18' 2023-12-08 18:06:15 +05:30
Peinthor Rene
bba554bcc4
linstor: Fix possible NPE if Linstor storage-pool data missing (#8319)
If Linstor doesn't return storage pool info, certain values are null.
Now we assume the values are 0 if we get null values.
2023-12-08 17:02:18 +05:30
Peinthor Rene
a15b706fbe
Linstor: Allow snapshot backup also to work on non hyperconverged setups (#8271)
On no access to the storage nodes, we now create a temporary resource from the snapshot and copy that data into the secondary storage. Revert works the same, just that we now also look additionally for any Linstor agent node.

Also enables now backup snapshot by default.

This whole BackupSnapshot functionality was introduced in 4.19,
so I would be happy if this still could be merged.
2023-12-05 12:59:52 +05:30
kishankavala
5651eab49c
ObjectStore Framework with MinIO and Simulator plugins (#7752)
This PR adds Object Storage feature to CloudStack.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BDRAFT%5D+CloudStack+Object+Store
2023-12-01 17:51:00 +05:30
João Jandre
26b01f6f3b
Flexible tags for hosts and storage pools (#7489)
Co-authored-by: João Jandre <joao@scclouds.com.br>
2023-11-30 09:36:47 +01:00
DK101010
6001772335
multi local storage handling for kvm (#6699)
Co-authored-by: DK101010 <dirk.klahre@itelligence.de>
Co-authored-by: João Jandre <48719461+JoaoJandre@users.noreply.github.com>
2023-11-16 16:43:42 +01:00
Peinthor Rene
68e504aff9
Linstor backup snaphots (#8067)
This PR adds an config option for the Linstor primary storage driver, that allows you to automatically backup
volume snapshots to the secondary storage.
Additionally it will not mangle the need java-linstor dependency into the client.jar, but instead just copy
the java-linstor.jar into lib.

Config option is called: lin.backup.snapshots and is default false

The scope of this change should be limited, as it only touches the Linstor driver and a part of copyAsync
was implemented with 2 new Linstor specific commands.
2023-11-09 09:38:10 +05:30
slavkap
2bb182c3e1
KVM Host HA enhancement for StorPool storage (#8045)
Extending the current functionality of KVM Host HA for the StorPool storage plugin and the option for easy integration for the rest of the storage plugins to support Host HA

This extension works like the current NFS storage implementation. It allows it to be used simultaneously with NFS and StorPool storage or only with StorPool primary storage.

If it is used with different primary storages like NFS and StorPool, and one of the health checks fails for storage, there is an option to report the failure to the management with the global config kvm.ha.fence.on.storage.heartbeat.failure. By default this option is disabled when enabled the Host HA service will continue with the checks on the host and eventually will fence the host
2023-11-04 12:35:37 +05:30
John Bampton
f090c77f41
misc: fix spelling (#7549)
Co-authored-by: Stephan Krug <stekrug@icloud.com>
2023-11-02 09:23:53 +01:00
Vishesh
5362bad442
Storage Management (#7949) 2023-11-01 10:46:22 +01:00
Abhishek Kumar
543c54c718
api,server,ui: snapshot copy, multi-zone replica (#7873)
This PR adds new functionality to copy snapshots across zones and take snapshots for multiple zones.

Copy functionality is similar to template copy. The source zone acts as the web server from where the destination zone(s) can download the snapshot files. For this purpose, a new API - `copySnapshot` has been added. The response for copySnapshot will be returning zone and download details from the first destination zone of the request. This behaviour is similar to the `copyTemplate` API.

In a similar manner, multiple zones can be selected while taking the snapshots or creating snapshot policies. For this snapshot will be taken in the base zone(in which volume is present) and then copied to the additional zones. A new parameter - `zoneids` has been added to `createSnapshot` and `createSnapshotPolicy` APIs.

As snapshots can be present on multiple zones (secondary stores), a new parameter `zoneid` has been added to delete the snapshot copy on a specific zone.

`listSnapshots` API has been updated to allow listing snapshot entries for different zones/datastores. New parameters - `showUnique`, `locationType` have been added.

Events generated during snapshot operations will now be linked to the snapshot itself rather than the volume of the snapshot.

`listSnapshotPolicies` and `createSnapshotPolicy` APIs will return zone details of the zones in which backup will be scheduled for the policy.

----
New API added
`copySnapshot`

Request and response params updated for APIs
```
- listSnapshots
- deleteSnapshot
- createTemplate
- listZones
- listSnapshotPolicies
- createSnapshotPolicy
```
UI updated for
- Snapshot detail view
- Create snapshot form
- Create snapshot policy form
- Create volume (from snapshot) form
- Create template (from snapshot) form

Doc PR: https://github.com/apache/cloudstack-documentation/pull/344
PR: https://github.com/apache/cloudstack/pull/7873
2023-10-23 09:01:58 +02:00
Abhishek Kumar
fcbf540369 Merge remote-tracking branch 'apache/4.18' into main 2023-10-19 10:48:00 +05:30
Peinthor Rene
67cb9b9e40
linstor: fix template copy on non hyperconverged setups (#8114)
Making a diskful resource was meant as an optimization,
but cannot work on non hyperconverged setups,
as the storage nodes (diskful) are not part of the cloudstack cluster.
2023-10-19 10:46:20 +05:30
Abhishek Kumar
be039a1e46 Merge remote-tracking branch 'apache/4.18' into main 2023-10-17 17:22:41 +05:30
Peinthor Rene
4a86a0d233
linstor: Fix template volume missing on copy node (#8082)
A TODO was overseen and never implemented,
which could trigger the following bug:

If Linstor didn't create a resource (diskless or diskfull) on
the cloudstack choosen node, it would not be able to copy the
template data there, it even seems no error was
triggered and the new template file silently just became
empty/corrupt.
2023-10-17 17:05:42 +05:30
Daan Hoogland
d627539688 Merge branch '4.18' 2023-10-02 16:03:28 +02:00
Peinthor Rene
96205a51ef
linstor: resize root disk on offerings with different size (#7952) 2023-10-02 15:58:00 +02:00
slavkap
3c608290db
Fix tests on CS build when cloudstack-management service is running (#8009) 2023-09-27 19:32:07 +05:30
Vishesh
84e770bf95
Remove powermock from pom.xml (#8000) 2023-09-26 13:31:56 +02:00
Wei Zhou
246bb24b0f Updating pom.xml version numbers for release 4.18.2.0-SNAPSHOT
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-12 17:26:53 +02:00
Wei Zhou
f6b2a58727 Merge branch '4.18' 2023-09-07 08:56:35 +02:00
Wei Zhou
4bdff06acd Updating pom.xml version numbers for release 4.18.1.0
Signed-off-by: Wei Zhou <weizhou@apache.org>
2023-09-07 08:50:50 +02:00
Marcus Sorensen
89e0a4ca25
PowerFlex: Handle missing volumes gracefully during delete volume (#7924) 2023-08-31 21:43:45 +02:00
Daan Hoogland
24ae5aa5fa Merge branch '4.18' 2023-08-25 14:27:34 +02:00
Rohit Yadav
e964395bd4
vmware: improve solidfire storage plugin integration and fix cases (#3) (#7761)
This fixes the following cases in which Solidfire storage integration
caused issues when using Solidfire datadisks with VMware:

1. Take Volume Snapshot of Solidfire data disk
2. Delete an active Instance with Solidfire data disk attached
3. Attach used existing Solidfire data disk to a running/stopped VM
4. Stop and Start an instance with Solidfire data disks attached
5. Expand disk by resizing Solidfire data disk by providing size
6. Expand disk by changing disk offering for the Solidfire data disk

Additional changes:
- Use VMFS6 as managed datastore type if the host supports
- Refactor detection and splitting of managed storage ds name in storage
  processor
- Restrict storage rescanning for managed datastore when resizing

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2023-08-25 11:36:03 +05:30
Daan Hoogland
ea832bce13 Merge branch '4.18' 2023-08-22 11:44:45 +02:00
Sina Kashipazha
d296f54c7f
Api: update command domainId/account descriptions (#7876) 2023-08-20 15:44:31 +02:00
Wei Zhou
540dcc1040 Merge remote-tracking branch 'apache/4.18' 2023-08-11 08:30:07 +02:00
Wei Zhou
d9319e645c
storpool: fix pom.xml which caused docker image build failures (#7845)
steps to reproduce the issue:
- git clone https://github.com/apache/cloudstack.git
- cd cloudstack
- rm -rf .git/
- run `mvn -P developer,systemvm clean install`

Without this PR, it fails with error

```
 > [ 8/10] RUN mvn -Pdeveloper -Dsimulator -DskipTests clean install:
668.1 [ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:4.9.10:revision (get-the-git-infos) on project cloud-plugin-storage-volume-storpool: .git directory is not found! Please specify a valid [dotGitDirectory] in your pom.xml -> [Help 1]
```
2023-08-10 16:28:49 +02:00
Daan Hoogland
7ee697fcc5 Merge release branch 4.18 to main
* 4.18:
  UI: allow new keys for VM details (#7793)
  Refactoring StorPool's smoke tests (#7392)
  UI: decode userdata in EditVM dialog (#7796)
  packaging: unalias cp before package upgrade (#7722)
  make NoopDbUpgrade do a systemvm template check (#7564)
  UI unit test: fix expected values (#7792)
2023-08-03 15:59:01 +02:00
John Bampton
6f4503488b
pre-commit: apply end-of-file-fixer to all files (#7551) 2023-08-02 13:47:21 +02:00
slavkap
da7fc72c66
Refactoring StorPool's smoke tests (#7392)
* Removed the hardcoded StorPool endpoint from tests

- removed the hardcoded enpoint of StorPool primary storage from tests
- added the git commit information into the maven build

* Convert indents to spaces

* update git-commit-id-plugin version
2023-08-02 14:37:37 +08:00
Rohit Yadav
62a8f4ef72 Merge remote-tracking branch 'origin/4.18' 2023-07-24 15:57:37 +05:30
Marcus Sorensen
63216425d5
Set encrypted PowerFlex disk format correctly (#7735)
Co-authored-by: Marcus Sorensen <mls@apple.com>
2023-07-24 13:13:46 +05:30
Daan Hoogland
afec876951 Merge release branch 4.18 to main
* 4.18:
  SSVM: 'allow from' private IP in other SSVMs if the public IP is in allowed internal sites cidrs (#7288)
  eof added to StorPoolStatsCollector (#7754)
2023-07-20 13:41:20 +02:00
dahn
9c5e489d82
eof added to StorPoolStatsCollector (#7754) 2023-07-19 12:36:44 +02:00
Daan Hoogland
6bb95c0200 Merge release branch 4.18 to main
* 4.18:
  Storage and volumes statistics tasks for StorPool primary storage (#7404)
  proper storage construction (#6797)
  guarantee MAC uniqueness (#7634)
  server: allow migration of all VMs with local storage on KVM (#7656)
  Add L2 networks to Zones with SG (#7719)
2023-07-19 10:59:19 +02:00
slavkap
f32a63be60
Storage and volumes statistics tasks for StorPool primary storage (#7404) 2023-07-19 10:48:36 +02:00
Daan Hoogland
a706bf2380 Merge branch '4.18' 2023-07-07 11:49:35 +02:00
Vishesh
594c70dde0
Sync precommit config from main (#7732)
Co-authored-by: John Bampton <jbampton@users.noreply.github.com>
Co-authored-by: dahn <daan@onecht.net>
2023-07-07 11:18:16 +02:00
Rohit Yadav
3b054b2665 Merge remote-tracking branch 'origin/4.18' 2023-07-03 13:17:25 +05:30
Harikrishna
70820137e6
scaleio: Avoid race condition while handling host disconnect and connect scenarios (#282) (#7689)
This PR fixes an intermittent issue where SDC id (local_path) is getting deleted and not getting populated when host connects back again.

Fix is to remove the code to delete the records from storage_pool_host_ref table. We are anyways updating the entry if the SDC ID is changed during agent restart which is anyways required inorder to get the new connections. I've quickly verified the host delete scenario to check the storage_pool_host_ref entries behavior, entries are getting deleted.
2023-07-03 12:58:42 +05:30
Wei Zhou
41403c9cf9 Merge remote-tracking branch 'apache/4.18' 2023-06-27 11:52:37 +02:00
Vishesh
c3718ab92e
Fix storpool test & remove powermock (#7652)
This PR fixes unit tests for plugins/storage/volume/storpool/ and removes powermock.
2023-06-27 14:55:19 +05:30