* Use free/total instead of free metric to calculate imbalance
* Filter out hosts for condensed while checking imbalance
* Make DRS more configurable
* code refactor
* Add unit tests
* fixup
* Fix validation for drs.imbalance.condensed.skip.threshold
* Add logging and other minor changes for drs
* Add some logging for drs
* Change format for drs imbalance to string
* Show drs imbalance as percentage
* Fixup label for memorytotal in en.json
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.
Implementation
There are three top level components:
Scheduler
A timer task which:
Generate DRS plan for clusters
Process DRS plan
Remove old DRS plan records
DRS Execution
We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.
Algorithm
Every algorithms implements two methods:
needsDrs - to check if drs is required for cluster
getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.
Algorithms
Condensed - Packs all the VMs on minimum number of hosts in the cluster.
Balanced - Distributes the VMs evenly across hosts in the cluster.
Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.
APIs Added
listClusterDrsPlan
id - ID of the DRS plan to list
clusterid - to list plans for a cluster id
generateClusterDrsPlan
id - cluster id
iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.
executeClusterDrsPlan
id - ID of the cluster for which DRS plan is to be executed.
migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.
Config Keys Added
ClusterDrsPlanExpireInterval
Key drs.plan.expire.interval
Scope Global
Default Value 30 days
Description The interval in days after which old DRS records will be cleaned up.
ClusterDrsEnabled
Key drs.automatic.enable
Scope Cluster
Default Value false
Description Enable/disable automatic DRS on a cluster.
ClusterDrsInterval
Key drs.automatic.interval
Scope Cluster
Default Value 60 minutes
Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.
ClusterDrsIterations
Key drs.max.migrations
Scope Cluster
Default Value 50
Description Maximum number of live migrations in a DRS execution.
ClusterDrsAlgorithm
Key drs.algorithm
Scope Cluster
Default Value condensed
Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.
ClusterDrsLevel
Key drs.imbalance
Scope Cluster
Default Value 0.5
Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
is allowed and 0.0 means imbalance is allowed.
ClusterDrsMetric
Key drs.imbalance.metric
Scope Cluster
Default Value memory
Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
* Extend addAnnotation and listAnnotations APIs
* Allow users to add, list and remove comments
* Add adminsonly UI and allow admins or owners to remove comments
* New annotations tab
* In progress: new comments section
* Address review comments
* Fix
* Fix annotationfilter and comments section
* Add keyword and delete action
* Fix and rename annotations tab
* Update annotation visibility API and update comments table accordingly
* Allow users seeing all the comments for their owned resources
* Extend comments for volumes and snapshots
* Extend comments to multiple entities
* Add uuid to ssh keypairs
* SSH keypair UI refactor
* Extend comments to the infrastructure entities
* Add missing entities
* Fix upgrade version for ssh keypairs
* Fix typo on DB upgrade schema
* Fix annotations table columns when there is no data
* Extend the list view of items showing they if they have comments
* Remove extra test
* Add annotation permissions
* Address review comments
* Extend marvin tests for annotations
* updating ui stuff
* addition to toggle visibility
* Fix pagination on comments section
* Extend to kubernetes clusters
* Fixes after last review
* Change default value for adminsonly column
* Remove the required field for the annotationfilter parameter
* Small fixes on visibility and other fixes
* Cleanup to reduce files changed
* Rollback extra line
* Address review comments
* Fix cleanup error on smoke test
* Fix sending incorrect parameter to checkPermissions method
* Add check domain access for the calling account for domain networks
* Fix only display annotations icon if there are comments the user can see
* Simply change the Save button label to Submit
* Change order of the Tools menu to provent users getting 404 error on clicking the text instead of expanding
* Remove comments when removing entities
* Address review comments on marvin tests
* Allow users to list annotations for an entity ID
* Allow users to see all comments for allowed entities
* Fix search filters
* Remove username from search filter
* Add pagination to the annotations tab
* Display username for user comments
* Fix add permissions for domain and resource admins
* Fix for domain admins
* Trivial but important UI fix
* Replace pagination for annotations tab
* Add confirmation for delete comment
* Lint warnings
* Fix reduced list as domain admin
* Fix display remove comment button for non admins
* Improve display remove action button
* Remove unused parameter on groupShow
* Include a clock icon to the all comments filter except for root admin
* Move cleanup SQL to the correct file after rebasing main
Co-authored-by: davidjumani <dj.davidjumani1994@gmail.com>
Fixes :
- Fixing scale router
- Fixing account actions
- Fixing user actions
- Adding message for create vm backup
- Fix default allowuserdrivenbackups in ImportBackupOfferings
- Fix typo in TakeSnapshot
- Ensuring zone mandatory in upload template
- Adding securitygroup to instacetab
- Adding related vms to routers
- Adding makeredundant to restart network
- Fixing no key in listview
- Link to ipaddress only if router path is publicip
- Show vpc routers only to admin
- Fix restartVPC args
- Fix storage action visibility
- Reorder routes to match legacy
- Reorder cluster tabs
- Fix number input width
- Fix create vpc
- List events also on fetchlatest
- Fix show domain actions
- Removing resource admin from default roles
- Fix missing store
- Adding createVPC view
- Adding attachiso view
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixes :
- Don't allow users in UI to delete/archive events
- Fix button name in VM deployment form in its network section to Create new network
- Refresh after template / iso upload
- Making external-id mandatory in ImportBackupOffering
- Fixing visibility of assignVirtualMachineToBackupOffering
- Removing link on traffic label
- Defensive check in TrafficTypesTab
- Ensuring we get the user info so that store.getters.user is never empty when the page is freshly loaded
- Changing from report bug to report issue
- Ordering projects in menu
- Changing router get health check results
- Show configureHAForHost based on hypervisor
- Fix scale and migrate router
- Fix scale and migrate systemvm
- Fix show actionbutton for assignVirtualMachineToBackupOffering
- Fix show actionbutton for stopKubernetesCluster
- Fix show actions for volumes
- Fix show actions for snapshots
- Fix show actions for vm snapshots
- Fix show actions for backups
- Adding loading for tags and annotations
- Enter to submit advanced search
- Fixing show Project instead of account when passed as projectaccount passed in account field
- Show project name instead of displaytext
- Fixing template and iso actions
- Fixing tags with projectid
- Fix security groups ingress/egress rules view
- Removing redundant allocationstate from zones
- Adding managedstate to clusters
- Adding capacity tab to clusters and pods
- Adding routerlink to events in dashboard
- Set autofocus to username in login
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
The new convention used:
- All lower keys
- Most keys start with `label.` prefix
- All spaces and underscores replaced with `.`
Quick ugly hack and similar for config `title`:
> IFS=$'\n'
> git grep -l "\$t('" | grep -v -e "\$t('label\." -e "\$t('message" -e "\$t('error" > ftr
> for f in $(cat ftr); do echo $f; for w in $(grep -Po '\$t\(\K[^)]*' $f | grep "^'" | grep -v -e ^\'message -e ^\'error -e ^\'label); do echo WORD IS $w; nw=$(echo $w | sed "s/$w/'label\.\L&/g" | sed "s/label\.'/label\./g" | sed "s/ /\./g" | sed "s/_/\./g"); echo $w = $nw; sed -i "s/\$t($w/\$t($nw/g" $f; done; done
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Changes permission checking behaviour where all list of APIs must be permitted to see a view.
The first API will be used to list/render the table. Remaining APIs will only be used to verify permissions. If any of the passed API is not found in the allowed APIs for logged-in user's role section/sub-section won't be displayed.
Fixes#241Fixes#244Fixes#240
Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This adds the add cluster form with domain/account dedication option.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
Adds dedicated section for zone/pod/cluster/host to dedicate that resource to a domain (and account). Implements the list, dedicate and release APIs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: Rohit Yadav <rohit@apache.org>
This implements actions args remapping framework which allows developers
to specify how to provide/show args to the user/admin or how to override
based on the resource by means of (a) `value(record)` function, (b)
statically defined `api` name, (c) `options` array.
For example, in the config file:
```
args: ['id', 'virtualmachineid', 'mode'],
mapping: {
id: {
api: 'listIsos'
},
virtualmachineid: {
value: (record, params) => { return record.id }
},
mode: {
options: ['http', 'nfs', 'something else']
}
}
```
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Fixes: #17Fixes: #18
These dependencies are necessary or eslint will fail with the standard vue.js rule set.
The PR also includes fixes for all lint errors.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Add actions from old UI to new UI
Fixes: #40
Doing the following:
Extract out actions from old UI to new for infra section - zones, pods, clusters, ...etc.
Support translation of labels for actions instead of hard coding them
Translation of action buttons in list view, detail view, forms and also their fields will now be human readable.
Some misc changes - like comparing API param names with case insensitive stuff, allowing reusing of same API for multiple actions by passing defaultArgs: to action in config files.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
