Vishesh ea90848429
Feature: Add support for DRS in a Cluster (#7723)
This pull request (PR) implements a Distributed Resource Scheduler (DRS) for a CloudStack cluster. The primary objective of this feature is to enable automatic resource optimization and workload balancing within the cluster by live migrating the VMs as per configuration.
Administrators can also execute DRS manually for a cluster, using the UI or the API.
Adds support for two algorithms - condensed & balanced. Algorithms are pluggable allowing ACS Administrators to have customized control over scheduling.

Implementation
There are three top level components:

    Scheduler
    A timer task which:

    Generate DRS plan for clusters
    Process DRS plan
    Remove old DRS plan records

    DRS Execution
    We go through each VM in the cluster and use the specified algorithm to check if DRS is required and to calculate cost, benefit & improvement of migrating that VM to another host in the cluster. On the basis of cost, benefit & improvement, the best migration is selected for the current iteration and the VM is migrated. The maximum number of iterations (live migrations) possible on the cluster is defined by drs.iterations which is defined as a percentage (as a value between 0 and 1) of total number of workloads.

    Algorithm
    Every algorithms implements two methods:
        needsDrs - to check if drs is required for cluster
        getMetrics - to calculate cost, benefit & improvement of a migrating a VM to another host.

Algorithms

    Condensed - Packs all the VMs on minimum number of hosts in the cluster.
    Balanced - Distributes the VMs evenly across hosts in the cluster.
    Algorithms use drs.level to decide the amount of imbalance to allow in the cluster.

APIs Added

listClusterDrsPlan

    id - ID of the DRS plan to list
    clusterid - to list plans for a cluster id

generateClusterDrsPlan

    id - cluster id
    iterations - The maximum number of iterations in a DRS job defined as a percentage (as a value between 0 and 1) of total number of workloads. Defaults to value of cluster's drs.iterations setting.

executeClusterDrsPlan

    id - ID of the cluster for which DRS plan is to be executed.
    migrateto - This parameter specifies the mapping between a vm and a host to migrate that VM. Format of this parameter: migrateto[vm-index].vm=<uuid>&migrateto[vm-index].host=<uuid>.

Config Keys Added

    ClusterDrsPlanExpireInterval
    Key drs.plan.expire.interval
    Scope Global
    Default Value 30 days
    Description The interval in days after which old DRS records will be cleaned up.

    ClusterDrsEnabled
    Key drs.automatic.enable
    Scope Cluster
    Default Value false
    Description Enable/disable automatic DRS on a cluster.

    ClusterDrsInterval
    Key drs.automatic.interval
    Scope Cluster
    Default Value 60 minutes
    Description The interval in minutes after which a periodic background thread will schedule DRS for a cluster.

    ClusterDrsIterations
    Key drs.max.migrations
    Scope Cluster
    Default Value 50
    Description Maximum number of live migrations in a DRS execution.

    ClusterDrsAlgorithm
    Key drs.algorithm
    Scope Cluster
    Default Value condensed
    Description DRS algorithm to execute on the cluster. This PR implements two algorithms - balanced & condensed.

    ClusterDrsLevel
    Key drs.imbalance
    Scope Cluster
    Default Value 0.5
    Description Percentage (as a value between 0.0 and 1.0) of imbalance allowed in the cluster. 1.0 means no imbalance
    is allowed and 0.0 means imbalance is allowed.

    ClusterDrsMetric
    Key drs.imbalance.metric
    Scope Cluster
    Default Value memory
    Description The cluster imbalance metric to use when checking the drs.imbalance.threshold. Possible values are memory and cpu.
2023-10-26 11:48:18 +05:30
..
2023-08-11 16:47:11 +02:00
2021-01-20 07:06:11 +05:30
2021-01-20 07:06:24 +05:30
2021-01-20 07:06:11 +05:30
2021-01-20 07:06:13 +05:30
2023-04-11 14:05:21 +05:30

CloudStack UI

A modern role-based progressive CloudStack UI based on VueJS and Ant Design.

Screenshot

Getting Started

Install node: (Debian/Ubuntu)

curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt-get install -y nodejs
# Or use distro provided: sudo apt-get install npm nodejs

Install node: (CentOS/Fedora/RHEL)

curl -sL https://rpm.nodesource.com/setup_14.x | sudo bash -
sudo yum install nodejs

Install node: (Mac OS)

brew install node@16

Optionally, you may also install system-wide dev tools:

sudo npm install -g @vue/cli npm-check-updates

Development

Clone the repository:

git clone https://github.com/apache/cloudstack.git
cd cloudstack/ui
npm install

Override the default CS_URL to a running CloudStack management server:

cp .env.local.example .env.local
Change the `CS_URL` in the `.env.local` file

To configure https, you may use .env.local.https.example.

To use the community Simulator QA server you can do this:

echo "CS_URL=https://qa.cloudstack.cloud" > .env.local

Build and run:

npm run serve
# Or run: npm start

Upgrade dependencies to the latest versions:

ncu -u

Run Lint and Unit Tests:

npm run lint
npm run test:unit

Fix issues and vulnerabilities:

npm audit

A basic development guide and explaination of the basic components can be found here

Production

Fetch dependencies and build:

npm install
npm run build

This creates a static webpack application in dist/, which can then be served from any web server or CloudStack management server (jetty).

To use CloudStack management server (jetty), you may copy the built UI to the webapp directory on the management server host. For example:

npm install
npm run build
cd dist
mkdir -p /usr/share/cloudstack-management/webapp/
cp -vr . /usr/share/cloudstack-management/webapp/
# Access UI at {management-server}:8080/client in browser

If the webapp directory is changed, please change the webapp.dir in the /etc/cloudstack/management/server.properties and restart the management server host.

To use a separate webserver, note that the API server is accessed through the path /client, which needs be forwarded to an actual CloudStack instance.

For example, a simple way to serve UI with nginx can be implemented with the following nginx configuration (to be put into /etc/nginx/conf.d/default.conf or similar):

server {
    listen       80;
    server_name  localhost;
    location / {
        # /src/ui/dist contains the built UI webpack
        root   /src/ui/dist;
        index  index.html;
    }
    location /client/ {
        # http://127.0.0.1:800 should be replaced your CloudStack management
        # server's actual URI
        proxy_pass   http://127.0.0.1:8000;
    }
}

Docker

A production-ready Docker container can also be built with the provided Dockerfile and build script.

Make sure Docker is installed, then run:

bash docker.sh

Change the example configuration in nginx/default.conf according to your needs.

Run UI:

docker run -ti --rm -p 8080:80 -v $(pwd)/nginx:/etc/nginx/conf.d:ro cloudstack-ui:latest

Packaging

The following is tested to work on any Ubuntu 18.04/20.04 base installation or docker container:

# Install nodejs (lts)
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt-get install -y nodejs debhelper rpm
# Install build tools
npm install -g @vue/cli webpack eslint
# Clone this repository and run package.sh
cd <cloned-repository>/packaging
bash -x package.sh

Documentation

Attributions

The UI uses the following:

History

The modern UI, originally called Primate, was created by Rohit Yadav over several weekends during late 2018 and early 2019. During ApacheCon CCCUS19, on 9th September 2019, Primate was introduced and demoed as part of the talk Modern UI for CloudStack (video). Primate was accepted by the Apache CloudStack project on 21 Oct 2019. The original repo was merged with the main apache/cloudstack repo on 20 Jan 2021.

License

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.